I want to execute one python script in Jupyter, but I don't want to use the web browser (IPython Interactive terminal), I want to run a single command in the Linux terminal to load & run the python script, so that I can get the output from Jupyter.
I tried to run jupyter notebook %run <my_script.py>, but it seems jupyter can't recognize %run variable.
Is it possible to do that?
The jupyter_client package has a jupyter run command since v5.0, so you can just do:
jupyter run my_script.py
You can use the jupyter console -i command to run an interactive jupyter session in your terminal. From there you can run import <my_script.py>. Do note that this is not the intended use case of either jupyter or the notebook environment. You should run scripts using your normal python interpreter instead.
You can run this command to run an interactive jupyter session in your terminal.
jupyter notebook
Related
I have some code in a .ipynb file and got it to the point where I don't really need the "interactive" feature of IPython Notebook. I would like to just run it straight from a Mac Terminal Command Line.
Basically, if this were just a .py file, I believe I could just do python filename.py from the command line. Is there something similar for a .ipynb file?
nbconvert allows you to run notebooks with the --execute flag:
jupyter nbconvert --execute <notebook>
If you want to run a notebook and produce a new notebook, you can add --to notebook:
jupyter nbconvert --execute --to notebook <notebook>
Or if you want to replace the existing notebook with the new output:
jupyter nbconvert --execute --to notebook --inplace <notebook>
Since that's a really long command, you can use an alias:
alias nbx="jupyter nbconvert --execute --to notebook"
nbx [--inplace] <notebook>
From the command line you can convert a notebook to python with this command:
jupyter nbconvert --to python nb.ipynb
https://github.com/jupyter/nbconvert
You may have to install the python mistune package:
sudo pip install -U mistune
In your Terminal run ipython:
ipython
then locate your script and put there:
%run your_script.ipynb
You can export all your code from .ipynb and save it as a .py script. Then you can run the script in your terminal.
Hope it helps.
Using ipython:
ipython --TerminalIPythonApp.file_to_run=<notebook>.ipynb
Normally, I would prefer this option as it is really self-describing. If you prefer to use less characters, use:
ipython -c "%run <notebook>.ipynb"
which is basically what Keto already suggested (unfortunately a little bit hidden) as a comment.
In my case, the command that best suited me was:
jupyter nbconvert --execute --clear-output <notebook>.ipynb
Why? This command does not create extra files (just like a .py file) and the output of the cells is overwritten everytime the notebook is executed.
If you run:
jupyter nbconvert --help
--clear-output
Clear output of current file and save in place, overwriting the existing notebook.
For new version instead of:
ipython nbconvert --to python <YourNotebook>.ipynb
You can use jupyter instend of ipython:
jupyter nbconvert --to python <YourNotebook>.ipynb
Update with quoted comment by author for better visibility:
Author's note "This project started before Jupyter's execute API, which is now the recommended way to run notebooks from the command-line. Consider runipy deprecated and unmaintained." – Sebastian Palma
Install runipy library that allows running your code on terminal
pip install runipy
After just compiler your code:
runipy <YourNotebookName>.ipynb
You can try cronjob as well. All information is here
I had the same problem and I found papermill. The advantages against the others solutions is that you can see the results while the notebook is running. I find this feature interesting when the notebook takes very long. It is very easy to use:
pip install papermill
papermill notebook.ipynb output.ipynb
It has also, other handy options as saving the output file to Amazon S3, Google Cloud, etc. See the page for more information.
You can also use the boar package to run your notebook within a python code.
from boar.running import run_notebook
outputs = run_notebook("nb.ipynb")
If you update your notebook, you won't have to convert it again to a python file.
More information at:
https://github.com/alexandreCameron/boar/blob/master/USAGE.md
There is now a jupyter run subcommand that will execute a notebook.
jupyter run notebook.ipynb
More on this command can be found in the Jupyter documentation.
From the terminal run
jupyter nbconvert --execute --to notebook --inplace --allow-errors --ExecutePreprocessor.timeout=-1 my_nb.ipynb
The default timeout is 30 seconds. -1 removes the restriction.
If you wish to save the output notebook to a new notebook you can use the flag --output my_new_nb.ipynb
You can also use jupytext https://jupytext.readthedocs.io/en/latest/index.html.
This allows you to pair your notebook. So for each ipynb file you have a .py file as well with some comments. The .py file can be executed as usual.
You can enjoy benefits of two worlds with the cost of one extra file though.
Oh, and by the way if you are using version control you can only commit .py files with a nice diff instead of the ugly .ipynb diffs.
(The syntax in the .py files is similar to Databricks notebooks iy you are familiar with them...)
In a batch file paste the below. Use ipython to run .ipynb files.
#echo on
call "C:\ProgramData\Anaconda3\Scripts\activate.bat"
ipython "path\test.ipynb"
pause
I've made some research on this topic and wrote an article on 4 ways to run Jupyter Notebook in command line below is summary of my findings.
1. Use nbconvert
The nbconvert package is already installed in Jupyter Notebook. It can be used to execute notebook. Additionally it has many features:
can export notebook to PDF or HTML,
can hide code in output notebook,
can execute notebook even with errors in cells.
Example notebook execution:
jupyter nbconvert --execute --to notebook --allow-errors your-notebook.ipynb
The above command will output your-notebook.nbconvert.ipynb file and will execute all cells even with errors.
2. Use papermill
The papermill allows you to parametrize notebook. You can define variables as parameters (with cell tags).
Example command:
papermill -p name Piotrek your-notebook.ipynb output-notebook.ipynb
Example notebook with injected parameters:
3. Manually download notebook as .py script
There is an option to manually download notebook as .py script:
After download you can add execution rights to the file and run it as a command line script.
4. Use jupytext
The jupytext package allows you to synchronize .ipynb file with .py file. You don't need to manually convert notebook to script.
Would someone please suggest a way to execute Windows PowerShell commands (i.e., commands more similar to Unix commands than Command-Prompt commands) in a Jupyter notebook? Please note that I am a layperson.
So far, I have added "C:\ProgramData\Anaconda3\Scripts" to my path system variable, which allows me to run "jupyter-notebook" from either Command Prompt or Windows PowerShell. I guess a shell looks in all folders in my path system variable for an application to run named "juypter-notebook". I guessed that running Jupyter Notebook from Windows PowerShell would allow my Jupyter-Notebook Chrome application to understand PowerShell commands, but this doesn't seem to be the case.
You can execute a PowerShell command from a Jupyter Notebook cell on a Windows machine with:
!Powershell.exe -Command "<PowerShell command>"
For example, if you wanted to print the first 10 lines of the file filename.txt, you could write:
!Powershell.exe -Command "type filename.txt -Head 10"
Source
I am trying to run my Jupyter program using shell script. However I am unsure of how to do so. I want to run my program on Jenkins, which is a platform that will require my code to be run in shell script in order to test it.
This appears to be covered in the docs. Example:
jupyter nbconvert --to notebook --execute mynotebook.ipynb
I have some code in a .ipynb file and got it to the point where I don't really need the "interactive" feature of IPython Notebook. I would like to just run it straight from a Mac Terminal Command Line.
Basically, if this were just a .py file, I believe I could just do python filename.py from the command line. Is there something similar for a .ipynb file?
nbconvert allows you to run notebooks with the --execute flag:
jupyter nbconvert --execute <notebook>
If you want to run a notebook and produce a new notebook, you can add --to notebook:
jupyter nbconvert --execute --to notebook <notebook>
Or if you want to replace the existing notebook with the new output:
jupyter nbconvert --execute --to notebook --inplace <notebook>
Since that's a really long command, you can use an alias:
alias nbx="jupyter nbconvert --execute --to notebook"
nbx [--inplace] <notebook>
From the command line you can convert a notebook to python with this command:
jupyter nbconvert --to python nb.ipynb
https://github.com/jupyter/nbconvert
You may have to install the python mistune package:
sudo pip install -U mistune
In your Terminal run ipython:
ipython
then locate your script and put there:
%run your_script.ipynb
You can export all your code from .ipynb and save it as a .py script. Then you can run the script in your terminal.
Hope it helps.
Using ipython:
ipython --TerminalIPythonApp.file_to_run=<notebook>.ipynb
Normally, I would prefer this option as it is really self-describing. If you prefer to use less characters, use:
ipython -c "%run <notebook>.ipynb"
which is basically what Keto already suggested (unfortunately a little bit hidden) as a comment.
In my case, the command that best suited me was:
jupyter nbconvert --execute --clear-output <notebook>.ipynb
Why? This command does not create extra files (just like a .py file) and the output of the cells is overwritten everytime the notebook is executed.
If you run:
jupyter nbconvert --help
--clear-output
Clear output of current file and save in place, overwriting the existing notebook.
For new version instead of:
ipython nbconvert --to python <YourNotebook>.ipynb
You can use jupyter instend of ipython:
jupyter nbconvert --to python <YourNotebook>.ipynb
Update with quoted comment by author for better visibility:
Author's note "This project started before Jupyter's execute API, which is now the recommended way to run notebooks from the command-line. Consider runipy deprecated and unmaintained." – Sebastian Palma
Install runipy library that allows running your code on terminal
pip install runipy
After just compiler your code:
runipy <YourNotebookName>.ipynb
You can try cronjob as well. All information is here
I had the same problem and I found papermill. The advantages against the others solutions is that you can see the results while the notebook is running. I find this feature interesting when the notebook takes very long. It is very easy to use:
pip install papermill
papermill notebook.ipynb output.ipynb
It has also, other handy options as saving the output file to Amazon S3, Google Cloud, etc. See the page for more information.
You can also use the boar package to run your notebook within a python code.
from boar.running import run_notebook
outputs = run_notebook("nb.ipynb")
If you update your notebook, you won't have to convert it again to a python file.
More information at:
https://github.com/alexandreCameron/boar/blob/master/USAGE.md
There is now a jupyter run subcommand that will execute a notebook.
jupyter run notebook.ipynb
More on this command can be found in the Jupyter documentation.
From the terminal run
jupyter nbconvert --execute --to notebook --inplace --allow-errors --ExecutePreprocessor.timeout=-1 my_nb.ipynb
The default timeout is 30 seconds. -1 removes the restriction.
If you wish to save the output notebook to a new notebook you can use the flag --output my_new_nb.ipynb
You can also use jupytext https://jupytext.readthedocs.io/en/latest/index.html.
This allows you to pair your notebook. So for each ipynb file you have a .py file as well with some comments. The .py file can be executed as usual.
You can enjoy benefits of two worlds with the cost of one extra file though.
Oh, and by the way if you are using version control you can only commit .py files with a nice diff instead of the ugly .ipynb diffs.
(The syntax in the .py files is similar to Databricks notebooks iy you are familiar with them...)
In a batch file paste the below. Use ipython to run .ipynb files.
#echo on
call "C:\ProgramData\Anaconda3\Scripts\activate.bat"
ipython "path\test.ipynb"
pause
I've made some research on this topic and wrote an article on 4 ways to run Jupyter Notebook in command line below is summary of my findings.
1. Use nbconvert
The nbconvert package is already installed in Jupyter Notebook. It can be used to execute notebook. Additionally it has many features:
can export notebook to PDF or HTML,
can hide code in output notebook,
can execute notebook even with errors in cells.
Example notebook execution:
jupyter nbconvert --execute --to notebook --allow-errors your-notebook.ipynb
The above command will output your-notebook.nbconvert.ipynb file and will execute all cells even with errors.
2. Use papermill
The papermill allows you to parametrize notebook. You can define variables as parameters (with cell tags).
Example command:
papermill -p name Piotrek your-notebook.ipynb output-notebook.ipynb
Example notebook with injected parameters:
3. Manually download notebook as .py script
There is an option to manually download notebook as .py script:
After download you can add execution rights to the file and run it as a command line script.
4. Use jupytext
The jupytext package allows you to synchronize .ipynb file with .py file. You don't need to manually convert notebook to script.
I am not very familiar with python/ipython but somebody was asking me whether it is possible to start an ipython notebook with a specific python file. It then could be used for debugging.
another software then would create a .py-file in the temp folder and would call an ipython notebook with this file.
Is it possible or does it make sense at all?
Since the question is quite broad and asking for recommendations, here are my suggestions:
cross-platform nbopen, which opens ipynb using command-line or optional explorer integration:
https://github.com/takluyver/nbopen
Please note that I have one open ticket for complete Windows explorer integration:
https://github.com/takluyver/nbopen/issues/12
[copied from github page]
Installation:
pip install nbopen
Usage:
nbopen AwesomeNotebook.ipynb
run ipynb without launching the browser interface, with many useful options:
https://github.com/paulgb/runipy
[copied from github page]
Installation:
$ pip install runipy
To run a .ipynb file as a script, run:
$ runipy MyNotebook.ipynb
To save the output of each cell back to the notebook file, run:
$ runipy -o MyNotebook.ipynb
To save the notebook output as a new notebook, run:
$ runipy MyNotebook.ipynb OutputNotebook.ipynb
To run a .ipynb file and generate an HTML report, run:
$ runipy MyNotebook.ipynb --html report.html
If you're talking about launching an iPython notebook server via Python, I use this:
#!/usr/bin/env python
from IPython.terminal.ipapp import launch_new_instance
from IPython.lib import passwd
from socket import gethostname
import warnings
warnings.filterwarnings("ignore", module = "zmq.*")
sys.argv.append("notebook")
sys.argv.append("--IPKernelApp.pylab='inline'")
sys.argv.append("--NotebookApp.ip=" + gethostname())
sys.argv.append("--NotebookApp.open_browser=False")
sys.argv.append("--NotebookApp.password=" + passwd())
launch_new_instance()
Obviously you can change the arguments if you so desire.
At my work we have one use case that does what you're saying--automatically generates a python file, then loads a new ipython server for the user to access it. However, it's a pretty special use case--for normal debugging, I would recommend just starting in iPython and not making your *.py file until the bugs are gone.
OR
If you're talking about actually automatically navigating to the page that corresponds to a python file made available by an ipython notebook server, then (1) make sure you're using ipython 2, and (2) figure out what you're desired url is (it should be deterministic) and (3) use the webbrowser module to navigate to that url.
import subprocess, os
def executeJupyter():
env_dir = "../main_env_dir/"
os.chdir(env_dir)
# source jupyter_env/bin/activate
env_activate = "jupyter_env/bin/activate_this.py"
activate_env = exec(open(env_activate).read(), {'__file__': env_activate})
# Open jupyter notebook as a subprocess
openJupyter = "jupyter notebook"
subprocess.Popen(openJupyter, shell=True)
executeJupyter()
Make sure you change the env_dir (directory where you have the env of your jupyter notebook) and the env_activate to your own.
To start a ipython notebook with a certain notebook directory, use the --notebook-dir command line option:
ipython notebook --notebook-dir=/Users/harold/temp/