Develop Reference

Asking for overwrite while trying to run script in ipython

I am fairly new to programming in python. I installed anaconda and am running iPython (the Jupyter qtconsole) v.4.3.0 and python v.3.6 on a Mac. Currently, I am trying to import a module with functions located in my home directory.
I have looked at stackoverflow and python documentation and found that it could be done with:
%run "Users/myUser/python_functions.py"
or
import python_functions
However, when I try both of these approaches, I get prompted to overwrite the file that I am running or importing:
File `python_functions.py` exists. Overwrite (y/[N])?
This is changing the previous file and not getting the functions I want to be imported.
What may explain this, and what can I do to import my module?

this is wrong but leaving it up for shame
import on ubuntu (and I'm guessing many other unix-like OSs including Mac) is a utility that saves any visible window on an X server and outputs it as an image file. You can capture a single window, the entire screen, or any rectangular portion of the screen.
My guess if you are running the import command in your console, and it's about to take a screenshot and save it over an existing file - python_functions
Before you the use the python import command, start a python interpreter:
$ python
>>>import yourfile
edit: on re-reading your question, I'm not so sure about my guess anymore, but leaving it up until you tell me I'm wrong :)

Running Jupyter qtconsole as an interpreter is likely causing the problem in this scenario. Instead using a IDE or command line interpreter will resolve it .
Since anaconda was installed, trying it with the IDE Spyder executes the code just fine without the overwrite prompt. It works on others (e.g PyCharm, Rodeo, etc.) as well.

Ipython - %edit <functionname > throws error when notepad is opened

When I try to edit a function ( printval in my example ) defined in Enthought canopy Ipython using the command,
%edit printval .
I get the following error message on a pop up with a notepad opened in the background.
The filename,directory name ,or volume label syntax is incorrect.
The notepad is blank after I click the pop up message. This error is happening in windows Ipython.
Also I invoked Ipython using the following command
ipython qtconsole --pylab=inline --colors=Linux
Can anyone help to find what can be the problem .

Works for me. Are you sure that the function name printval is already in your ipython namespace when you give the command %edit printval? You can check this by seeing whether printval? refers you to your function as expected, or gives an error. If printval isn't in your ipython namespace, then ipython has no way to know what you mean by printval.
For example, if you have already done from mymodule import printval,
then %edit printval should work.
If this doesn't address the issue, please provide a minimal example including all steps to reproduce the problem.
UPDATE after this followup information from user: " File: c:\users\tester\<ipython-input-5-cc5d372a3507>"
This shows that the function was not defined in a file as typically done, but rather interactively at the ipython prompt. This complicates the answer, thus:
When a function is defined in a (non-temporary) file, then %edit functionname works as desired in all versions of ipython - it opens that file for editing.
However when the function is defined interactively at an ipython prompt, as in this user's case, then the actual behavior depends on the version of ipython and which ipython/jupyter interface you are using. What one would like is for a temporary file to be opened, the function definition be copied into that file for editing, and then after the editor closes, for the function definition to be re-executed (i.e. for the function to be re-defined) based on those edits.
Indeed, this desired behavior is implemented in simple text-mode ipython terminal (e.g. if you type ipython at a Canopy Command Prompt).
However, this desired behavior has never been implemented in ipython/jupyter QtConsole (e.g. if you type jupyter qtconsole at a Canopy Command Prompt), and as far as I can tell, it is not currently planned by the jupyter team, most of whose effort seems to be focused on the jupyter notebook and jupyter lab projects.
Because Canopy is a GUI application, Canopy's ipython prompt is a QtConsole, not an ipython terminal prompt. Therefore Canopy does not implement the behavior that you want, and will not in the foreseeable future.
The (minor) good news is that in the most recent version of QtConsole, the error message is more explicit. It says "InteractivelyDefined" which at least gives a hint that the function that you were editing was defined interactively and is therefore not directly editable.
The workaround, of course, is to manually copy-paste the function definition source code into the Canopy editor (which automatically strips out the ipython prompts from the clipboard, making this very simple to do), edit the function there, and run that file.

Eclipse - Python and Ipython?

I'm following along a book about the use of python for finance. And the author uses this line:
%matplotlib inline
Which does not compile.
So far I've figured out that it's Ipython that is being used, and that it's another interpreter than the python.
I'm using Eclipse with pyDev, and I'm looking for a solution so that I may continoue doing so.
I installed ipython through pip with pip install ipython and top of my .py file write
import IPython.ipapi
ip = IPython.ipapi.get()
But I'm getting
Traceback (most recent call last):
File "C:\Users\Rasmus\workspace\PythonFinance\src\FinanceExamples.py", line 5, in <module>
import IPython.ipapi
ImportError: No module named 'IPython.ipapi'
the ip = IPython.ipapi.get() does not give any errors. But if I comment out ip = IPython.ipapi.get() then it will.
Is it not possible to run Ipython with a python interpreter? Or have I missunderstood the entire relationship between them both.
I use python 3.5

Open Eclipse/PyDev and go to Window-->Preferences-->PyDev-->Interactive Console.
In the Initial Interpreter Commands filed add this 2 commands:
import IPython; IPython.embed()
After clicking in Apply and Close, press CTRL+ALT+ENTER (or select the code that you want to execute +F2) and IPython console will run. However, it seems not work very well. You will face on the message below if you try to run %matplotlib inline command:
IPython.core.error.UsageError: Invalid GUI request 'inline', valid ones are:dict_keys(['qt4', 'osx', 'pyglet', 'none', 'tk', 'qt', 'gtk3', 'glut', 'wx', 'gtk'])

You can use IPython interpreter in Eclipse/PyDev interactively - in a PyDev Console. (You can start it in the Console Menu. More: http://www.pydev.org/manual_adv_interactive_console.html.)
You don't need to add any extra lines to your file. All magic commands are usually executed in the console. (I think extra code is needed only if you want to run some magics in a script, not interactively). You can run your code line by line copying it from the editor or - faster- highlighting it and pressing F2.
The command %matplotlib inline probably wouldn't work in PyDev Console. It's designed rather for IPython Notebook. Nevertheless, try running it there and you should get some error with a hint what to use instead "inline", for example:
IPython.core.error.UsageError: Invalid GUI request 'inline', valid ones are:dict_keys(['qt4', 'osx', 'pyglet', 'none', 'tk', 'qt', 'gtk3', 'glut', 'wx', 'gtk'])
Also, you get some help displayed when you execute %matplotlib? (with the question mark).

tried with:
import matplotlib.pyplot as plt
and run
plt.show()
Looks like that gives the same results.

Python code snippet that runs always when a ipython notebook closes

I would like to add a python code to an ipython notebook that will run every time I close the ipython tab. I tried to see if I can set a cell to do it but I had no luck.
Is this possible either using an ipython API or some other hook mechanism?

One option could be using the atexit python module to register an exit handler. This would work if your page in the IPython notebook is actually a python process.

Keep Jupyter notebook running after closing browser tab

I use Jupyter Notebook to run a series of experiments that take some time.
Certain cells take way too much time to execute so it's normal that I'd like to close the browser tab and come back later. But when I do the kernel interrupts running.
I guess there is a workaround for this but I can't find it

The simplest workaround to this seems to be the built-in cell magic %%capture:
%%capture output
# Time-consuming code here
Save, close tab, come back later. The output is now stored in the output variable:
output.show()
This will show all interim print results as well as the plain or rich output cell.

TL;DR:
Code doesn't stop on tab closes, but the output can no longer find the current browser session and loses data on how it's supposed to be displayed, causing it to throw out all new output received until the code finishes that was running when the tab closed.
Long Version:
Unfortunately, this isn't implemented (Nov 24th). If there's a workaround, I can't find it either. (Still looking, will update with news.) There is a workaround that saves output then reprints it, but won't work if code is still running in that notebook. An alternative would be to have a second notebook that you can get the output in.
I also need this functionality, and for the same reason. The kernel doesn't shut down or interrupt on tab closes. And the code doesn't stop running when you close a tab. The warning given is exactly correct, "The kernel is busy, outputs may be lost."
Running
import time
a = 0
while a < 100:
a+=1
print(a)
time.sleep(1)
in one box, then closing the tab, opening it up again, and then running
print(a)
from another box will cause it to hang until the 100 seconds have finished and the code completes, then it will print 100.
When a tab is closed, when you return, the python process will be in the same state you left it (when the last save completed). That was their intended behavior, and what they should have been more clear about in their documentation. The output from the run code actually gets sent to the browser upon reopening it, (lost the reference that explains this,) so hacks like the one in this comment will work as it can receive those and just throw them into some cell.
Output is kind of only saved in an accessible way through the endpoint connection. They've been working on this for a while (before Jupyter), although I cannot find the current bug in the Jupyter repository (this one references it, but is not it).
The only general workaround seems to be finding a computer you can always leave on, and leaving that on the page while it runs, then remote in or rely on autosave to be able to access it elsewhere. This is a bad way to do it, but unfortunately, the way I have to for now.
Related questions:
Closed IPython Notebook that was running code
Confirms that output will not be updated, but does not mention the interrupt functionality.
IPython Notebook - Keep printing to notebook output after closing browser
Offers a workaround in a link. Referenced above

First, install
runipy
pip install runipy
And now run your notebook in the background with the below command:
nohup runipy YourNotebook.ipynb OutputNotebook.ipynb >> notebook.log &
now the output file will be saved and also you can see the logs while running with:
tail -f notebook.log

I am struggling with this issue as well for some time now.
My workaround was to write all my logs to a file, so that when my browser closes (indeed when a lot of logs come through browser it hangs up too) I can see the kernel job process by opening the log file (the log file can be open using Jupyter too).
#!/usr/bin/python
import time
import datetime
import logging
logger = logging.getLogger()
def setup_file_logger(log_file):
hdlr = logging.FileHandler(log_file)
formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
hdlr.setFormatter(formatter)
logger.addHandler(hdlr)
logger.setLevel(logging.INFO)
def log(message):
#outputs to Jupyter console
print('{} {}'.format(datetime.datetime.now(), message))
#outputs to file
logger.info(message)
setup_file_logger('out.log')
for i in range(10000):
log('Doing hard work here i=' + str(i))
log('Taking a nap now...')
time.sleep(1000)

With JupyterLab:
This is not a problem if you are using JupyterLab (with current release v3.x.x).
To be more specific, not a problem means that, after we close the tab/browser, the notebook's kernel is kept running (so long as the jupyter server/your terminal is not closed). But the printing output of the cell (if there is any) is interrupted.
So, when we reopen the notebook, variables and etc. are all kept and updated, except the interrupted printing output.
If you care about the printing info in this case, you could try to logging it to a file. OR try using Jupyter's execute API (see below).
With Jupyter Notebook:
If you are still sticking with legacy (e.g. version 5.x/6.x) Jupyter Notebook, well, it is still not possible in the past (i.e prior to 2022).
BUT, with the planned new Notebook v7 release, by reusing the the JupyterLab codebase, this problem will also be solved in the new Jupyter Notebook.
So, try using JupyterLab or wait and updating to Notebook v7:
$ jupyter lab --version
$ 3.4.4
$ # OR waite and update the notebook, untill
$ # make sure the installed version of notebook is v7
$ jupyter notebook --version
$ 6.4.12
With Jupyter's execute API:
Other workaround is by using Jupyter's execute API:
$ jupyter nbconvert --to notebook --execute mynotebook.ipynb
This is like running the notebook as a .py file, i.e. from the command line, not a web browser UI mode.
After its execution, a new file named mynotebook.nbconvert.ipynb will be produced, and all printing output will be kept in it, but all variables will be lost. What we could do is pickling the variables that we care about.
And I don't think using runipy is still a good choice, since it's deprecated and unmaintained (after Jupyter's execute API).
ref:
Q: is it possible to make a jupyter notebook run even if the page is closed?
A: This is being solved in JupyterLab and will be solved in the future Notebook v7 release.

If you've set all cells to run and want to periodically check what's being printed, the following code would be a better option than %%capture. You can always open up the log file while kernel is busy.
import sys
sys.stdout = open("my_log.txt", "a")

I've constructed this awhile ago using jupyter nbconvert, essentially running a notebook in the background without any UI:
nohup jupyter nbconvert --ExecutePreprocessor.timeout=-1 --CodeFoldingPreprocessor.remove_folded_code=False --ExecutePreprocessor.allow_errors=True --ExecutePreprocessor.kernel_name=python3 --execute --to notebook --inplace ~/mynotebook.ipynb > ~/stdout.log 2> ~/stderr.log &
timeout=-1 no time out
remove_folded_code=False if you have Codefolding extension enabled
allow_errors=True ignore errored cells and continue running the notebook to the end
kernel_name if you have multiple kernels, check with jupyter kernelspec list