I want to use the useful ! call-to-shell feature of ipython in regular python script, i.e. I want the following code:
for i in range(5):
!echo {i}
To work in an in-house python module that can be imported (thus simply using ! will not work).
I've tried using subprocess but wasn't able to get the output of the shell run into the Jupyter notebook, where most of our scripts are running at.
So, what I'm looking for is using ! functionality as an IPython import some how..
Related
In ipython cells you can execute shell commands as follows:
ipython:
print("asdf")
!echo asdf
However, if you try to import this code from file,
asdf.py:
def asdf():
print("asdf")
!echo asdf
ipython:
from asdf import asdf
it results in an error.
!echo asdf
^
SyntaxError: invalid syntax
The use case for that is repetative massive scripts in google colab utilizing ffmpeg, wget, mount, e.t.c.
While you can use os.system or subprocess, they are not as interactive in providing real-time stdout.
Use case solution
If you want to reuse code that has ipython magic functions in it, simply use %run instead of the import.
ipython:
%run utils.ipynb
utils.ipynb:
def asdf():
print("asdf")
!echo asdf
Example above works fine.
Taken from this answer.
On reusing ipython code
Why the original question was an A-B problem of some kind? The reason is that you should not really drag ipython additional functionality to the pure python execution.
If you have a, let's say, google colab notebook with useful utils,
then just !wget it from the public link and %run it.
If you have a python script to execute, do a regular import on .py files.
Sure, there are ways to import .ipynb files to python code: Jupyter notebook's Notebook Loader, ipynb package, nbimporter package (but even the nbimporter's author says you should not really do that. Just convert your notebook to .py using VSCode or another tool).
So, if you use ipython research environment and features, just stick with it. Or switch to pure python environment with proper modules, testing, e.t.c.
Also, if you're struggling with doing wget:
file_id = "1xE8Db1zvQ7v-z13CO2qc3F6wJmtr3YHu" # can be extracted from public link
file_out_name = "utils.ipynb"
!wget "https://docs.google.com/uc?export=download&id=""$file_id" -O "$file_out_name"
However, on the problem of calling cmd, I see no way of executing shell commands interactively in python with one line. While subprocess can do that, it takes multiple lines to achieve.
I'm trying to execute some startup commands before starting or resetting the Python Interactive Window (IPython) within VSCode.
As I have been check in the documentation and in the parameters settings, I was able to identify the correct parameter:
Python > Data Science: Run Startup Commands
A series of Python instructions or iPython magic commands separated by '\n' that will be
executed when the interactive window loads. For instance, set this to
'%load_ext autoreload\n%autoreload 2' to automatically reload changes
made to imported files without having to restart the interactive
session.
However, when I insert my desirable python commands they not work as expected. As an example I tried to load the numpy package inserting 'import numpy as np', but when I start a new Python Interactive Window it outputs:
np is not defined
What I'm doing wrong? Thank you!
Remove the quotes '', just import numpy as np and don't forget to separate instructions with \n.
I'm thinking similar to atom with the hydrogen plugin?
When doing data science in Python I tend to get stuck in a "first create it in jupyter notebook, then re-create it in an actual python script so we can go into production". Using Atom i've been able to just create the code directly in a python script, but still have the great interactive features of jupyter.
I was really hoping that jupyterlab would be similar, but as far as i can tell, you only get the interactive features in the python notebooks and not in python scripts?
To write/save
%%writefile myfile.py
write/save cell contents into myfile.py (use -a to append). Another alias: %%file myfile.py
To run
%run myfile.py
run myfile.py and output results in the current cell
To load/import
%load myfile.py
load "import" myfile.py into the current cell
For more magic and help
%lsmagic
list all the other cool cell magic commands.
%COMMAND-NAME?
for help on how to use a certain command. i.e. %run?
You might like using the Spyder IDE for this. You can use it to edit and execute jupyter notebooks and python scripts:
https://docs.spyder-ide.org/current/plugins/notebook.html
I am fairly new to programming in python. I installed anaconda and am running iPython (the Jupyter qtconsole) v.4.3.0 and python v.3.6 on a Mac. Currently, I am trying to import a module with functions located in my home directory.
I have looked at stackoverflow and python documentation and found that it could be done with:
%run "Users/myUser/python_functions.py"
or
import python_functions
However, when I try both of these approaches, I get prompted to overwrite the file that I am running or importing:
File `python_functions.py` exists. Overwrite (y/[N])?
This is changing the previous file and not getting the functions I want to be imported.
What may explain this, and what can I do to import my module?
this is wrong but leaving it up for shame
import on ubuntu (and I'm guessing many other unix-like OSs including Mac) is a utility that saves any visible window on an X server and outputs it as an image file. You can capture a single window, the entire screen, or any rectangular portion of the screen.
My guess if you are running the import command in your console, and it's about to take a screenshot and save it over an existing file - python_functions
Before you the use the python import command, start a python interpreter:
$ python
>>>import yourfile
edit: on re-reading your question, I'm not so sure about my guess anymore, but leaving it up until you tell me I'm wrong :)
Running Jupyter qtconsole as an interpreter is likely causing the problem in this scenario. Instead using a IDE or command line interpreter will resolve it .
Since anaconda was installed, trying it with the IDE Spyder executes the code just fine without the overwrite prompt. It works on others (e.g PyCharm, Rodeo, etc.) as well.
i want to interactively debug code that is deeply hidden in some class.
I know that it is possible open the ipdb using
from IPython.core.debugger import Pdb
Pdb().set_trace()
This work also in ipython (jupyter) notebook.
Another possibility that I find very convenient when I am working in the terminal is to open a shell at a specific point using
from IPython import embed
embed()
and then use ipython directly to interact with variables etc.
However, if I use this in notebook, I get a ipython shell embedded into the notebook, and this shell does not even work well (e.g., if I use print, the kernel blocks).
What I would find most convenient is to interrupt the kernel execution at some predefined point in the code and keep all current variables, so that I can use another cell for debugging and the possibility to, e.g., develop and test new code with the current variables.
Is there some way to achieve this?