For teaching purposes I want an IPython notebook that displays (as output from a cell) the function source code, but I want to be able to reference this in multiple notebooks. Hence I would like to display the function code, in a similar way to using the %psource magic, but appropriately syntax highlighted.
This is a similar question to this question, but I want to be able to apply it to a single function within a file, rather than to the complete file at once.
Using the suggestion from the previous question I hacked a short code that works in simple cases:
def print_source(module, function):
"""For use inside an IPython notebook: given a module and a function, print the source code."""
from inspect import getmembers, isfunction, getsource
from pygments import highlight
from pygments.lexers import PythonLexer
from pygments.formatters import HtmlFormatter
from IPython.core.display import HTML
internal_module = __import__(module)
internal_functions = dict(getmembers(internal_module, isfunction))
return HTML(highlight(getsource(internal_functions[function]), PythonLexer(), HtmlFormatter(full=True)))
Two questions:
This gist suggests that showing the whole function could be done by defining appropriate cell magic. Is it possible to define an appropriate cell magic to just show a single function, as above?
Is there a way of doing this without importing the entire module, or a more robust way of doing this?
1) Magics are just simple function not difficult to define, you could have a look here Customizing IPython - Config.ipynb if I remember correctly. still I'm not sure it is worth definig a magic in your case.
2) Most of the time, no. You have to import the module as we need live code to know where it is defined.
In general, finding the code of a function is not always super easy. On python 3 you can always access the code object, but most of the time, as soon as you have things like decorated function, or dynamically generated function, it becomes difficult. I suppose you could also inspire from psource/pinfo2 and have them return info instead of paging it.
Related
SciPy's website has a tutorial that references a python function source that lists the source code of functions written in python, but I cannot use it in python or find documentation for it online. The reference is at the bottom of this page. I see that the inspect module has similar functions but I'm still curious as to what function in what module they are referring to.
np.source is a utility function in numpy. This can be called on any numpy/scipy or Python function or class, though compiled built-ins won't show anything.
The scipy and numpy API docs also have a [source] link that takes you to a source file.
I think np.source is relatively new, but as a long time ipython user I've been getting the same information with its ?? magic.
np.source(np.source) gives me
def source(object, output=sys.stdout):
"""
...
"""
# Local import to speed up numpy's import time.
import inspect
try:
print("In file: %s\n" % inspect.getsourcefile(object), file=output)
print(inspect.getsource(object), file=output)
except:
print("Not available for this object.", file=output)
In [425]: np.source?? shows the same thing but with some color coding.
I am working on a python project with Canopy, using my own library, which I modify from time to time to change or add functions inside.
At the beginning of myfile.py I have from my_library import * but if I change a function in this library and compute again myfile.py it keep using the previous version of my function.
I tried the reload function :
import my_library
reload(my_library)
from other_python_file import *
from my_library import *
and it uses my recently changed library.
But if it is :
import my_library
reload(my_library)
from my_library import *
from other_python_file import *
It gives me the result due to the version loaded the first time I launched myfile.py.
Why is there a different outcome inverting the 3rd and 4th line ?
Without seeing the source code, it's hard to be certain. (For future reference, it is most useful to post a minimal example, which I suspect would be about 10 lines of code in this case.)
However from your description of the problem, my guess is that your other_python_file also imports my_library. So when you do from other_python_file import *, you are also importing everything that it has already imported from my_library, which in your second example, will override the imports directly from my_library (Since you didn't reload other_python_file, it will still be using the previous version of my_library.)
This is one out of approximately a zillionteen reasons why you should almost never use the form from xxx import * except on the fly in interactive mode (and even there it can be dangerous but can be worth the tradeoff for the convenience). In a python source file, there's no comparable justification for this practice. See the final point in the Imports section of PEP-8.
In Python 2.7, I'm getting
'module' has no attribute
, and/or
'name' is not defined
errors when I try to split up a large python file.
(I have already read similar posts and the Python modules documentation)
Say you have a python file that is structured like this:
<imports>
<50 global variables defined>
<100 lengthy functions that each use most or all of the globals
defined above, and also call each other>
<main() that calls some of the functions and uses the globals>
So I can easily categorize groups of functions together, create a python file for each group, and put them there. The problem is whenever I try to call any of them from the main python file, I get the errors listed above. I think the problem is related to circular dependencies. Since all of the functions rely on the globals, and each other, they are circularly dependent.
If I have main_file.py, group_of_functions_1.py, and group_of_functions_2.py,
main_file.py will have:
import group_of_functions_1.py
import group_of_functions_2.py
and group_of_functions_1.py will have
import main_file.py
import group_of_functions_2.py
and group_of_functions_2.py will have
import main_file.py
import group_of_functions_1.py
Regardless of whether I use "import package_x" or "from package_x import *" the problem remains.
If I take the route of getting rid of the globals, then most of the functions will have 50 parameters they will be passing around which then also need to be returned
What is the right way to clean this up?
(I have already read similar posts and the Python modules documentation)
One of the sources of your errors is likely the following:
import group_of_functions_1.py
import group_of_functions_2.py
When importing, you don't add .py to the end of the module name. Do this instead:
import group_of_functions_1
import group_of_functions_2
I have a vaguely defined function of a class Graph of a module I call gt (it is graph-tool). so i declare g = gt.graph() then want to use g.degree_property_map but do not know how. Therefore I want to see where in code g.degree_property_map or in this case just the function, is defined. How can I find that? I'm working on command line on a vm.
Thanks
For reference the library in question is graph-tool - http://projects.skewed.de/graph-tool/
Also I am currently importing it using from graph_tool.all import * . that is of course somewhat of a problem.
You could use inspect.getsource(gt.Graph.degree_property_map). (You have to import inspect.)
Of course, what you pass into getsource() will change depending on how you imported Graph. So if you used from graphtools.all import *, you'd just need to use inspect.getsource(Graph.degree_property_map).
If you open interactive python (type python and hit ENTER on the command line), you should be able to run the command help(<graph's module name>), then, under the FILE section of the help documentation that is generated, you should see the absolute path to the code you are interested in.
For example, I just ran:
import numpy
help(numpy)
# Returned documentation containing:
# FILE
# /usr/lib/python2.7/dist-packages/numpy/__init__.py
Also,
import my_module # A module I just created that contains the "Line" class
help(my_module)
# Returned documentation containing:
# FILE
# /home/<my user name>/Programming/Python/my_module.py
If it is a normal function (not a builtin, ufunc, etc) you can try using the func_code attribute
For example:
>>> inspect.iscode
<function iscode at 0x02EAEF30>
>>> inspect.iscode.func_code
<code object iscode at 02EB2B60, file "C:\Python27\lib\inspect.py", line 209>
Never mind I just did help(graph_tool) and manually jumped through code. Thanks for the help though!
This is a very basic question - but I haven't been able to find an answer by searching online.
I am using python to control ArcGIS, and I have a simple python script, that calls some pre-written code.
However, when I make a change to the pre-written code, it does not appear to result in any change. I import this module, and have tried refreshing it, but nothing happens.
I've even moved the file it calls to another location, and the script still works fine. One thing I did yesterday was I added the folder where all my python files are to the sys path (using sys.append('path') ), and I wonder if that made a difference.
Thanks in advance, and sorry for the sloppy terminology.
It's unclear what you mean with "refresh", but the normal behavior of Python is that you need to restart the software for it to take a new look on a Python module and reread it.
If your changes isn't taken care of even after restart, then this is due to one of two errors:
The timestamp on the pyc-file is incorrect and some time in the future.
You are actually editing the wrong file.
You can with reload re-read a file even without restarting the software with the reload() command. Note that any variable pointing to anything in the module will need to get reimported after the reload. Something like this:
import themodule
from themodule import AClass
reload(themodule)
from themodule import AClass
One way to do this is to call reload.
Example: Here is the contents of foo.py:
def bar():
return 1
In an interactive session, I can do:
>>> import foo
>>> foo.bar()
1
Then in another window, I can change foo.py to:
def bar():
return "Hello"
Back in the interactive session, calling foo.bar() still returns 1, until I do:
>>> reload(foo)
<module 'foo' from 'foo.py'>
>>> foo.bar()
'Hello'
Calling reload is one way to ensure that your module is up-to-date even if the file on disk has changed. It's not necessarily the most efficient (you might be better off checking the last modification time on the file or using something like pyinotify before you reload), but it's certainly quick to implement.
One reason that Python doesn't read from the source module every time is that loading a module is (relatively) expensive -- what if you had a 300kb module and you were just using a single constant from the file? Python loads a module once and keeps it in memory, until you reload it.
If you are running in an IPython shell, then there are some magic commands that exist.
The IPython docs cover this feature called the autoreload extension.
Originally, I found this solution from Jonathan March's blog posting on this very subject (see point 3 from that link).
Basically all you have to do is the following, and changes you make are reflected automatically after you save:
In [1]: %load_ext autoreload
In [2]: %autoreload 2
In [3]: Import MODULE
In [4]: my_class = Module.class()
my_class.printham()
Out[4]: ham
In [5]: #make changes to printham and save
In [6]: my_class.printham()
Out[6]: hamlet
I used the following when importing all objects from within a module to ensure web2py was using my current code:
import buttons
import table
reload(buttons)
reload(table)
from buttons import *
from table import *
I'm not really sure that is what you mean, so don't hesitate to correct me. You are importing a module - let's call it mymodule.py - in your program, but when you change its contents, you don't see the difference?
Python will not look for changes in mymodule.py each time it is used, it will load it a first time, compile it to bytecode and keep it internally. It will normally also save the compiled bytecode (mymodule.pyc). The next time you will start your program, it will check if mymodule.py is more recent than mymodule.pyc, and recompile it if necessary.
If you need to, you can reload the module explicitly:
import mymodule
[... some code ...]
if userAskedForRefresh:
reload(mymodule)
Of course, it is more complicated than that and you may have side-effects depending on what you do with your program regarding the other module, for example if variables depends on classes defined in mymodule.
Alternatively, you could use the execfile function (or exec(), eval(), compile())
I had the exact same issue creating a geoprocessing script for ArcGIS 10.2. I had a python toolbox script, a tool script and then a common script. I have a parameter for Dev/Test/Prod in the tool that would control which version of the code was run. Dev would run the code in the dev folder, test from test folder and prod from prod folder. Changes to the common dev script would not run when the tool was run from ArcCatalog. Closing ArcCatalog made no difference. Even though I selected Dev or Test it would always run from the prod folder.
Adding reload(myCommonModule) to the tool script resolved this issue.
The cases will be different for different versions of python.
Following shows an example of python 3.4 version or above:
hello import hello_world
#Calls hello_world function
hello_world()
HI !!
#Now changes are done and reload option is needed
import importlib
importlib.reload(hello)
hello_world()
How are you?
For earlier python versions like 2.x, use inbuilt reload function as stated above.
Better is to use ipython3 as it provides autoreload feature.