SciPy's website has a tutorial that references a python function source that lists the source code of functions written in python, but I cannot use it in python or find documentation for it online. The reference is at the bottom of this page. I see that the inspect module has similar functions but I'm still curious as to what function in what module they are referring to.
np.source is a utility function in numpy. This can be called on any numpy/scipy or Python function or class, though compiled built-ins won't show anything.
The scipy and numpy API docs also have a [source] link that takes you to a source file.
I think np.source is relatively new, but as a long time ipython user I've been getting the same information with its ?? magic.
np.source(np.source) gives me
def source(object, output=sys.stdout):
"""
...
"""
# Local import to speed up numpy's import time.
import inspect
try:
print("In file: %s\n" % inspect.getsourcefile(object), file=output)
print(inspect.getsource(object), file=output)
except:
print("Not available for this object.", file=output)
In [425]: np.source?? shows the same thing but with some color coding.
Related
How can I subscribe to an event using ALMemory's subscribeToEvent function, which requires the correct Python scope while using ROS (rospy) that initiates modules outside my code?
This question is similar to this Naoqi subscription question, but differs in the sense that rospy was not used, which is critical to the implementation.
My current code is as such;
mainBehaviour.py
from nao.nao import Nao
class mainBehaviour(AbstractBehaviour):
def init(self):
global mainBehaviourMethod
mainBehaviourMethod = self
self.nao.setCallOnFall("mainBehaviourMethod", "robotHasFallen")
def update(self):
print("Update")
def robotHasFallen(self, eventName, val, subscriberIdentifier):
print("FALLING")
nao.py
from naoqi import ALProxy, ALModule, ALBroker
import rospy
from math import radians
import weakref
class Nao(object):
def __init__(self):
self.nao_ip = rospy.get_param("nao_ip")
self.port = 9559
self.memory_proxy = ALProxy("ALMemory", self.nao_ip, self.port)
def setCallOnFall(self, module, method):
self.memory_proxy.subscribeToEvent("robotHasFallen", module, method)
What I want is that mainBehaviour.py, using nao.py, has its function robotHasFallen fire when the robot has fallen. However, the current code does not produce this behaviour (it ignores any fall), but does not produce an error either. I attempted to implement this using this ALMemory tutorial. However, this tutorial makes use of a single python file, in which methods aren't instantiated by ROS. Therefore I cannot make use of the line pythonModule = myModule("pythonModule"). I attempted to still obtain this Python scope (on which the answer to the previously linked question states "the python variable must have the same name than the module name you create"), by declaring a global variable pointing to self.
How can I create the desired behaviour, detecting a fallen robot using subscribeToEvent, using ROS with its consequences of not initiating modules myself and therefore not being able to pass its constructor?
I cannot install any further libraries because I use a university computer.
Your example code uses the "naoqi" library, but it's now more convenient to use the "qi" library (you can get it with "pip install qi", it's already present on your robot in versions 2.1 or above). In that version you can directly pass a callback, see here for a helper library that allows you to do events.connect("MyALMemoryKey", my_callback) (you pass the function and not just it's name - and it doesn't care where the function comes from).
Under the hood, it does this:
ALMemory.subscriber("MyALMemoryKey").signal.connect(my_callback)
(note that here ALMemory is a Service (qi framework) and not a module (naoqi framework).
You can directly use that helper lib (see the doc here, and some samples using it here), or use the same logic in your code (it's not very complicated, once you have a working example to start from).
Is it possible to import a Python module from over the internet using the http(s), ftp, smb or any other protocol? If so, how? If not, why?
I guess it's about making Python use more the one protocol(reading the filesystem) and enabling it to use others as well. Yes I agree it would be many folds slower, but some optimization and larger future bandwidths would certainly balance it out.
E.g.:
import site
site.addsitedir("https://bitbucket.org/zzzeek/sqlalchemy/src/e8167548429b9d4937caaa09740ffe9bdab1ef61/lib")
import sqlalchemy
import sqlalchemy.engine
Another version,
I like this answer. when applied it, i simplified it a bit - similar to the look and feel of javascript includes over HTTP.
This is the result:
import os
import imp
import requests
def import_cdn(uri, name=None):
if not name:
name = os.path.basename(uri).lower().rstrip('.py')
r = requests.get(uri)
r.raise_for_status()
codeobj = compile(r.content, uri, 'exec')
module = imp.new_module(name)
exec (codeobj, module.__dict__)
return module
Usage:
redisdl = import_cdn("https://raw.githubusercontent.com/p/redis-dump-load/master/redisdl.py")
# Regular usage of the dynamic included library
json_text = redisdl.dumps(host='127.0.0.1')
Tip - place the import_cdn function in a common library, this way you could re-use this small function
Bear in mind It will fail when no connectivity to that file over http
In principle, yes, but all of the tools built-in which kinda support this go through the filesystem.
To do this, you're going to have to load the source from wherever, compile it with compile, and exec it with the __dict__ of a new module. See below.
I have left the actually grabbing text from the internet, and parsing uris etc as an exercise for the reader (for beginners: I suggest using requests)
In pep 302 terms, this would be the implementation behind a loader.load_module function (the parameters are different). See that document for details on how to integrate this with the import statement.
import imp
modulesource = 'a=1;b=2' #load from internet or wherever
def makemodule(modulesource,sourcestr='http://some/url/or/whatever',modname=None):
#if loading from the internet, you'd probably want to parse the uri,
# and use the last part as the modulename. It'll come up in tracebacks
# and the like.
if not modname: modname = 'newmodulename'
#must be exec mode
# every module needs a source to be identified, can be any value
# but if loading from the internet, you'd use the URI
codeobj = compile(modulesource, sourcestr, 'exec')
newmodule = imp.new_module(modname)
exec(codeobj,newmodule.__dict__)
return newmodule
newmodule = makemodule(modulesource)
print(newmodule.a)
At this point newmodule is already a module object in scope, so you don't need to import it or anything.
modulesource = '''
a = 'foo'
def myfun(astr):
return a + astr
'''
newmod = makemodule(modulesource)
print(newmod.myfun('bat'))
Ideone here: http://ideone.com/dXGziO
Tested with python 2, should work with python 3 (textually compatible print used;function-like exec syntax used).
This seems to be a use case for a self-written import hook. Look up in PEP 302 how exactly they work.
Essentially, you'll have to provide a finder object which, in turn, provides a loader object. I don't understand the process at the very first glance (otherwise I'd be more explicit), but the PEP contains all needed details for implementing the stuff.
As glglgl's has it this import hook has been implemented for Python2 and Python3 in a module called httpimport.
It uses a custom finder/loader object to locate resources using HTTP/S.
Additionally, the import_cdn function in Jossef Harush's answer is almost identically implemented in httpimport's github_repo, and bitbucket_repo functions.
#Marcin's answer contains a good portion of the code of the httpimport's loader class.
For teaching purposes I want an IPython notebook that displays (as output from a cell) the function source code, but I want to be able to reference this in multiple notebooks. Hence I would like to display the function code, in a similar way to using the %psource magic, but appropriately syntax highlighted.
This is a similar question to this question, but I want to be able to apply it to a single function within a file, rather than to the complete file at once.
Using the suggestion from the previous question I hacked a short code that works in simple cases:
def print_source(module, function):
"""For use inside an IPython notebook: given a module and a function, print the source code."""
from inspect import getmembers, isfunction, getsource
from pygments import highlight
from pygments.lexers import PythonLexer
from pygments.formatters import HtmlFormatter
from IPython.core.display import HTML
internal_module = __import__(module)
internal_functions = dict(getmembers(internal_module, isfunction))
return HTML(highlight(getsource(internal_functions[function]), PythonLexer(), HtmlFormatter(full=True)))
Two questions:
This gist suggests that showing the whole function could be done by defining appropriate cell magic. Is it possible to define an appropriate cell magic to just show a single function, as above?
Is there a way of doing this without importing the entire module, or a more robust way of doing this?
1) Magics are just simple function not difficult to define, you could have a look here Customizing IPython - Config.ipynb if I remember correctly. still I'm not sure it is worth definig a magic in your case.
2) Most of the time, no. You have to import the module as we need live code to know where it is defined.
In general, finding the code of a function is not always super easy. On python 3 you can always access the code object, but most of the time, as soon as you have things like decorated function, or dynamically generated function, it becomes difficult. I suppose you could also inspire from psource/pinfo2 and have them return info instead of paging it.
Is it possible to import a Python module from over the internet using the http(s), ftp, smb or any other protocol? If so, how? If not, why?
I guess it's about making Python use more the one protocol(reading the filesystem) and enabling it to use others as well. Yes I agree it would be many folds slower, but some optimization and larger future bandwidths would certainly balance it out.
E.g.:
import site
site.addsitedir("https://bitbucket.org/zzzeek/sqlalchemy/src/e8167548429b9d4937caaa09740ffe9bdab1ef61/lib")
import sqlalchemy
import sqlalchemy.engine
Another version,
I like this answer. when applied it, i simplified it a bit - similar to the look and feel of javascript includes over HTTP.
This is the result:
import os
import imp
import requests
def import_cdn(uri, name=None):
if not name:
name = os.path.basename(uri).lower().rstrip('.py')
r = requests.get(uri)
r.raise_for_status()
codeobj = compile(r.content, uri, 'exec')
module = imp.new_module(name)
exec (codeobj, module.__dict__)
return module
Usage:
redisdl = import_cdn("https://raw.githubusercontent.com/p/redis-dump-load/master/redisdl.py")
# Regular usage of the dynamic included library
json_text = redisdl.dumps(host='127.0.0.1')
Tip - place the import_cdn function in a common library, this way you could re-use this small function
Bear in mind It will fail when no connectivity to that file over http
In principle, yes, but all of the tools built-in which kinda support this go through the filesystem.
To do this, you're going to have to load the source from wherever, compile it with compile, and exec it with the __dict__ of a new module. See below.
I have left the actually grabbing text from the internet, and parsing uris etc as an exercise for the reader (for beginners: I suggest using requests)
In pep 302 terms, this would be the implementation behind a loader.load_module function (the parameters are different). See that document for details on how to integrate this with the import statement.
import imp
modulesource = 'a=1;b=2' #load from internet or wherever
def makemodule(modulesource,sourcestr='http://some/url/or/whatever',modname=None):
#if loading from the internet, you'd probably want to parse the uri,
# and use the last part as the modulename. It'll come up in tracebacks
# and the like.
if not modname: modname = 'newmodulename'
#must be exec mode
# every module needs a source to be identified, can be any value
# but if loading from the internet, you'd use the URI
codeobj = compile(modulesource, sourcestr, 'exec')
newmodule = imp.new_module(modname)
exec(codeobj,newmodule.__dict__)
return newmodule
newmodule = makemodule(modulesource)
print(newmodule.a)
At this point newmodule is already a module object in scope, so you don't need to import it or anything.
modulesource = '''
a = 'foo'
def myfun(astr):
return a + astr
'''
newmod = makemodule(modulesource)
print(newmod.myfun('bat'))
Ideone here: http://ideone.com/dXGziO
Tested with python 2, should work with python 3 (textually compatible print used;function-like exec syntax used).
This seems to be a use case for a self-written import hook. Look up in PEP 302 how exactly they work.
Essentially, you'll have to provide a finder object which, in turn, provides a loader object. I don't understand the process at the very first glance (otherwise I'd be more explicit), but the PEP contains all needed details for implementing the stuff.
As glglgl's has it this import hook has been implemented for Python2 and Python3 in a module called httpimport.
It uses a custom finder/loader object to locate resources using HTTP/S.
Additionally, the import_cdn function in Jossef Harush's answer is almost identically implemented in httpimport's github_repo, and bitbucket_repo functions.
#Marcin's answer contains a good portion of the code of the httpimport's loader class.
I have a vaguely defined function of a class Graph of a module I call gt (it is graph-tool). so i declare g = gt.graph() then want to use g.degree_property_map but do not know how. Therefore I want to see where in code g.degree_property_map or in this case just the function, is defined. How can I find that? I'm working on command line on a vm.
Thanks
For reference the library in question is graph-tool - http://projects.skewed.de/graph-tool/
Also I am currently importing it using from graph_tool.all import * . that is of course somewhat of a problem.
You could use inspect.getsource(gt.Graph.degree_property_map). (You have to import inspect.)
Of course, what you pass into getsource() will change depending on how you imported Graph. So if you used from graphtools.all import *, you'd just need to use inspect.getsource(Graph.degree_property_map).
If you open interactive python (type python and hit ENTER on the command line), you should be able to run the command help(<graph's module name>), then, under the FILE section of the help documentation that is generated, you should see the absolute path to the code you are interested in.
For example, I just ran:
import numpy
help(numpy)
# Returned documentation containing:
# FILE
# /usr/lib/python2.7/dist-packages/numpy/__init__.py
Also,
import my_module # A module I just created that contains the "Line" class
help(my_module)
# Returned documentation containing:
# FILE
# /home/<my user name>/Programming/Python/my_module.py
If it is a normal function (not a builtin, ufunc, etc) you can try using the func_code attribute
For example:
>>> inspect.iscode
<function iscode at 0x02EAEF30>
>>> inspect.iscode.func_code
<code object iscode at 02EB2B60, file "C:\Python27\lib\inspect.py", line 209>
Never mind I just did help(graph_tool) and manually jumped through code. Thanks for the help though!