Suppose I have a Python program that runs slow- after profiliing and I have identified the bottleneck. One particular function from a 3rd party module I imported is particularly slow.
For this particular case, I know that function is implemented in Python (Used Eclipse and it's easy to jump to the function definition). So I know that I can convert that function into Cython as a speed-up option. (If it is already implemented in C, there is no point in writing it in Cython...).
If I don't have an IDE, what would be an easy option to determine this?
I know that I can go to the directory where the module is installed and infer that it is in C if the module is in .so. But is there any alternative?
Thanks
Check whether it is an instance of types.FunctionType:
>>> import types
>>> isinstance(len, types.FunctionType)
False
>>> def mylen(): pass
...
>>> isinstance(mylen, types.FunctionType)
True
Probably you'd be safer to check for isinstance(X, (types.FunctionType, types.LambdaType).
C functions are instances of builtin_function_or_method:
>>> len.__class__
<type 'builtin_function_or_method'>
>>> np.vdot.__class__
<type 'builtin_function_or_method'>
You can access this type as types.BuiltinFunctionType/types.BuiltinMethodType.
Alternatively you can check whether the function has a __code__ attribute. Since C functions do not have bytecode, they can't have __code__.
Note sometimes what seems like a function is actually a class, e.g. enumerate but some 3rd party library may do the same. This means that you should also check whether a class is implemented in C or not. This one is harder since all classes are instances of type. A way may be to check whether the class has a __dict__ in its dir, and if it doesn't have you should check for __slots__.
Something like the following should be pretty accurate:
def is_implemented_in_c(obj):
if isinstance(obj, (types.FunctionType, types.LambdaType)):
return False
elif isinstance(obj, type):
if '__dict__' in dir(obj): return False
return not hasattr(obj, '__slots__')
# We accept also instances of classes.
# Return True for instances of C classes, False for python classes.
return not isinstance(obj, types.InstanceType)
Example usage:
>>> is_implemented_in_c(enumerate)
True
>>> is_implemented_in_c(len)
True
>>> is_implemented_in_c(np.vdot)
True
>>> is_implemented_in_c(lambda x: True)
False
>>> is_implemented_in_c(object)
True
>>> class A(object):
... __slots__ = ('a', 'b')
...
>>> is_implemented_in_c(A)
False
In python I can create a class without class statement:
MyClass = type('X', (object,), dict(a=1))
Is there a way to create a function without 'def'?
Thats as far as i got...
d={} # func from string
exec'''\
def synthetics(s):
return s*s+1
''' in d
>>> d.keys()
['__builtins__', 'synthetics']
>>> d['synthetics']
<function synthetics at 0x00D09E70>
>>> foo = d['synthetics']
>>> foo(1)
2
Technically, yes, this is possible. The type of a function is, like all other types, a constructor for instances of that type:
FunctionType = type(lambda: 0)
help(FunctionType)
As you can see from the help, you need at minimum code and globals. The former is a compiled bytecode object; the latter is a dictionary.
To make the code object, you can use the code type's constructor:
CodeType = type((lambda: 0).func_code)
help(CodeType)
The help says this is "not for the faint of heart" and that's true. You need to pass bytecode and a bunch of other stuff to this constructor. So the easiest way to get a code object is from another function, or using the compile() function. But it is technically possible to generate code objects completely synthetically if you understand Python bytecode well enough. (I have done this, on a very limited basis, to construct signature-preserving wrapper functions for use in decorators.)
PS -- FunctionType and CodeType are also available via the types module.
There might be a more direct way than the following, but here's a full-blown function without def. First, use a trivial lambda expression to get a function object:
>>> func = lambda: None
Then, compile some source code to get a code object and use that to replace the lambda's code:
>>> func.__code__ = compile("print('Hello, world!')", "<no file>", "exec")
>>> func()
Hello, world!
If I import a module:
import foo
How can I find the names of the classes it contains?
You can use the inspect module to do this. For example:
import inspect
import foo
for name, obj in inspect.getmembers(foo):
if inspect.isclass(obj):
print name
Check in dir(foo). By convention, the class names will be those in CamelCase.
If foo breaks convention, you could I guess get the class names with something like [x for x in dir(foo) if type(getattr(foo, x)) == type], but that's ugly and probably quite fragile.
From the question How to check whether a variable is a class or not? we can check if a variable is a class or not with the inspect module (example code shamelessly lifted from the accepted answer):
>>> import inspect
>>> class X(object):
... pass
...
>>> inspect.isclass(X)
True
So to build a list with all the classes from a module we could use
import foo
classes = [c for c in dir(foo) if inspect.isclass(getattr(foo, c))]
Update: The above example has been edited to use getattr(foo, c) rather than use foo.__getattribute__(c), as per #Chris Morgan's comment.
You can check the type of elements found via dir(foo): classes will have type type.
import foo
classlist = []
for i in dir(foo):
if type(foo.__getattribute__(i)) is type:
classlist.append(i)
You can get a lot of information about a Python module, say foo, by importing it and then using help(module-name) as follows:
>>> import foo
>>> help(foo)
I'm starting to code in various projects using Python (including Django web development and Panda3D game development).
To help me understand what's going on, I would like to basically 'look' inside the Python objects to see how they tick - like their methods and properties.
So say I have a Python object, what would I need to print out its contents? Is that even possible?
Python has a strong set of introspection features.
Take a look at the following built-in functions:
type()
dir()
id()
getattr()
hasattr()
globals()
locals()
callable()
type() and dir() are particularly useful for inspecting the type of an object and its set of attributes, respectively.
object.__dict__
I'm surprised no one's mentioned help yet!
In [1]: def foo():
...: "foo!"
...:
In [2]: help(foo)
Help on function foo in module __main__:
foo()
foo!
Help lets you read the docstring and get an idea of what attributes a class might have, which is pretty helpful.
First, read the source.
Second, use the dir() function.
If this is for exploration to see what's going on, I'd recommend looking at IPython. This adds various shortcuts to obtain an objects documentation, properties and even source code. For instance appending a "?" to a function will give the help for the object (effectively a shortcut for "help(obj)", wheras using two ?'s ("func??") will display the sourcecode if it is available.
There are also a lot of additional conveniences, like tab completion, pretty printing of results, result history etc. that make it very handy for this sort of exploratory programming.
For more programmatic use of introspection, the basic builtins like dir(), vars(), getattr etc will be useful, but it is well worth your time to check out the inspect module. To fetch the source of a function, use "inspect.getsource" eg, applying it to itself:
>>> print inspect.getsource(inspect.getsource)
def getsource(object):
"""Return the text of the source code for an object.
The argument may be a module, class, method, function, traceback, frame,
or code object. The source code is returned as a single string. An
IOError is raised if the source code cannot be retrieved."""
lines, lnum = getsourcelines(object)
return string.join(lines, '')
inspect.getargspec is also frequently useful if you're dealing with wrapping or manipulating functions, as it will give the names and default values of function parameters.
If you're interested in a GUI for this, take a look at objbrowser. It uses the inspect module from the Python standard library for the object introspection underneath.
You can list the attributes of a object with dir() in the shell:
>>> dir(object())
['__class__', '__delattr__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__']
Of course, there is also the inspect module: http://docs.python.org/library/inspect.html#module-inspect
Try ppretty
from ppretty import ppretty
class A(object):
s = 5
def __init__(self):
self._p = 8
#property
def foo(self):
return range(10)
print ppretty(A(), indent=' ', depth=2, width=30, seq_length=6,
show_protected=True, show_private=False, show_static=True,
show_properties=True, show_address=True)
Output:
__main__.A at 0x1debd68L (
_p = 8,
foo = [0, 1, 2, ..., 7, 8, 9],
s = 5
)
While pprint has been mentioned already by others I'd like to add some context.
The pprint module provides a capability to “pretty-print” arbitrary
Python data structures in a form which can be used as input to the
interpreter. If the formatted structures include objects which are not
fundamental Python types, the representation may not be loadable. This
may be the case if objects such as files, sockets, classes, or
instances are included, as well as many other built-in objects which
are not representable as Python constants.
pprint might be in high-demand by developers with a PHP background who are looking for an alternative to var_dump().
Objects with a dict attribute can be dumped nicely using pprint() mixed with vars(), which returns the __dict__ attribute for a module, class, instance, etc.:
from pprint import pprint
pprint(vars(your_object))
So, no need for a loop.
To dump all variables contained in the global or local scope simply use:
pprint(globals())
pprint(locals())
locals() shows variables defined in a function.
It's also useful to access functions with their corresponding name as a string key, among other usages:
locals()['foo']() # foo()
globals()['foo']() # foo()
Similarly, using dir() to see the contents of a module, or the attributes of an object.
And there is still more.
"""Visit http://diveintopython.net/"""
__author__ = "Mark Pilgrim (mark#diveintopython.org)"
def info(object, spacing=10, collapse=1):
"""Print methods and doc strings.
Takes module, class, list, dictionary, or string."""
methodList = [e for e in dir(object) if callable(getattr(object, e))]
processFunc = collapse and (lambda s: " ".join(s.split())) or (lambda s: s)
print "\n".join(["%s %s" %
(method.ljust(spacing),
processFunc(str(getattr(object, method).__doc__)))
for method in methodList])
if __name__ == "__main__":
print help.__doc__
Others have already mentioned the dir() built-in which sounds like what you're looking for, but here's another good tip. Many libraries -- including most of the standard library -- are distributed in source form. Meaning you can pretty easily read the source code directly. The trick is in finding it; for example:
>>> import string
>>> string.__file__
'/usr/lib/python2.5/string.pyc'
The *.pyc file is compiled, so remove the trailing 'c' and open up the uncompiled *.py file in your favorite editor or file viewer:
/usr/lib/python2.5/string.py
I've found this incredibly useful for discovering things like which exceptions are raised from a given API. This kind of detail is rarely well-documented in the Python world.
There is a very cool tool called objexplore. Here is a simple example on how to use its explore function on a pandas DataFrame.
import pandas as pd
df=pd.read_csv('https://storage.googleapis.com/download.tensorflow.org/data/heart.csv')
from objexplore import explore
explore(df)
Will pop up the following in your shell:
Two great tools for inspecting code are:
IPython. A python terminal that allows you to inspect using tab completion.
Eclipse with the PyDev plugin. It has an excellent debugger that allows you to break at a given spot and inspect objects by browsing all variables as a tree. You can even use the embedded terminal to try code at that spot or type the object and press '.' to have it give code hints for you.
If you want to look at parameters and methods, as others have pointed out you may well use pprint or dir()
If you want to see the actual value of the contents, you can do
object.__dict__
pprint and dir together work great
There is a python code library build just for this purpose: inspect Introduced in Python 2.7
If you are interested to see the source code of the function corresponding to the object myobj, you can type in iPython or Jupyter Notebook:
myobj??
In Python 3.8, you can print out the contents of an object by using the __dict__. For example,
class Person():
pass
person = Person()
## set attributes
person.first = 'Oyinda'
person.last = 'David'
## to see the content of the object
print(person.__dict__)
{"first": "Oyinda", "last": "David"}
If you are looking for a slightly more delicate solution, you could try objprint. A positive side of it is that it can handle nested objects. For example:
from objprint import objprint
class Position:
def __init__(self, x, y):
self.x = x
self.y = y
class Player:
def __init__(self):
self.name = "Alice"
self.age = 18
self.items = ["axe", "armor"]
self.coins = {"gold": 1, "silver": 33, "bronze": 57}
self.position = Position(3, 5)
objprint(Player())
Will print out
<Player
.name = 'Alice',
.age = 18,
.items = ['axe', 'armor'],
.coins = {'gold': 1, 'silver': 33, 'bronze': 57},
.position = <Position
.x = 3,
.y = 5
>
>
import pprint
pprint.pprint(obj.__dict__)
or
pprint.pprint(vars(obj))
If you want to look inside a live object, then python's inspect module is a good answer. In general, it works for getting the source code of functions that are defined in a source file somewhere on disk. If you want to get the source of live functions and lambdas that were defined in the interpreter, you can use dill.source.getsource from dill. It also can get the code for from bound or unbound class methods and functions defined in curries... however, you might not be able to compile that code without the enclosing object's code.
>>> from dill.source import getsource
>>>
>>> def add(x,y):
... return x+y
...
>>> squared = lambda x:x**2
>>>
>>> print getsource(add)
def add(x,y):
return x+y
>>> print getsource(squared)
squared = lambda x:x**2
>>>
>>> class Foo(object):
... def bar(self, x):
... return x*x+x
...
>>> f = Foo()
>>>
>>> print getsource(f.bar)
def bar(self, x):
return x*x+x
>>>
vars(obj) returns the attributes of an object.
In addition if you want to look inside list and dictionaries, you can use pprint()
Many good tipps already, but the shortest and easiest (not necessarily the best) has yet to be mentioned:
object?
Try using:
print(object.stringify())
where object is the variable name of the object you are trying to inspect.
This prints out a nicely formatted and tabbed output showing all the hierarchy of keys and values in the object.
NOTE: This works in python3. Not sure if it works in earlier versions
UPDATE: This doesn't work on all types of objects. If you encounter one of those types (like a Request object), use one of the following instead:
dir(object())
or
import pprint
then:
pprint.pprint(object.__dict__)