I need to understand the code base of a relatively big Python library. The code being too convoluted, I thought it would be a good start to know which methods call which other methods when I execute an imported function that I need to use.
Is it possible to do something along the lines of:
from library import function
get_traceback(function)
And get a list of all the functions being called in the background by the library?
Related
I am currently in the process of writing a small python module using the Python API, that will speed up some of the slower python code, that is repeatedly run in a in a simulation of sorts. My issue is that currently this code is takes a bunch of arguments, that in many use cases won't change. For example the function signature will be like: func(x,a,b,c,d,e), but after an initialisation only x will change. I therefore will have the python code littered with lambda x : func(x,a,b,c,d,e) where I wrap these before use. I have observed that this actually introduces quite a bit of calling overhead.
My idea to fix this was to create a PyObject* that is essentially C++ lambda instead of the python one. The main issue with this is that I have not found a way to create PyObjects from C++ lambdas, or even lower level functions. Since functions/lambdas in python can be passed as arguments I assume it is possible, but is there a clean way I'm missing.
I would seriously consider using swig instead of pybind11 for example. It's just peace of mind. If you don't want to use swig directly, you can at least see what swig does to wrap up features like proxy objects.
http://www.swig.org/Doc2.0/SWIGPlus.html#SWIGPlus_nn38
I'm trying to extract all the "runnable" code given a function in a module. Right now I'm only able to extract the functions in any imported module that are called inside the starting function. However, some modules have "outside" expressions (i.e. some variables defined globally in the module, or functions called in the same level).
With inspect and dis I did the work to extract the functions, but, is there any way of extracting the "non-function" of a module?
If anybody wonders what am I doing, is a packer for python. What I want to achieve is that this tools only packs the required code given a starting function.
Also, if is there something already out there that does what I'm trying to achieve, I'd like to know.
importing a module as a code object
You can use importlib's get_code() method which returns the code object of the module and then you can modify or create a new code object extracting the required parts. exec method can be used for executing code object.
Alternatively, using the built-in compile function, you can directly compile source code into byte code then follow the same procedure as mentioned above.
References:
Modifying python bytecode
assembling-python-module-on-fly-dynamic-import
I've split a program into three scripts. One of them, 'classes.py', is a module defining all the classes I need. Another one is a sort of setup module, call it 'setup.py', which instantiates a lot of objects from 'classes.py' (it's just a bunch of variable assignments with a few for loops, no functions or classes). It has a lot of strings and stuff I don't want to see when I'm working on the third script which is the program itself, i.e. the script that actually does something with all of the above.
The only way I got this to work was to add, in the 'setup.py' script:
from classes import *
This allows me to write quickly in the setup file without having the namespace added everywhere. And, in the main script:
import setup
This has the advantages of PyCharm giving me full code completion for classes and methods, which is nice.
What I'd like to achieve is having the main script import the classes, and then run the setup script to create the objects I need, with two simple commands. But I can't import the classes script into the main script because then the setup script can't do anything, having no class definitions. Should I import the classes into both scripts, or do something else entirely?
Import in each file. Consider this SO post. From the answer by Mr Fooz there,
Each module has its own namespace. So for boo.py to see something from an external module, boo.py must import it itself.
It is possible to write a language where namespaces are stacked the way you expect them to: this is called dynamic scoping. Some languages like the original lisp, early versions of perl, postscript, etc. do use (or support) dynamic scoping.
Most languages use lexical scoping instead. It turns out this is a much nicer way for languages to work: this way a module can reason about how it will work based on its own code without having to worry about how it was called.
See this article for additional details: http://en.wikipedia.org/wiki/Scope_%28programming%29
Intuitively this feels nicer too, as you can immediately (in the file itself) see which dependencies the code has - this will allow you to understand your code much better, a month, or even a year from now.
I would like to know if there is any way we can call a function written in Lua from Python code. If so, could you please suggest me the best possible way to do the same?
Basically, I have 2 different modules one is written in Lua and other in Python and wanted to use them as it is without rewriting Lua module into Python.
What is the easiest way to record function calls for debugging in Python? I'm usually interested in particular functions or all functions from a given class. Or sometimes even all functions called on a particular object attribute. Seeing the call arguments would be useful, too.
I can imagine writing decorators for all that, but then I'd still have to modify the source code in different places. And writing a class decorator which modifies all methods isn't that straightforward.
Is there a solution where I don't have to modify my source code? Ideally something which doesn't slow down Python too much.
You ought to be able to implement something that does what you want using either sys.setprofile() or perhaps sys.settrace(). They both let you define a function to be called when specific "events" occur, like function calls, and pass additional information to which can be used to to determine the function/method being called and examine its arguments.
If you look around, there's probably sample usage code to use as a good starting point.
Except decorators, for Python >= 3.0 you could use new __getattribute__ method for a class, which will be called every time you call any method of the object.
You could look through Lutz "Learning Python" chapters 31, 37 about it.