The following code to generate a quick plot of some data works fine when I run it in the cell within my notebook:
def xyz_plot(df=df, sensor='acc', position='t', activity='walking_treadmill_flat',
person=np.random.randint(1,9)):
sensors = [f'{position}_{i}{sensor}' for i in ['x','y','z']]
subset = df.query(f"person=='{person}' & activity=='{activity}'")
for j in sensors:
sns.lineplot(subset.seconds[100:200], subset[f'{j}'][100:200], label=f'{j}', legend='full').set(xlabel='', ylabel='Seconds', title=f'Person:{person}')
However, when I save it in my_functions.py file which I have imported as follows it no longer works and cannot find my data frame. How can I fix this?
from my_functions import xyz_plot
When you call your function, pass the argument to it, rather than setting it as a default argument!
If you must have default arguments, make sure they're not mutable
function declaration
def myfunction(arg, arg2=None):
work_with_arg(arg)
function call
from mylibrary import myfunction
...
myfunction(dataframe) # arg refers to the dataframe in the function
This structure should also be used to prevent problems with your person argument (see note about mutable default args), as the random value will be calculated only once on import / function declaration (which may be desired, but probably isn't..)
>>> import numpy as np
>>> def broken(x=np.random.randint(1,9)):
... print(x)
...
>>> broken()
7
>>> broken()
7
>>> broken()
7
>>> broken()
7
more proper
>>> def happy(x=None):
... if x is None:
... x=np.random.randint(1,9)
... print(x)
...
>>> happy()
4
>>> happy()
7
Related
Say I have two Python functions defined in a module called funcs.py:
def my_fun1(X):
return X * 2
def my_fun2(X):
return X / 2
and a dataclass defined in a file called testclass.py:
from dataclasses import dataclass
import funcs
#dataclass
class TestClass:
MY_FUN = funcs.my_fun1
MY_FUNS = [funcs.my_fun1, funcs.my_fun2]
N_X = [3, 4]
If I then run some test code:
import numpy as np
from testclass import TestClass
tc = TestClass()
data = [np.random.randn(x, 2) for x in tc.N_X]
print(tc.MY_FUN(data[0]))
it gives me the error:
TypeError: my_fun1() takes 1 positional argument but 2 were given
If I then change the definition of my_fun1 to:
def my_fun1(self, X):
It works fine, but when I try:
print(tc.MY_FUNS[0](data[0]))
I get the error:
TypeError: my_fun1() missing 1 required positional argument: 'X'
I can get around this by removing the self from the definitions and declaring a single function in the test class as a singleton list, e.g., MY_FUN = [funcs.my_fun1], however this feels like a bit of a "hacky" work-around, and I would like to be able to declare either a single function or list of functions without having to change the function definitions down the track.
Is there any way of doing this? Why does Python need the self argument sometimes, and not others in this instance?
Suppose I have a Python function as defined below:
def foo(arg1,arg2):
#do something with args
a = arg1 + arg2
return a
I can get the name of the function using foo.func_name. How can I programmatically get its source code, as I typed above?
If the function is from a source file available on the filesystem, then inspect.getsource(foo) might be of help:
If foo is defined as:
def foo(arg1,arg2):
#do something with args
a = arg1 + arg2
return a
Then:
import inspect
lines = inspect.getsource(foo)
print(lines)
Returns:
def foo(arg1,arg2):
#do something with args
a = arg1 + arg2
return a
But I believe that if the function is compiled from a string, stream or imported from a compiled file, then you cannot retrieve its source code.
The inspect module has methods for retrieving source code from python objects. Seemingly it only works if the source is located in a file though. If you had that I guess you wouldn't need to get the source from the object.
The following tests inspect.getsource(foo) using Python 3.6:
import inspect
def foo(arg1,arg2):
#do something with args
a = arg1 + arg2
return a
source_foo = inspect.getsource(foo) # foo is normal function
print(source_foo)
source_max = inspect.getsource(max) # max is a built-in function
print(source_max)
This first prints:
def foo(arg1,arg2):
#do something with args
a = arg1 + arg2
return a
Then fails on inspect.getsource(max) with the following error:
TypeError: <built-in function max> is not a module, class, method, function, traceback, frame, or code object
Just use foo?? or ??foo.
If you are using IPython, then you need to type foo?? or ??foo to see the complete source code. To see only the docstring in the function, use foo? or ?foo. This works in Jupyter notebook as well.
In [19]: foo??
Signature: foo(arg1, arg2)
Source:
def foo(arg1,arg2):
#do something with args
a = arg1 + arg2
return a
File: ~/Desktop/<ipython-input-18-3174e3126506>
Type: function
dis is your friend if the source code is not available:
>>> import dis
>>> def foo(arg1,arg2):
... #do something with args
... a = arg1 + arg2
... return a
...
>>> dis.dis(foo)
3 0 LOAD_FAST 0 (arg1)
3 LOAD_FAST 1 (arg2)
6 BINARY_ADD
7 STORE_FAST 2 (a)
4 10 LOAD_FAST 2 (a)
13 RETURN_VALUE
While I'd generally agree that inspect is a good answer, I'd disagree that you can't get the source code of objects defined in the interpreter. If you use dill.source.getsource from dill, you can get the source of functions and lambdas, even if they are defined interactively.
It also can get the code for from bound or unbound class methods and functions defined in curries... however, you might not be able to compile that code without the enclosing object's code.
>>> from dill.source import getsource
>>>
>>> def add(x,y):
... return x+y
...
>>> squared = lambda x:x**2
>>>
>>> print getsource(add)
def add(x,y):
return x+y
>>> print getsource(squared)
squared = lambda x:x**2
>>>
>>> class Foo(object):
... def bar(self, x):
... return x*x+x
...
>>> f = Foo()
>>>
>>> print getsource(f.bar)
def bar(self, x):
return x*x+x
>>>
To expand on runeh's answer:
>>> def foo(a):
... x = 2
... return x + a
>>> import inspect
>>> inspect.getsource(foo)
u'def foo(a):\n x = 2\n return x + a\n'
print inspect.getsource(foo)
def foo(a):
x = 2
return x + a
EDIT: As pointed out by #0sh this example works using ipython but not plain python. It should be fine in both, however, when importing code from source files.
Since this post is marked as the duplicate of this other post, I answer here for the "lambda" case, although the OP is not about lambdas.
So, for lambda functions that are not defined in their own lines: in addition to marko.ristin's answer, you may wish to use mini-lambda or use SymPy as suggested in this answer.
mini-lambda is lighter and supports any kind of operation, but works only for a single variable
SymPy is heavier but much more equipped with mathematical/calculus operations. In particular it can simplify your expressions. It also supports several variables in the same expression.
Here is how you can do it using mini-lambda:
from mini_lambda import x, is_mini_lambda_expr
import inspect
def get_source_code_str(f):
if is_mini_lambda_expr(f):
return f.to_string()
else:
return inspect.getsource(f)
# test it
def foo(arg1, arg2):
# do something with args
a = arg1 + arg2
return a
print(get_source_code_str(foo))
print(get_source_code_str(x ** 2))
It correctly yields
def foo(arg1, arg2):
# do something with args
a = arg1 + arg2
return a
x ** 2
See mini-lambda documentation for details. I'm the author by the way ;)
You can use inspect module to get full source code for that. You have to use getsource() method for that from the inspect module. For example:
import inspect
def get_my_code():
x = "abcd"
return x
print(inspect.getsource(get_my_code))
You can check it out more options on the below link.
retrieve your python code
to summarize :
import inspect
print( "".join(inspect.getsourcelines(foo)[0]))
Please mind that the accepted answers work only if the lambda is given on a separate line. If you pass it in as an argument to a function and would like to retrieve the code of the lambda as object, the problem gets a bit tricky since inspect will give you the whole line.
For example, consider a file test.py:
import inspect
def main():
x, f = 3, lambda a: a + 1
print(inspect.getsource(f))
if __name__ == "__main__":
main()
Executing it gives you (mind the indention!):
x, f = 3, lambda a: a + 1
To retrieve the source code of the lambda, your best bet, in my opinion, is to re-parse the whole source file (by using f.__code__.co_filename) and match the lambda AST node by the line number and its context.
We had to do precisely that in our design-by-contract library icontract since we had to parse the lambda functions we pass in as arguments to decorators. It is too much code to paste here, so have a look at the implementation of this function.
If you're strictly defining the function yourself and it's a relatively short definition, a solution without dependencies would be to define the function in a string and assign the eval() of the expression to your function.
E.g.
funcstring = 'lambda x: x> 5'
func = eval(funcstring)
then optionally to attach the original code to the function:
func.source = funcstring
Rafał Dowgird's answer states:
I believe that if the function is compiled from a string, stream or imported from a compiled file, then you cannot retrieve its source code.
However, it is possible to retrieve the source code of a function compiled from a string, provided that the compiling code also added an entry to the linecache.cache dict:
import linecache
import inspect
script = '''
def add_nums(a, b):
return a + b
'''
bytecode = compile(script, 'unique_filename', 'exec')
tmp = {}
eval(bytecode, {}, tmp)
add_nums = tmp["add_nums"]
linecache.cache['unique_filename'] = (
len(script),
None,
script.splitlines(True),
'unique_filename',
)
print(inspect.getsource(add_nums))
# prints:
# """
# def add_nums(a, b):
# return a + b
# """
This is how the attrs library creates various methods for classes automatically, given a set of attributes that the class expects to be initialized with. See their source code here. As the source explains, this is a feature primarily intended to enable debuggers such as PDB to step through the code.
I believe that variable names aren't stored in pyc/pyd/pyo files, so you can not retrieve the exact code lines if you don't have source files.
Suppose I have a Python function as defined below:
def foo(arg1,arg2):
#do something with args
a = arg1 + arg2
return a
I can get the name of the function using foo.func_name. How can I programmatically get its source code, as I typed above?
If the function is from a source file available on the filesystem, then inspect.getsource(foo) might be of help:
If foo is defined as:
def foo(arg1,arg2):
#do something with args
a = arg1 + arg2
return a
Then:
import inspect
lines = inspect.getsource(foo)
print(lines)
Returns:
def foo(arg1,arg2):
#do something with args
a = arg1 + arg2
return a
But I believe that if the function is compiled from a string, stream or imported from a compiled file, then you cannot retrieve its source code.
The inspect module has methods for retrieving source code from python objects. Seemingly it only works if the source is located in a file though. If you had that I guess you wouldn't need to get the source from the object.
The following tests inspect.getsource(foo) using Python 3.6:
import inspect
def foo(arg1,arg2):
#do something with args
a = arg1 + arg2
return a
source_foo = inspect.getsource(foo) # foo is normal function
print(source_foo)
source_max = inspect.getsource(max) # max is a built-in function
print(source_max)
This first prints:
def foo(arg1,arg2):
#do something with args
a = arg1 + arg2
return a
Then fails on inspect.getsource(max) with the following error:
TypeError: <built-in function max> is not a module, class, method, function, traceback, frame, or code object
Just use foo?? or ??foo.
If you are using IPython, then you need to type foo?? or ??foo to see the complete source code. To see only the docstring in the function, use foo? or ?foo. This works in Jupyter notebook as well.
In [19]: foo??
Signature: foo(arg1, arg2)
Source:
def foo(arg1,arg2):
#do something with args
a = arg1 + arg2
return a
File: ~/Desktop/<ipython-input-18-3174e3126506>
Type: function
dis is your friend if the source code is not available:
>>> import dis
>>> def foo(arg1,arg2):
... #do something with args
... a = arg1 + arg2
... return a
...
>>> dis.dis(foo)
3 0 LOAD_FAST 0 (arg1)
3 LOAD_FAST 1 (arg2)
6 BINARY_ADD
7 STORE_FAST 2 (a)
4 10 LOAD_FAST 2 (a)
13 RETURN_VALUE
While I'd generally agree that inspect is a good answer, I'd disagree that you can't get the source code of objects defined in the interpreter. If you use dill.source.getsource from dill, you can get the source of functions and lambdas, even if they are defined interactively.
It also can get the code for from bound or unbound class methods and functions defined in curries... however, you might not be able to compile that code without the enclosing object's code.
>>> from dill.source import getsource
>>>
>>> def add(x,y):
... return x+y
...
>>> squared = lambda x:x**2
>>>
>>> print getsource(add)
def add(x,y):
return x+y
>>> print getsource(squared)
squared = lambda x:x**2
>>>
>>> class Foo(object):
... def bar(self, x):
... return x*x+x
...
>>> f = Foo()
>>>
>>> print getsource(f.bar)
def bar(self, x):
return x*x+x
>>>
To expand on runeh's answer:
>>> def foo(a):
... x = 2
... return x + a
>>> import inspect
>>> inspect.getsource(foo)
u'def foo(a):\n x = 2\n return x + a\n'
print inspect.getsource(foo)
def foo(a):
x = 2
return x + a
EDIT: As pointed out by #0sh this example works using ipython but not plain python. It should be fine in both, however, when importing code from source files.
Since this post is marked as the duplicate of this other post, I answer here for the "lambda" case, although the OP is not about lambdas.
So, for lambda functions that are not defined in their own lines: in addition to marko.ristin's answer, you may wish to use mini-lambda or use SymPy as suggested in this answer.
mini-lambda is lighter and supports any kind of operation, but works only for a single variable
SymPy is heavier but much more equipped with mathematical/calculus operations. In particular it can simplify your expressions. It also supports several variables in the same expression.
Here is how you can do it using mini-lambda:
from mini_lambda import x, is_mini_lambda_expr
import inspect
def get_source_code_str(f):
if is_mini_lambda_expr(f):
return f.to_string()
else:
return inspect.getsource(f)
# test it
def foo(arg1, arg2):
# do something with args
a = arg1 + arg2
return a
print(get_source_code_str(foo))
print(get_source_code_str(x ** 2))
It correctly yields
def foo(arg1, arg2):
# do something with args
a = arg1 + arg2
return a
x ** 2
See mini-lambda documentation for details. I'm the author by the way ;)
You can use inspect module to get full source code for that. You have to use getsource() method for that from the inspect module. For example:
import inspect
def get_my_code():
x = "abcd"
return x
print(inspect.getsource(get_my_code))
You can check it out more options on the below link.
retrieve your python code
to summarize :
import inspect
print( "".join(inspect.getsourcelines(foo)[0]))
Please mind that the accepted answers work only if the lambda is given on a separate line. If you pass it in as an argument to a function and would like to retrieve the code of the lambda as object, the problem gets a bit tricky since inspect will give you the whole line.
For example, consider a file test.py:
import inspect
def main():
x, f = 3, lambda a: a + 1
print(inspect.getsource(f))
if __name__ == "__main__":
main()
Executing it gives you (mind the indention!):
x, f = 3, lambda a: a + 1
To retrieve the source code of the lambda, your best bet, in my opinion, is to re-parse the whole source file (by using f.__code__.co_filename) and match the lambda AST node by the line number and its context.
We had to do precisely that in our design-by-contract library icontract since we had to parse the lambda functions we pass in as arguments to decorators. It is too much code to paste here, so have a look at the implementation of this function.
If you're strictly defining the function yourself and it's a relatively short definition, a solution without dependencies would be to define the function in a string and assign the eval() of the expression to your function.
E.g.
funcstring = 'lambda x: x> 5'
func = eval(funcstring)
then optionally to attach the original code to the function:
func.source = funcstring
Rafał Dowgird's answer states:
I believe that if the function is compiled from a string, stream or imported from a compiled file, then you cannot retrieve its source code.
However, it is possible to retrieve the source code of a function compiled from a string, provided that the compiling code also added an entry to the linecache.cache dict:
import linecache
import inspect
script = '''
def add_nums(a, b):
return a + b
'''
bytecode = compile(script, 'unique_filename', 'exec')
tmp = {}
eval(bytecode, {}, tmp)
add_nums = tmp["add_nums"]
linecache.cache['unique_filename'] = (
len(script),
None,
script.splitlines(True),
'unique_filename',
)
print(inspect.getsource(add_nums))
# prints:
# """
# def add_nums(a, b):
# return a + b
# """
This is how the attrs library creates various methods for classes automatically, given a set of attributes that the class expects to be initialized with. See their source code here. As the source explains, this is a feature primarily intended to enable debuggers such as PDB to step through the code.
I believe that variable names aren't stored in pyc/pyd/pyo files, so you can not retrieve the exact code lines if you don't have source files.
I often write code that looks like this and am looking for a better suggestion. Basically, I usually create some general function myfuc_general that handles all the cases I need with parameters, usually with optional parameters. However, often I usually run 2 (possibly more) specific functions. Everything is the same except one of the arguments is different, in this case a. I run them so often that I actually prefer to just have two additional functions so I don't have to remember what the optional parameter needs to be.
So for myfunct_specific1, I am running a=10 and for myfunct_specific2, a=20. Is there something better to do than this? This seems pretty sloppy and it has the disadvantage in the event that I need to change the myfuct_general call, then I have to change all the other functions.
def myfunc_general(constant, a=1,b=2):
return constant+a+b
def myfunct_specific1(constant,b=2):
a=10
return myfunc_general(constant,a,b=2)
def myfunct_specific2(constant,b=2):
a=20
return myfunc_general(constant,a,b=2)
print myfunct_specific1(3) #15
print myfunct_specific2(3) #25
edit (addition):
iCodez thank you for suggestion. I have this particular situation and it is throwing me an error. Help? Thanks again
def myfunc_general(constant, constant2, a=0,b=2):
return constant+constant2+b+a
import functools
myfunct_specific=functools.partial(myfunc_general,constant2=30)
print myfunct_specific
print myfunct_specific(3,5,b=3)
Traceback (most recent call last):
File "C:/Python27/test", line 8, in <module>
print myfunct_specific(3,5,b=3)
TypeError: myfunc_general() got multiple values for keyword argument 'constant2'
You can use functools.partial to make this a lot easier:
from functools import partial
def myfunc_general(constant, a=1, b=2):
return constant+a+b
myfunct_specific1 = partial(myfunc_general, a=10)
myfunct_specific2 = partial(myfunc_general, a=20)
Below is a demonstration:
>>> from functools import partial
>>>
>>> def myfunc_general(constant, a=1, b=2):
... return constant+a+b
...
>>> myfunct_specific1 = partial(myfunc_general, a=10)
>>> myfunct_specific2 = partial(myfunc_general, a=20)
>>>
>>> print myfunct_specific1(3)
15
>>> print myfunct_specific2(3)
25
>>>
I have a python script that starts with importing a python module that contains data. A very simplified example is given below:
my_data1.py
bar = 10
baz = [1, 5, 7]
...
my_func.py
from my_data1 import *
def foo():
'''
function that uses the things defined in data (scalar, list, dicts, etc.)
in my_data
'''
return [bar] + baz
This works great for one set of data; however, I have my_data1.py, ..., my_data36.py.
my_data36.py
bar = 31
baz = [-1, 58, 8]
...
that I want to import and then run foo() with that data. I wanted to do something like this:
def foo(my_data):
from my_data import *
results = []
for i in range(1,37):
results.append(foo('my_data{}'.format(i)))
This doesn't work. Ideas?
Use __import__. It takes a string as parameter identifying the module to be imported, and returns the module, which then you can pass as an argument to your functions.
def processDataSet (module):
print (module.baz)
for m in ['data1.py', 'data2.py', 'data69.py']:
processDataSet (__import__ (m) )
"from module import * is invalid inside function definitions." from http://docs.python.org/2/howto/doanddont.html#inside-function-definitions