Is it possible to use with statement in Python anonymous functions? For example, I have a function that writes 1 to a file:
def write_one(filename):
with open(filename, 'wt') as fp:
fp.write('1')
But this function is to be organized in a dict:
my_functions = {
....
}
Obviously I can write this statement to add this function to the dict:
my_functions['write_one'] = write_one
But the problem is the name write_one still exists in the current scope. How can I introduce an anonymous function without polluting the current namespace?
For simple functions, I can use lambda. For slightly complicated functions, I can return a tuple to execute multiple statements (to be precise, expressions). But I didn't find a way to cleverly use lambda so that it can work with with statements. If this is impossible, where it says so in its documentation?
The solution with a del write_one doesn't look good to me. I don't want this name to be introduced at all in the current namespace.
In a word, what I want is something like this:
my_functions['write_one'] = def(filename):
with open(filename, 'wt') as fp:
fp.write('1')
This is kind of awkward with Python's indentation-based rules, I know. But it does its job.
Lambda expressions are quite restricted in what they can do. From the docs
Note that functions created with lambda expressions cannot contain statements or annotations.
Just use a full function definition. If you really want to avoid polluting the namespace, just del the name afterwards.
Or if you simply want to avoid the module namespace from having a bunch of these small functions for code-completion purposes, use _ in the front of the function name.
If you truly want to avoid it, you could use the function constructor and dynamically compile code, etc. Or use some other kind of dynamic code execution, e.g. using eval or exec. But that is almost certainly not worth the trouble.
Especially if you can just del the name after you are done using it.
Or perhaps the best approach is to put all these functions in another namespace, like another module.
There are not many namespaces in Python - a function is basically the local one. That gives a good hint to a possible solution:
def gen_functions():
def f1(i):
print(i)
def f2(i):
print(i+1)
return f1,f2
my_funcs = dict(zip(('a','b'),gen_functions()))
my_funcs['a'](2)
my_funcs['b'](7)
f1(3)
Run this:
>py bla.py
2
8
Traceback (most recent call last):
File "bla.py", line 11, in <module>
f1(3)
NameError: name 'f1' is not defined
So you can make arbitrarily complex functions - to hide them from the global namespace you just enclose them in another function.
Related
I'm writing some code for a calculus package in python. Ideally, I would like to support compositions of user-defined functions. For example, a user could define functions within my SingleVariable class like so:
f = SingleVariable('sin(x)')
g = SingleVariable('x+4')
and then a composition
h = SingleVariable('f(g(x))')
These functions can be integrated, differentiated, etc. The method used to evaluate the function at a point is as follows:
def infix_to_function(infix, variable):
def f(x):
expression = infix.replace(variable, str(x)).replace('^', '**')
return eval(expression)
return f
This works fine for functions like sin and ln because they can be loaded ahead of time so as to be recognized by eval. However, this obviously doesn't work for user-defined functions because they don't exist in the namespace in which the function is actually evaluated. Returning to the example functions f and g, i would like a function defined as
h = SingleVariable('f(g(x))')
to be equivalent to
h = SingleVariable('sin(x+4)')
I'm writing the package in PyCharm and testing it by importing to a Jupyter Notebook. I thought about adding the function to a list of some sort when a new SingleVariable object is initialized, but that would require somehow grabbing the name of the variable it is being assigned to.
Is there any way to add the functions I define in the Jupyter Notebook to the namespace of the PyCharm package so that they can be recognized by eval and have the behavior described?
You can try passing in globals() to eval like eval(expression, globals()). I believe Jupyter Notebooks will put any functions that have been defined into the global namespace.
However, using exec() and globals() can cause a lot of problems - so make sure you really understand what you are doing.
It will be a bit more complicated but if you plan to build this out to be very general you may want to look at a lexing libraries like http://www.dabeaz.com/ply/example.html which can help you ensure that the input provide is valid and parsed correctly.
As a small example of the type of problems you can have - based on your sample code I think:
f = SingleVariable('sin(s)')
may not work the way you expect...
I have a body of python code that contains inline functions within functions. I'd like to unit test the make_exciting inner function, so I'm trying to figure out how to invoke it directly.
def say_something_exciting(name, phrase):
def make_exciting(phrase):
return phrase + "!"
return "%s says '%s'" % (name, make_exciting(phrase))
Function say_something_exciting is written at the top level of a .py file, and is not inside a class. The py file is in the org.something module. Tried:
Invoking the function directly via org.something.say_something_exciting.make_exciting("Hello") - error: 'function' object has no attribute 'make_exciting'
Inspecting dir(org.something.say_something_exciting) and org.something.say_something_exciting.__dict__ for any paths to traverse, didn't see make_exciting anywhere.
internal_function = org.something.say_something_exciting.__dict__.get('make_exciting'), but internal_function is None.
How can I access (unit test) this inner function? This may suggest what I'm asking isn't possible. I'm generally familiar with unit testing and how to use the unittest module; accessing the function is the problem. If it's not possible, how should I re-write this code to support testing (if other than promote the inner function to a top-level function)?. Thanks!
UPDATE: In Java I often give class methods default/package visibility so they're less visible but still accessible to unit tests, looking for a python equivalent.
I don't think you can.
You'll need to return the inner function, or bind it elsewhere. It gets ugly though. You can bind the declared function to somewhere else, from inside the function:
import testmodule
testmodule.testfunc = make_exciting
Or you can return it, for example when a keyword-argument is given, such as export=True.
I already use this function to change some string to class object.
But now I have defined a new module. How can I implement the same functionality?
def str2class(str):
return getattr(sys.modules[__name__], str)
I want to think some example, but it is hard to think. Anyway, the main problem is maybe the file path problem.
If you really need an example, the GitHub code is here.
The Chain.py file needs to perform an auto action mechanism. Now it fails.
New approach:
Now I put all files under one filefold, and it works, but if I use the modules concept, it fails. So if the problem is in a module file, how can I change the string object to relative class object?
Thanks for your help.
You can do this by accessing the namespace of the module directly:
import module
f = module.__dict__["func_name"]
# f is now a function and can be called:
f()
One of the greatest things about Python is that the internals are accessible to you, and that they fit the language paradigm. A name (of a variable, class, function, whatever) in a namespace is actually just a key in a dictionary that maps to that name's value.
If you're interested in what other language internals you can play with, try running dir() on things. You'd be surprised by the number of hidden methods available on most of the objects.
You probably should write this function like this:
def str2class(s):
return globals()[s]
It's really clearer and works even if __name__ is set to __main__.
The python documentation states "execfile() cannot be used reliably to modify a function’s locals." on the page http://docs.python.org/2/library/functions.html#execfile
Can anyone provide any further details on this statement? The documentation is fairly minimal. The statement seems very contradictory to "If both dictionaries are omitted, the expression is executed in the environment where execfile() is called." which is also in the documentation. Is there a special case when excecfile is used within a function then execfile is then acting similar to a function in that it creates a new scoping level?
If I use execfile in a function such as
def testfun():
execfile('thefile.py',globals())
def testfun2():
print a
and there are objects created by the commands in 'thefile.py' (such as the object 'a'), how do I know if they are going to be local objects to testfun or global objects? So, in the function testfun2, 'a' will appear to be a global? If I omit globals() from the execfile statement, can anyone give a more detailed explanation why objects created by commands in 'thefile.py' are not available to 'testfun'?
In Python, the way names are looked up is highly optimized inside functions. One of the side effects is that the mapping returned by locals() gives you a copy of the local names inside a function, and altering that mapping does not actually influence the function:
def foo():
a = 'spam'
locals()['a'] = 'ham'
print(a) # prints 'spam'
Internally, Python uses the LOAD_FAST opcode to look up the a name in the current frame by index, instead of the slower LOAD_NAME, which would look for a local name (by name), then in the globals() mapping if not found in the first.
The python compiler can only emit LOAD_FAST opcodes for local names that are known at compile time; but if you allow the locals() to directly influence a functions' locals then you cannot know all the local names ahead of time. Nested functions using scoped names (free variables) complicates matters some more.
In Python 2, you can force the compiler to switch off the optimizations and use LOAD_NAME always by using an exec statement in the function:
def foo():
a = 'spam'
exec 'a == a' # a noop, but just the presence of `exec` is important
locals()['a'] = 'ham'
print(a) # prints 'ham'
In Python 3, exec has been replaced by exec() and the work-around is gone. In Python 3 all functions are optimized.
And if you didn't follow all this, that's fine too, but that is why the documentation glosses over this a little. It is all due to an implementation detail of the CPython compiler and interpreter that most Python users do not need to understand; all you need to know that using locals() to change local names in a function does not work, usually.
Locals are kind of weird in Python. Regular locals are generally accessed by index, not by name, in the bytecode (as this is faster), but this means that Python has to know all the local variables at compile time. And that means you can't add new ones at runtime.
Now, if you use exec in a function, in Python 2.x, Python knows not to do this and falls back to the slower method of accessing local variables by name, and you can make new ones programmatically. (This trick was removed in Python 3.) You'd think Python would also do this for execfile(), but it doesn't, because exec is a statement and execfile() is a function call, and the name execfile might not refer to the built-in function at runtime (it can be reassigned, after all).
What will happen in your example function? Well, try it and find out! As the documentation for execfile states, if you don't pass in a locals dict, the dict you pass in as globals will be used. You pass in globals() (your module's real global variables) so if it assigns to a, then a becomes a global.
Now you might try something like this:
def testfun():
execfile('thefile.py')
def testfun2():
print a
return testfun2
exec ""
The exec statement at the end forces testfun() to use the old-style name-based local variables. It doesn't even have to be executed, as it is not here; it just has to be in the function somewhere.
But this doesn't work either, because the name-based locals don't support nesting functions with free variables (a in this case). That functionality also requires Python know all the local variables at function definition time. You can't even define the above function—Python won't let you.
In short, trying to deal with local variables programmatically is a pain and the documentation is correct: execfile() cannot reliably be used to modify a function's locals.
A better solution, probably, is to just import the file as a module. You can do this within the function, then access values in the module the usual way.
def testfun():
import thefile
print thefile.a
If you won't know the name of the file to be imported until runtime, you can use __import__ instead. Also, you may need to modify sys.path to make sure the directory you want to import from is first in the path (and put it back afterward, probably).
You can also just pass in your own dictionary to execfile and afterward, access the variables from the executed file using myVarsDict['a'] and so on.
I am not really a programmer but a computational statistician, so I may understand complex algorithms but not simple programming constructs.
My original problem is to check within a function if a module function is callable. I looked around and decided to go for a try (call function) - except (import module) to make it simple. I'd love to search sys.mod for this but I am running in some identifiability problems.
My current problem is that there are many ways of importing a function from a module: import module will define the function as module.function but from module import function will define it as function. Not to mention from module import function as myfunction. Therefore the same function can be called in several different ways.
My question is: is there a unique "signature" for a function that can be traced if the module is loaded? It would be fantastic to have the actual call alias to it.
ps besides: mod is mathematical function and sys.mod returns a list of loaded modules, but python (2.7) does not complain when you shadow the built-in mod function by doing the following, from sys import mod. I find this a bit awkward - is there any way to avoid this sort of shadowing programatically?
My original problem is to check within a function if a module function is callable.
By definition, all functions are callable. This will test if an object is callable: http://docs.python.org/library/functions.html#callable
Therefore the same function can be called in several different ways.
Yes, but it will be the same object. You can just use f is g to test if f and g are the same object.
Update: Why would you need to use a unique ID? Seriously, don't do this. You have is for identity tests, and the __hash__ method to define the hash function applicable.
It would be fantastic to have the actual call alias to it.
Not sure at all what you mean, but I think you just want it to always be one object. Which it is already.
mod is mathematical function and sys.mod returns a list of loaded modules, but python (2.7) does not complain to from sys import mod. I find this a bit awkward?
Then don't do that. You know about the import ... as syntax. Also mod is not by default in the global namespace (the operator % is for that).
Finally, python does complain about your import line:
>>> from sys import mod
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: cannot import name mod
(Thanks to kindall for pointing this out).
Assume I have a module with the following:
def foo(): pass
bar = foo
You can easily see that they're the same functions by using is or id():
>>> import demo
>>> from demo import *
>>> demo.foo is foo
True
>>> id(demo.foo) == id(foo)
True
>>> demo.bar is foo
True
They all refer to the same code object, it's just stored under different names in the scope dictionary.
# define modulus f
def mod(a,b):
return b % a
print mod(5,2)
alias:
modulus=mod
print modulus(5,2)
this is pretty pythonic construct, and it is pretty intuitive for mathematicians
different ways of import serve to help you place a function into different "name space" for later use in your program, sometimes you wish to use a function a lot so you choose variant that is shorter to write.
you can also do something like:
myat=math.atanh
to make alias in another "name space"
and use it as:
myat(x)
as it would use math.atanh(x) - becomes shorter to write
Typical programmers approach would be define all you want to use and then use it. What you are trying in my belief is to do it "lazy" => import module when you need a function. That is why you wish to know if function is "callable".
Python is not functional programming language (e.g. like haskel) so that you can load or refer "on demand".
hope this helps.