I have used the following script to run the script directly and just to make a bash command line for running it outside the script (e.g. job scheduler).
def qsubcommand(func):
def wrapper(*args, **kwargs):
if kwargs.get('test', False):
cmdl = ' '.join(['this.py', func.__name__, *map(str, args)])
return cmdl
else:
return func(*args, **kwargs)
return wrapper
#qsubcommand
def calculate(value1, value2):
# do something
if __name__ == '__main__':
if len(sys.argv) > 1:
func, args = sys.argv[1], sys.argv[2:]
if func in locals().keys():
locals()[func](*args)
else:
raise NotImplementedError
I have a lot of functions like 'calculate'.
I'm working with the script for running and testing a program.
# When I want to run directly:
>>> calculate(4, 5)
# When I want to just print command line:
>>> calculate(4, 5, test=True)
'this.py calculate 4 5'
However, I want to use it in a context-dependent manner like below.
# When I want to run directly:
>>> test = False
>>> calculate(4, 5)
# When I want to just print command line:
>>> test = True
>>> calculate(4, 5)
'this.py calculate 4 5'
How can I modify to let the function recognize the variable outside the scope.
Is it possible to access a variable outside the function?
Thank you for your kind answers in advance.
Just put this on the part of the function where you want to check the variable:
if 'test' in globals() and test:
# do test
else:
# do normal
Functions can always access variables which are outside the function scope, they just can't edit them if you don't use the global keyword.
Related
I'm making a programming language with Python. I want to be able to run functions by typing:
task $function:
log "Hey!"
and then you can call it by typing:
$function
Right now, I have this code:
def runFunction(self, f):
## Here's where I need help. How will you make a function that runs a function that doesn't exist in
## the code?
...
elif i == 'task':
self.string = line[5:]
self.output = self.string.replace('{', '')
self.function = self.output.replace('}', '')
...
elif i not in self.keywords and '$' in i:
if i == self.function:
self.runFunction(self.function)
What should I do to make a runFunction() function?
What you could do?
You could use python built in function called eval() which lets you evaluate expressions dynamically.
Solution
def runFunction(self, f):
eval(f)
Where f has to be a string
And if you want to create functions:
You could use exec() which can evaluate a string.
Demonstration:
exec("def myFunction(a,b): return a + b")
and use it like
eval("myFunction(1,2)")
I'm trying to build a decorator-based dispatcher, such as you find used by Flask or Pyramid. I got something that works, but ran into a bit of a catch-22. The following code works, but only because foo() gets executed and sets the .mq_path-attribute. When starting the application and building a list of the dispatchable functions no attributes are thus set yet. I want to execute foo() driven by events.
I could "manually" prepare a list of functions ahead and updated as I add functions, but I enjoy the way Flask works, by just adding a decorator to a function that handles an URL (or in this case a MQ path).
list_of_paths = []
path_dispatcher = {}
def handle_mq(path):
def decorator(fn):
def decorated(*args,**kwargs):
decorated.mq_path = path
print "Hello from the handle_mq() decorator, your path is: {0}".format(path)
return fn(*args,**kwargs)
return decorated
return decorator
#handle_mq('/some/path')
def foo():
print "foo!"
foo() # <- this code only works if I first call the decorated function
for k, v in globals().items():
if hasattr(v, 'mq_path'):
list_of_paths.append(v.mq_path)
path_dispatcher[v.mq_path] = v
print list_of_paths
print path_dispatcher
path_dispatcher['/some/path']()
So basically the question is, how to gather a list of the decorated functions before they are first executed?
I'm on Python 2.7.
I found the answer!
list_of_paths = []
path_dispatcher = {}
def handle_mq(path):
def decorator(fn):
list_of_paths.append(path)
path_dispatcher[path] = fn
def decorated(*args,**kwargs):
print "Hello from handl_mq decorator, your path is: {0}".format(path)
return fn(*args,**kwargs)
return decorated
return decorator
#handle_mq('/some/path')
def foo():
print "foo!"
print list_of_paths
print path_dispatcher
path_dispatcher['/some/path']()
I have a python function that runs other functions.
def main():
func1(a,b)
func2(*args,*kwargs)
func3()
Now I want to apply exceptions on main function. If there was an exception in any of the functions inside main, the function should not stop but continue executing next line. In other words, I want the below functionality
def main():
try:
func1()
except:
pass
try:
func2()
except:
pass
try:
func3()
except:
pass
So is there any way to loop through each statement inside main function and apply exceptions on each line.
for line in main_function:
try:
line
except:
pass
I just don't want to write exceptions inside the main function.
Note : How to prevent try catching every possible line in python? this question comes close to solving this problem, but I can't figure out how to loop through lines in a function.
If you have any other way to do this other than looping, that would help too.
What you want is on option that exists in some languages where an exception handler can choose to proceed on next exception. This used to lead to poor code and AFAIK has never been implemented in Python. The rationale behind is that you must explicitely say how you want to process an exception and where you want to continue.
In your case, assuming that you have a function called main that only calls other function and is generated automatically, my advice would be to post process it between its generation and its execution. The inspect module can even allow to do it at run time:
def filter_exc(func):
src = inspect.getsource(func)
lines = src.split('\n')
out = lines[0] + "\n"
for line in lines[1:]:
m = re.match('(\s*)(.*)', line)
lead, text = m.groups()
# ignore comments and empty lines
if not (text.startswith('#') or text.strip() == ""):
out += lead + "try:\n"
out += lead + " " + text + "\n"
out += lead + "except:\n" + lead + " pass\n"
return out
You can then use the evil exec (the input in only the source from your function):
exec(filter_exc(main)) # replaces main with the filtered version
main() # will ignore exceptions
After your comment, you want a more robust solution that can cope with multi line statements and comments. In that case, you need to actually parse the source and modify the parsed tree. ast module to the rescue:
class ExceptFilter(ast.NodeTransformer):
def visit_Expr(self, node):
self.generic_visit(node)
if isinstance(node.value, ast.Call): # filter all function calls
# print(node.value.func.id)
# use a dummy try block
n = ast.parse("""try:
f()
except:
pass""").body[0]
n.body[0] = node # make the try call the real function
return n # and use it
return node # keep other nodes unchanged
With that example code:
def func1():
print('foo')
def func2():
raise Exception("Test")
def func3(x):
print("f3", x)
def main():
func1()
# this is a comment
a = 1
if a == 1: # this is a multi line statement
func2()
func3("bar")
we get:
>>> node = ast.parse(inspect.getsource(main))
>>> exec(compile(ExceptFilter().visit(node), "", mode="exec"))
>>> main()
foo
f3 bar
In that case, the unparsed node(*) write as:
def main():
try:
func1()
except:
pass
a = 1
if (a == 1):
try:
func2()
except:
pass
try:
func3('bar')
except:
pass
Alternatively it is also possible to wrap every top level expression:
>>> node = ast.parse(inspect.getsource(main))
>>> for i in range(len(node.body[0].body)): # process top level expressions
n = ast.parse("""try:
f()
except:
pass""").body[0]
n.body[0] = node.body[0].body[i]
node.body[0].body[i] = n
>>> exec(compile(node, "", mode="exec"))
>>> main()
foo
f3 bar
Here the unparsed tree writes:
def main():
try:
func1()
except:
pass
try:
a = 1
except:
pass
try:
if (a == 1):
func2()
except:
pass
try:
func3('bar')
except:
pass
BEWARE: there is an interesting corner case if you use exec(compile(... in a function. By default exec(code) is exec(code, globals(), locals()). At top level, local and global dictionary is the same dictionary, so the top level function is correctly replaced. But if you do the same in a function, you only create a local function with the same name that can only be called from the function (it will go out of scope when the function will return) as locals()['main'](). So you must either alter the global function by passing explicitely the global dictionary:
exec(compile(ExceptFilter().visit(node), "", mode="exec"), globals(), globals())
or return the modified function without altering the original one:
def myfun():
# print(main)
node = ast.parse(inspect.getsource(main))
exec(compile(ExceptFilter().visit(node), "", mode="exec"))
# print(main, locals()['main'], globals()['main'])
return locals()['main']
>>> m2 = myfun()
>>> m2()
foo
f3 bar
(*) Python 3.6 contains an unparser in Tools/parser, but a simpler to use version exists in pypi
You could use a callback, like this:
def main(list_of_funcs):
for func in list_of_funcs:
try:
func()
except Exception as e:
print(e)
if __name__ == "__main__":
main([func1, func2, func3])
I am trying to learn decorators by following the 'python decorators in 12 steps'. http://simeonfranklin.com/blog/2012/jul/1/python-decorators-in-12-steps/
I tried replicating one of the In one of the code examples:
def outer(some_func):
def inner():
print ("before some_func")
ret = some_func()
return ret + 1
return inner
def foo():
return 1
def main():
decorated = outer(foo)
decorated()
if __name__ == "__main__": main()
This results in :
before some_func
In the example under Decorators!
>>> def outer(some_func):
... def inner():
... print ("before some_func")
... ret = some_func() # 1
... return ret + 1
... return inner
>>> def foo():
... return 1
>>> decorated = outer(foo) # 2
>>> decorated()
Returns:
before some_func
2
The main difference is that in the example, they are using Python running directly from cmd and I am using the Sublime text with python build as well as using a main() function. In my mind, I feel like these are doing the exact same thing. Is there something different between running code in the cmd versus with sublime that I am not getting?
The interactive Python interpreter automatically prints the results of a line if it is not assigned to a variable. This is useful for debugging. For example, if you call foo(), it will automatically print 1. The call below would not result in any extra prints in the interactive interpreter.
>>> result = decorated()
Let's say I have some python code in a string
code = """
a = 42
a
"""
and I exec that string of code:
result = exec(code)
Then result will always be None. Is there any way at all to get the value of the last expression evaluated? In this case, that would be 5, since a was the last expression.
EDIT: Here's another example of the functionality I'm asking about. Let's say we have the python code (stored in the variable code)
a = 100
sqrt(a)
Then how can I execute the code in such a way as to give me the result 10 - that is, sqrt(a)?
EDIT EDIT: A further example: the code I wish to exec is
function_a()
function_b()
function_c()
Is there any way I can define some kind of magic_exec function so that
magic_exec(code)
will provide me with the value of function_c()?
The request is certainly valid because I need such a function as well during the creation of a Python-based environment. I solved the problem with the following code that utilizes the Python ast mechanism:
def my_exec(script, globals=None, locals=None):
'''Execute a script and return the value of the last expression'''
stmts = list(ast.iter_child_nodes(ast.parse(script)))
if not stmts:
return None
if isinstance(stmts[-1], ast.Expr):
# the last one is an expression and we will try to return the results
# so we first execute the previous statements
if len(stmts) > 1:
exec(compile(ast.Module(body=stmts[:-1]), filename="<ast>", mode="exec"), globals, locals)
# then we eval the last one
return eval(compile(ast.Expression(body=stmts[-1].value), filename="<ast>", mode="eval"), globals, locals)
else:
# otherwise we just execute the entire code
return exec(script, globals, locals)
The code should be pretty self-explanatory, basically it
separate the script into multiple statements
if the last one is an expression, execute the first part as statements, and the last part as expression.
Otherwise execute the entire script as statements.
This doesn't get you the last evaluated value, but gets the whole list of local variables.
>>> loc = {}
>>> exec(code, {}, loc)
>>> loc
{'a': 42}
exec('a = 4')
print a % prints 4
>>> code = """
... a = 42
... b = 53"""
>>> exec(code)
>>> a
42
>>> b
53
Or if you're saying you don't know the last thing is b for instance, then you can have this:
code = """
a = 4
b = 12
abc_d=13
"""
t = re.findall(r'''.*?([A-Za-z0-9_]+)\s*?=.*?$''', code)
assert(len(t)==1)
print t[0] % prints 13
To be honest I can't say I'm very happy with this. It feels very hacky and I haven't tested it all that heavily. On the other hand I'm quite pleased with it. Was quite fun to do. Anyway, hope this helps you or at least comes close to what you want. locals() gives a dict so the output list order does not match the input order for the items that failed the first eval. If you don't want ';' as delimiters then you can change it to '\n'.
import math
def magic_exec(_command):
_command = _command.split(';')
_result = None
_before = list(locals()) # Get list of current local variables
for _code in _command:
_code = _code.strip() # .strip() prevent IndentationError
try:
if eval(_code) != None: # For functions with no return
_result = eval(_code)
except (NameError, SyntaxError):
try:
_before = list(locals())
exec(_code)
except NameError as e: # For undefined variables in _command
print("An Error Occurred with line ' {0} ' as was skipped: {1}".format(_code, e))
del _code # delete temp var _code
# Get new list of locals that didn't exist at the start
_after = [val for val in list(locals()) if val not in _before]
if _after:
return eval(_after[0])
else:
return _result
#Dummy class and functions
class Class1(object):
def __init__(self, x):
self._x = x
def get_val(self):
return self._x
def __repr__(self):
return type(self).__name__
def func1(x):
return x + x
def func2(x):
print(x*x)
if __name__ == '__main__':
code = \
"""
a = 42; a; v; y = 2; b = func1(5); s = 'Hello'; func2(10); c = 25; l = []; l.append('Value');
t = math.sqrt(c); pass; 20*10; print('TEST'); math.sqrt(c); d = Class1('World'); d.get_val();
def func3(x): return x ** 2; s = func3(15)
"""
values = magic_exec(code)
print(values)
I would like to add to user2283347's excellent answer that it works only up to Python 3.7. In Python 3.8 the signature of ast.Module.__init__ has changed. It now requires a second argument which in our case can be an empty list.
Details: ast.Module(body=stmts[:-1]) in
if len(stmts) > 1:
exec(compile(ast.Module(body=stmts[:-1]), filename="<ast>", mode="exec"), globals, locals)
has to be changed to
ast.Module(stmts[:-1], []) if you use Python 3.8 or above (note the second argument []). Otherwise the following TypeError will be raised:
TypeError: required field "type_ignores" missing from Module
Unfortunately this change is not very well documented. I found the solution after extensive Googling here: "IPython broken on 3.8-dev" .