I'll try to be clear, please do any questions you need.
I'm working on mezcla, just trying to make things a little bit more pythonic.
To be specific, in debug there's a function called assertion, which takes an expression and evaluates it, giving an error message which doesn't raise an exception.
It works giving to the function an expression, like
from debug import assertion
def func():
return 2+2==5,
def probe(expr):
print(assertion(expr))
probe(
func()
)
##OR alone like
assertion(
2+2
==5)
And it should take the expression itself and print it. I'm looking for a way to get and evaluate a multiline expression, just like icecream, for example, can do this way:
In [2]: ic(2+2
...: ==5
...: )
ic| 2+2
==5: False
I tried this massive code, it reads the expression from ipython history, or it reads the file and line with inspect and iterates line by line of the script looking for the closure parenthesis.
def read_line(filename, line_number):
"""Returns contents of FILENAME at LINE_NUMBER"""
# ex: "debugging" in read_line(os.path.join(os.getcwd(), "debug.py"), 3)
try:
with open(filename) as file:
line_contents = file.readlines()[line_number - 1]
except OSError:
line_contents = "<stdin>"
except:
line_contents = "???"
return line_contents
def multi_assert(assertion, line_number=1):
"""Handles multiline assertions until 10 lines"""
line = ""
counter = 0
#While count(just for prevent infinite loop)
while counter < 10:
counter += 1
#Reads line, appends it and compares number of ()
line += read_line(assertion, line_number)
if line.count("(") > line.count(")"):
line_number += 1
else:
return line
break
def assertion(expression, message=None):
"""Issue warning if EXPRESSION doesn't hold, along with optional MESSAGE
Note: This is a "soft assertion" that doesn't raise an exception (n.b., provided the test doesn't do so)"""
if (not expression):
# Get source information for failed assertion
(_frame, filename, line_number, _function, _context, _index) = inspect.stack()[1]
# Read statement in file and extract assertion expression
# Calls to multi_assert to handle multiline
statement = multi_assert(str(filename), line_number + 1)
# If statement is from stdin, tries to get assert from ipython history
if statement == "<stdin>" and _context != None:
try:
ip = get_ipython()
statement = str(ip.history_manager.get_tail(1, raw=True, include_latest=True))
except:
statement = str(_context).replace(")\\n']", "")
return statement
It works, but it's too heavy, hacky and not specially ellegant, so i'm looking for any other way that return the gived assertion even if is multiline. Any kind of suggestion will be accepted and appreciated. Thanks
Related
I've written simple code to handle just one case and correct the indentation (again simple and it relies on the user taking caution while using it) of a string containing a Python function declared using the def keyword and execute it.
def fix_index(string):
i=0;
t=string.find("def")+3;
string=string.replace(string[string.find("def"):t], "#")
while string.find(" ") != -1:
string = string.replace(" ", "")
i += 1
l=list(string);l[string.find(":")-i+2]+="$$$$"
return "".join(l).replace("$$$$", " ").replace("#", "def ").lstrip();
def switch(exp):
def exec(obj):
items = obj.items();
for k, v in items:
if(k==exp):
print(fix_index(v))
return eval(fix_index(v))();
return {"case":exec};
bread = "bread"
switch(bread)["case"]({
"cheese":
"""
def a():
print("cheese");
""",
"bread":
"""
def b():
print("bread");
"""
})
the output for the formatted function string:
C:\Users\User>python -u "c:\Users\User\folder\switch.py"
def b():
print("bread");
the error I'm getting:
Traceback (most recent call last):
File "c:\Users\User\folder\switch.py", line 27, in <module>
switch(bread)["case"]({
File "c:\Users\User\folder\switch.py", line 21, in exec
return eval(fix_index(v))();
File "<string>", line 1
def b():
^
SyntaxError: invalid syntax
I've also just realized I didn't name the function what I indented intended to (should've posted this when awake to avoid accidental pun).
Anyways what I fail to understand is what part in my produced string exactly contains "invalid syntax".
I'll appreciate any help.
if what you are looking for is to reproduce a switch statement, you can do it with the following function:
def switch(v): yield lambda *c: v in c
It simulates a switch statement using a single pass for loop with if/elif/else conditions that don't repeat the switching value:
for example:
for case in switch(x):
if case(3):
# ... do something
elif case(4,5,6):
# ... do something else
else:
# ... do some other thing
It can also be used in a more C style:
for case in switch(x):
if case(3):
# ... do something
break
if case(4,5,6):
# ... do something else
break
else:
# ... do some other thing
For your example, it could look like this:
meal = "bread"
for case in switch(meal):
if case("cheese"):
print("Cheese!")
elif case("bread"):
print("Bread!")
or this:
meal = "bread"
for case in switch(meal):
if case("cheese"):
print("Cheese!")
break
if case("bread"):
print("Bread!")
break
I am writing code for a math calculator, and am running into a problem. When I ask the user to input a function to derive, and they input a number and then x, like 3x, instead of 3*x. The code crashes because it cannot parse it. To get around this I have a function to sanitize it, but it won't even run for some reason.
class Derivatives:
def __init__(self):
x=sympy.Symbol('x')
self.func=self.clean_func((input("Enter f(x): ")))
print(self.func)
print(sympy.diff(self.func, x))
def clean_func(self, func):
clean_func=""
print("c")
for i in range(len(func)):
if func[i].isalpha or isinstance(func[i], int) or func[i] in self.math_symbols :
if func[i]== "^" :
clean_func+="**"
else:
clean_func+=func[i]
for c in range(48, 57) :
if func[i:i+1] == (chr(c) + 'x') :
clean_func+="*" #helps with parsing
return clean_func
and I get this error in a=Derivatives()
Enter f(x): 3x
Traceback (most recent call last):
File "<ipython-input-48-f4cd19afbde9>", line 1, in <module>
a=Derivatives()
File "/Volumes/PROJECTS/M_series.py", line 71, in __init__
x=sympy.Symbol('x')
File "/Users/thomastierney/anaconda/lib/python3.5/site-packages/sympy/core/function.py", line 1691, in diff
return Derivative(f, *symbols, **kwargs)
File "/Users/thomastierney/anaconda/lib/python3.5/site-packages/sympy/core/function.py", line 1018, in __new__
expr = sympify(expr)
File "/Users/thomastierney/anaconda/lib/python3.5/site-packages/sympy/core/sympify.py", line 324, in sympify
raise SympifyError('could not parse %r' % a, exc)
SympifyError: Sympify of expression 'could not parse '3x'' failed, because of exception being raised:
SyntaxError: invalid syntax (<string>, line 1)
A few problems:
Using func as the name of a string is misleading (Python allows first-order functions, therefore you would expect it to actually be a function); let's call it fn_str instead.
If clean_func worked properly, it would return a sympy-parseable string - which you then pass to diff without converting from a string to a function.
The construction
for i in range(len(mystring)):
do_something(mystring[i])
is more clearly written as
for ch in mystring:
do_something(ch)
isinstance(func[i], int) doesn't work because func[i] is a one-character string, not an integer. Try str.isdecimal() instead.
self.math_symbols is not defined.
Python strings are immutable; every time you do mystring += ch it actually creates an entirely new string. Because of this, it is much more efficient to append to a list of string-chunks and then "".join() them when finished.
for c in range(48, 57): Python ranges do not include the end value, ie this will produce 48, 49, 50, ... 56. This means chr(57) ie "9" is never tested for.
Similarly, string slices do not include the end offset, so func[i:i+1] is one character, exactly equivalent to func[i]. You meant func[i:i+2].
Also, something like 3. is a valid Python float, but your code will not catch 3.x. Doing proper tokenization instead of character-by-character comparison would catch that.
The class is not a Derivative; you are trying to stuff your program into a mislabeled class.
A cleaned-up version:
import string
import sympy
# This could easily be just a function;
# I made it a class to keep the namespace clean
class FnStrCleaner:
# valid characters
NUMCHARS = set(string.digits + ".")
VARCHARS = set(string.ascii_letters)
OPCHARS = set("()+-*/^")
LEGALCHARS = NUMCHARS | VARCHARS | OPCHARS
#classmethod
def clean(cls, fn_str):
# generator expression - skip nonlegal chars
good_chars = (ch for ch in fn_str if ch in cls.LEGALCHARS)
out = []
# simple FSM to process character stream
prev_num = False # preceding character was part of a number
for ch in good_chars:
if prev_num:
if ch in cls.NUMCHARS:
out.append(ch)
# prev_num = True
elif ch in cls.VARCHARS:
out.append('*' + ch)
prev_num = False
else: # ch in cls.OPCHARS
out.append(ch)
prev_num = False
else:
if ch in cls.NUMCHARS:
out.append(ch)
prev_num = True
elif ch in cls.VARCHARS:
out.append(ch)
# prev_num = False
else: # ch in cls.OPCHARS
out.append('**' if ch == '^' else ch)
# prev_num = False
return "".join(out)
def get_function(prompt, locals=None):
fn_str = input(prompt)
fn_str = FnStrCleaner.clean(fn_str)
return sympy.sympify(fn_str, locals=locals)
def main():
x = sympy.Symbol("x")
f = get_function("Enter f(x): ", {'x': x})
df = sympy.diff(f, x)
print("f(x) =", f)
print("f'(x) =", df)
if __name__ == "__main__":
main()
which runs like
Enter f(x): 29x^3 + 12x^2 - 9x + 5
f(x) = 29*x**3 + 12*x**2 - 9*x + 5
f'(x) = 87*x**2 + 24*x - 9
Let's say I have some python code in a string
code = """
a = 42
a
"""
and I exec that string of code:
result = exec(code)
Then result will always be None. Is there any way at all to get the value of the last expression evaluated? In this case, that would be 5, since a was the last expression.
EDIT: Here's another example of the functionality I'm asking about. Let's say we have the python code (stored in the variable code)
a = 100
sqrt(a)
Then how can I execute the code in such a way as to give me the result 10 - that is, sqrt(a)?
EDIT EDIT: A further example: the code I wish to exec is
function_a()
function_b()
function_c()
Is there any way I can define some kind of magic_exec function so that
magic_exec(code)
will provide me with the value of function_c()?
The request is certainly valid because I need such a function as well during the creation of a Python-based environment. I solved the problem with the following code that utilizes the Python ast mechanism:
def my_exec(script, globals=None, locals=None):
'''Execute a script and return the value of the last expression'''
stmts = list(ast.iter_child_nodes(ast.parse(script)))
if not stmts:
return None
if isinstance(stmts[-1], ast.Expr):
# the last one is an expression and we will try to return the results
# so we first execute the previous statements
if len(stmts) > 1:
exec(compile(ast.Module(body=stmts[:-1]), filename="<ast>", mode="exec"), globals, locals)
# then we eval the last one
return eval(compile(ast.Expression(body=stmts[-1].value), filename="<ast>", mode="eval"), globals, locals)
else:
# otherwise we just execute the entire code
return exec(script, globals, locals)
The code should be pretty self-explanatory, basically it
separate the script into multiple statements
if the last one is an expression, execute the first part as statements, and the last part as expression.
Otherwise execute the entire script as statements.
This doesn't get you the last evaluated value, but gets the whole list of local variables.
>>> loc = {}
>>> exec(code, {}, loc)
>>> loc
{'a': 42}
exec('a = 4')
print a % prints 4
>>> code = """
... a = 42
... b = 53"""
>>> exec(code)
>>> a
42
>>> b
53
Or if you're saying you don't know the last thing is b for instance, then you can have this:
code = """
a = 4
b = 12
abc_d=13
"""
t = re.findall(r'''.*?([A-Za-z0-9_]+)\s*?=.*?$''', code)
assert(len(t)==1)
print t[0] % prints 13
To be honest I can't say I'm very happy with this. It feels very hacky and I haven't tested it all that heavily. On the other hand I'm quite pleased with it. Was quite fun to do. Anyway, hope this helps you or at least comes close to what you want. locals() gives a dict so the output list order does not match the input order for the items that failed the first eval. If you don't want ';' as delimiters then you can change it to '\n'.
import math
def magic_exec(_command):
_command = _command.split(';')
_result = None
_before = list(locals()) # Get list of current local variables
for _code in _command:
_code = _code.strip() # .strip() prevent IndentationError
try:
if eval(_code) != None: # For functions with no return
_result = eval(_code)
except (NameError, SyntaxError):
try:
_before = list(locals())
exec(_code)
except NameError as e: # For undefined variables in _command
print("An Error Occurred with line ' {0} ' as was skipped: {1}".format(_code, e))
del _code # delete temp var _code
# Get new list of locals that didn't exist at the start
_after = [val for val in list(locals()) if val not in _before]
if _after:
return eval(_after[0])
else:
return _result
#Dummy class and functions
class Class1(object):
def __init__(self, x):
self._x = x
def get_val(self):
return self._x
def __repr__(self):
return type(self).__name__
def func1(x):
return x + x
def func2(x):
print(x*x)
if __name__ == '__main__':
code = \
"""
a = 42; a; v; y = 2; b = func1(5); s = 'Hello'; func2(10); c = 25; l = []; l.append('Value');
t = math.sqrt(c); pass; 20*10; print('TEST'); math.sqrt(c); d = Class1('World'); d.get_val();
def func3(x): return x ** 2; s = func3(15)
"""
values = magic_exec(code)
print(values)
I would like to add to user2283347's excellent answer that it works only up to Python 3.7. In Python 3.8 the signature of ast.Module.__init__ has changed. It now requires a second argument which in our case can be an empty list.
Details: ast.Module(body=stmts[:-1]) in
if len(stmts) > 1:
exec(compile(ast.Module(body=stmts[:-1]), filename="<ast>", mode="exec"), globals, locals)
has to be changed to
ast.Module(stmts[:-1], []) if you use Python 3.8 or above (note the second argument []). Otherwise the following TypeError will be raised:
TypeError: required field "type_ignores" missing from Module
Unfortunately this change is not very well documented. I found the solution after extensive Googling here: "IPython broken on 3.8-dev" .
I'm looking to write up a basic Bioinformatics course on Codeacademy. They have a nice interface for writing up a course, but it's a bit slow for testing, as one must save, then preview, then run.
So I'm looking to write up a little testing environment that mimics their one. How it appears to work is that the user-input code is read in to a function as a string, all str instances in the code are converted to unicode (I've just used regex for this) and then the code is executed with exec.
The tricky part seems to be when I want to incorporate the Submission Test.
Submission Tests need to return True, False, or a str, and are written as the body of a function. So for example:
A simplified version of what I'm looking to do:
# The submission test must be a function.
def test_code(code, CC, error):
# Use information from errors in student code
if error:
return "Yada yada %s" %error
# Use information in the raw student code
if len(code.split("\n")) is not 2:
return "This should be accomplished in 2 lines"
# Have direct access to variables from the student code
# I'd like to avoid params['y'] if possible.
try:
y
except NameError:
return "Please use the variable y"
if y is not 8:
return "Wrong! Check stuff"
# Use information from print output
if str(y) not in CC:
return "Remember to print your variable!"
return True
# Read in student code
student_code = """y = 8
print y
potato"""
# Catch print output
CC = StringIO.StringIO()
sys.stdout = CC
# Execute student code and catch errors
try:
exec student_code
except Exception as e:
error = e
# Start outputting to the terminal again
sys.stdout = sys.__stdout__
# Run the submission test
submission_test = test_code(student_code, CC.split("\n"), error)
# Output the result of the submission test
if submission_test is True:
print("Well done!")
elif submission_test is False:
print("Oops! You failed... Try again!")
else:
print(submission_test)
However, I can't seem to get the variables from exec code to pass through to the submission test function (test_code in this case).
I could just execute the code in the Submission Test, but I'd like to avoid that if possible, otherwise it will have to be added to each test, which just seems unpythonic!
Any help would be greatly appreciated :)
If you exec mystr in somedict, then somedict has a reference to every variable assigned during the execution of mystr as Python code. In addition, you can pass variables in this way, too.
>>> d = {'y': 3}
>>> exec "x = y" in d
>>> d['x']
3
You need to pass the dictionary you got from running the user code in, so that the submission check code can verify the values in it are appropriate.
It sounds like you want the code and the test to run in the same environment. Both are being submitted as strings, so perhaps the easiest way is to concatenate both and run exec on the combined string:
from __future__ import unicode_literals
def wrap_body(body):
indent = ' '*4
return 'def test():\n' + indent + body.replace('\n','\n'+indent)
code = '''
bool_1 = True
bool_2 = False
str_1 = 'Hello there!'
str_2 = "I hope you've noticed the apostrophe ;)"
'''
code_test = '''
try:
bool_1, bool_2, str_1, str_2
except NameError:
return "Please do not alter the variable names!"
if (bool_1 == bool_2 or str_1 == str_2):
return "Please ensure that all of your variables are different " \
"from one another"
if type(bool_1) != bool: return "bool_1 is incorrect"
if type(bool_2) != bool: return "bool_2 is incorrect"
if type(str_1) != unicode: return "str_1 is incorrect"
if type(str_2) != unicode: return "str_2 is incorrect"
return True
'''
code_test = wrap_body(code_test)
template = code + code_test
namespace = {}
try:
exec template in namespace
print(namespace['test']())
except Exception as err:
print(err)
Okay, my colleague figured this one out.
It uses an element of Devin Jeanpierre's answer.
We use the exec code in dictionary method, then pass the dictionary into the checking function, then, within the checking function we unpack the dictionary into the globals().
# The submission test must be a function.
def test_code(code, CC, error, code_vars):
# unpack the student code namespace into the globals()
globs = globals()
for var, val in code_vars.items():
globs[var] = val
# Use information from errors in student code
if error:
return "Yada yada %s" %error
# Use information in the raw student code
if len(code.split("\n")) is not 2:
return "This should be accomplished in 2 lines"
# Have direct access to variables from the student code
# I'd like to avoid params['y'] if possible.
try:
y
except NameError:
return "Please use the variable y"
if y is not 8:
return "Wrong! Check stuff"
# Use information from print output
if str(y) not in CC:
return "Remember to print your variable!"
return True
# Read in student code
student_code = """y = 8
print y
potato"""
# Catch print output
CC = StringIO.StringIO()
sys.stdout = CC
# create the namespace for the student code
code_vars = {}
# Execute student code and catch errors
try:
# execute the student code in the created namespace
exec student_code in code_vars
except Exception as e:
error = e
# Start outputting to the terminal again
sys.stdout = sys.__stdout__
# Run the submission test
submission_test = test_code(student_code, CC.split("\n"), error, code_vars)
# Output the result of the submission test
if submission_test is True:
print("Well done!")
elif submission_test is False:
print("Oops! You failed... Try again!")
else:
print(submission_test)
I'm writing a Python generator which looks like "cat". My specific use case is for a "grep like" operation. I want it to be able to break out of the generator if a condition is met:
summary={}
for fn in cat("filelist.dat"):
for line in cat(fn):
if line.startswith("FOO"):
summary[fn] = line
break
So when break happens, I need the cat() generator to finish and close the file handle to fn.
I have to read 100k files with 30 GB of total data, and the FOO keyword happens in the header region, so it is important in this case that the cat() function stops reading the file ASAP.
There are other ways I can solve this problem, but I'm still interested to know how to get an early exit from a generator which has open file handles. Perhaps Python cleans them up right away and closes them when the generator is garbage collected?
Thanks,
Ian
Generators have a close method that raises GeneratorExit at the yield statement. If you specifically catch this exception, you can run some tear-down code:
import contextlib
with contextlib.closing( cat( fn ) ):
...
and then in cat:
try:
...
except GeneratorExit:
# close the file
If you'd like a simpler way to do this (without using the arcane close method on generators), just make cat take a file-like object instead of a string to open, and handle the file IO yourself:
for filename in filenames:
with open( filename ) as theFile:
for line in cat( theFile ):
...
However, you basically don't need to worry about any of this, because the garbage collection will handle it all. Still,
explicit is better than implicit
By implementing the context protocol and the iterator protocol in the same object, you can write pretty sweet code like this:
with cat("/etc/passwd") as lines:
for line in lines:
if "mail" in line:
print line.strip()
break
This is a sample implementation, tested with Python 2.5 on a Linux box. It reads the lines of /etc/passwd until it finds the one for user audio, and then stops:
from __future__ import with_statement
class cat(object):
def __init__(self, fname):
self.fname = fname
def __enter__(self):
print "[Opening file %s]" % (self.fname,)
self.file_obj = open(self.fname, "rt")
return self
def __exit__(self, *exc_info):
print "[Closing file %s]" % (self.fname,)
self.file_obj.close()
def __iter__(self):
return self
def next(self):
line = self.file_obj.next().strip()
print "[Read: %s]" % (line,)
return line
def main():
with cat("/etc/passwd") as lines:
for line in lines:
if "mail" in line:
print line.strip()
break
if __name__ == "__main__":
import sys
sys.exit(main())
Or even simpler:
with open("/etc/passwd", "rt") as f:
for line in f:
if "mail" in line:
break
File objects implement the iterator protocol (see http://docs.python.org/library/stdtypes.html#file-objects)
Please also consider this example:
def itertest():
try:
for i in xrange(1000):
print i
yield i
finally:
print 'finally'
x = itertest()
for i in x:
if i > 2:
break
print 'del x'
del x
print 'exit'
0
1
2
3
del x
finally
exit
It shows that finally is run after the iterator is cleaned up. I think __del__(self) is calling self.close(), see also here: https://docs.python.org/2.7/reference/expressions.html#generator.close
There seems to be another possibility using try..finally (tested on Python 2.7.6):
def gen():
i = 0
try:
while True:
print 'yield %i' % i
yield i
i += 1
print 'will never get here'
finally:
print 'done'
for i in gen():
if i > 1:
print 'break'
break
print i
Gives me the following printout:
yield 0
0
yield 1
1
yield 2
break
done