I want to create a Python script that takes in a string representation of a dictionary and outputs a list of tuples representing the items of the dictionary. The rub is, I want it to take in variables that have not been defined. An example illustrates this best:
Input: {'test': test}
Output: [('test': test)]
I've created some code to read in the dictionary and define the variables that haven't yet been defined, but when I eval() the result, it substitutes in the actual value of the variable rather than the variable name.
Here is the code:
import sys
import re
if __name__ == "__main__":
instr = sys.argv[1]
success = False
while not success:
try:
indict = eval(instr)
success = True
except NameError, e:
defname = re.search("name '([^\']*)' is not defined", str(e))
locals()[defname.group(1)] = 0
print indict
I was hoping, for the above defined input value, that the dict it printed out would match the input string perfectly, but instead it has substituted in the value of 0 for test. Thus the script prints out:
{'test': 0}
I have tried ast.literal_eval, but it throws a ValueError: malformed string for any variable name in the literal, even if it's been defined.
Thus, my question is: Is there a way to convert the input dictionary without losing the variable names?
I would use the ast module, but not literal_eval. Walk the tree, when you find a variable reference, just insert the name of the variable into the output.
You can trick eval into giving you what you want:
class Reflector(object):
def __getitem__(self, name):
return name
s = "{'test':test}"
print eval(s, globals(), Reflector())
produces:
{'test': 'test'}
The usual caveats about the dangers of eval hold though: if you have any doubts about the source of these strings, you are opening yourself up for hacking.
I think the problem you are having is that you want to evaluate the name of a variable instead of the value of that variable. In particular, {'test':test} would never be the output of printing a dictionary because test is not a value in python, it is possibly the name of a variable. The closest you can get is assigning
locals()[defname.group(1)] = defname.group(1)
to get
{'test':'test'}
I think a reasonable workaround is to pass eval something that preserves undefined variables in a useful way. You can pass any dict subclass to eval, and you can override the __missing__ method of dict to return an 'undefined variable' object:
>>> class Var(object):
... def __init__(self, name):
... self.name = name
... def __repr__(self):
... return "Var(%r)" % (self.name,)
...
>>> class UndefSubst(dict):
... def __missing__(self, key):
... return Var(key)
...
>>> foo = 'bar'
>>> eval("{'baz': quux, 1: foo}", UndefSubst(locals()))
{1: 'bar', 'baz': Var('quux')}
I solved this problem by substituting in unique strings for the names of the variables. This produces the desired result:
import sys
import re
if __name__ == "__main__":
instr = sys.argv[1]
subs = []
success = False
while not success:
try:
indict = eval(instr)
success = True
except NameError, e:
defname = re.search("name '([^\']*)' is not defined", str(e)).group(1)
subs.append(defname)
locals()[defname] = '//substitute{%d}' % (len(subs) - 1)
outlist = indict.items()
outstr = str(outlist)
for i, sub in enumerate(subs):
outstr = outstr.replace("'//substitute{%d}'" % i, sub)
print outstr
Input: {"test": test}
Output: [('test', test)]
Input: {"test": [{"test": test}, {word: "word"}]}
Output: [('test', [{'test': test}, {word: 'word'}])] (note that this is desirable, I don't want the inner dictionary zipped).
The minor downside is that I can never have whatever substitution string I elect to use anywhere in the input string, but I am reasonably certain this will not be an issue for what I wish to use this script for.
Related
I have been investigating and found out that using if in is the fastest compare to ->
benchmark
and I have been trying to create a function where I can pass arguments on what path I want the if in will follow e.g.
def main():
d = {"foo": "spam"}
if "bar" in d:
if "eggs" in d["bar"]:
d["bar"]["eggs"]
else:
{}
else:
{}
But instead of having a long code, I was trying to do a function where I can pass argument e.g. get_path(json_data, 'foo', 'eggs') which would try to do something similar to the code above and return if value found else return empty.
My question is how can I create a function where we can pass argument to do the if in checks and return the value if it's found?
You could pass your keys as tuple/list:
def main(data, keys):
for k in keys:
if k not in data:
return {}
data = data[k]
return data
d = {"foo": "spam", "bar": {"eggs": "HAM!"}}
print(main(d, ('bar', 'eggs')))
Out:
HAM!
This is a nice little problem that has a fairly easy solution as long as everything is dicts:
def get_path(data, *path):
node = data
for step in path:
if step in node:
node = node[step]
else:
return {} # None might be more appropriate here
return node
note that it won't work quite right if you encounter a list along the way: although lists support [] and they support in, in means something different to them ("is this value found", rather than "is this key found"), so the test generally won't succeed.
I have a function func :
def func(a):
print a
func.__name_ = "My_Function"
To get the name of the function I will just do func.__name__.
But is there a way to call func() by using its name i.e. "My_Function"?
Edit: There are going to be large number of functions for which this has to be done. Is there any better solution apart from keeping a mapping of function.name to the function. ?
It will be much easier to use a dictionary:
def func():
print('func')
func_dict = {"My_Function": func}
func_dict["My_Function"]()
# 'func'
Assuming that you want to access functions that are defined in global scope (not part of any class/object):
def foo():
print "foo"
def bar():
print "bar"
foo.__name__ = "1"
bar.__name__ = "2"
# locals().items() will return the current local symbol table, which is an iterable and
# it contains the globally defined functions.
symbol_table = locals().items()
# we will generate a list of functions that their __name__ == "1"
one_functions = [value for key, value in symbol_table if callable(value) and value.__name__ == "1"]
# now we can call the first item on the list - foo()
one_functions[0]()
I'm trying to write a function right now, and its purpose is to go through an object's __dict__ and add an item to a dictionary if the item is not a function.
Here is my code:
def dict_into_list(self):
result = {}
for each_key,each_item in self.__dict__.items():
if inspect.isfunction(each_key):
continue
else:
result[each_key] = each_item
return result
If I'm not mistaken, inspect.isfunction is supposed to recognize lambdas as functions as well, correct? However, if I write
c = some_object(3)
c.whatever = lambda x : x*3
then my function still includes the lambda. Can somebody explain why this is?
For example, if I have a class like this:
class WhateverObject:
def __init__(self,value):
self._value = value
def blahblah(self):
print('hello')
a = WhateverObject(5)
So if I say print(a.__dict__), it should give back {_value:5}
You are actually checking if each_key is a function, which most likely is not. You actually have to check the value, like this
if inspect.isfunction(each_item):
You can confirm this, by including a print, like this
def dict_into_list(self):
result = {}
for each_key, each_item in self.__dict__.items():
print(type(each_key), type(each_item))
if inspect.isfunction(each_item) == False:
result[each_key] = each_item
return result
Also, you can write your code with dictionary comprehension, like this
def dict_into_list(self):
return {key: value for key, value in self.__dict__.items()
if not inspect.isfunction(value)}
I can think of an easy way to find the variables of an object through the dir and callable methods of python instead of inspect module.
{var:self.var for var in dir(self) if not callable(getattr(self, var))}
Please note that this indeed assumes that you have not overrided __getattr__ method of the class to do something other than getting the attributes.
I'm trying to define variable inside function. vars() shows variable is created, but gives me NameError: exception. What am I doing wrong?
def a(str1):
vars() [str1] = 1
print vars()
print b
a('b')
output:
{'str1': 'b', 'b': 1}
exception:
NameError: global name 'b' is not defined
You're invoking undefined behaviour. From the documentation of vars():
Note The returned dictionary should not be modified: the effects on the corresponding symbol table are undefined.
Other answers give possible solutions.
Your code works for me. Perhaps you should try an alternative approach:
exec(str1 + '=1')
This will execute b=1
If you don't understand why a construct doesn't work, neither will the next person who has to read your code. If you mean
b = 1
you should say that. In this case vars() gives you access to the function local dictionary so your code is equivalent to
def a():
b = 1
where b is local to a and evaporates when it goes out of scope upon exit from a.
premature optimization is the root of many people's attempt to second-guess Python
from itertools import izip
import timeit
import msw.wordlist
def zip_list(p):
"""construct a dictionary of length 100 from a list of strings"""
return dict(zip(p[:100], p[100:]))
def izip_list(p):
"""as zip_list but avoids creating a new list to feed to dict()"""
return dict(izip(p[:100], p[100:]))
def pass_list(p):
"""take 100 elements of a list and do nothing"""
for i in p[:100]:
pass
def exec_pass_list(p):
"""exec() a no-op 100 times"""
for i in xrange(100):
exec('pass')
# returns a list of 64'000 unique lowercase dictionary words for tests
words = msw.wordlist.Wordlist()
words.shuffle()
words = words[:200]
print 'words', words[:5], '...'
for func in ['zip_list', 'izip_list', 'pass_list', 'exec_pass_list']:
t = timeit.Timer('%s(words)' % func,
'from __main__ import words, %s' % func)
print func, t.repeat(number=10**5)
which yields:
words ['concatenated', 'predicament', 'shtick', 'imagine', 'stationing'] ...
zip_list [1.8603439331054688, 1.8597819805145264, 1.8571949005126953]
izip_list [1.5500969886779785, 1.5501470565795898, 1.5536530017852783]
pass_list [0.26778006553649902, 0.26837801933288574, 0.26767921447753906]
exec_pass_list [74.459679126739502, 75.221366882324219, 77.538936853408813]
I didn't bother trying implement whatever the OP was trying to do because being 50 times slower to not construct a dictionary sort makes further testing kinda stupid.
I have a variable x in python. How can i find the string 'x' from the variable. Here is my attempt:
def var(v,c):
for key in c.keys():
if c[key] == v:
return key
def f():
x = '321'
print 'Local var %s = %s'%(var(x,locals()),x)
x = '123'
print 'Global var %s = %s'%(var(x,locals()),x)
f()
The results are:
Global var x = 123
Local var x = 321
The above recipe seems a bit un-pythonesque. Is there a better/shorter way to achieve the same result?
Q: I have a variable x in python. How can i find the string 'x' from the variable.
A: If I am understanding your question properly, you want to go from the value of a variable to its name. This is not really possible in Python.
In Python, there really isn't any such thing as a "variable". What Python really has are "names" which can have objects bound to them. It makes no difference to the object what names, if any, it might be bound to. It might be bound to dozens of different names, or none.
Consider this example:
foo = 1
bar = foo
baz = foo
Now, suppose you have the integer object with value 1, and you want to work backwards and find its name. What would you print? Three different names have that object bound to them, and all are equally valid.
print(bar is foo) # prints True
print(baz is foo) # prints True
In Python, a name is a way to access an object, so there is no way to work with names directly. You might be able to search through locals() to find the value and recover a name, but that is at best a parlor trick. And in my above example, which of foo, bar, and baz is the "correct" answer? They all refer to exactly the same object.
P.S. The above is a somewhat edited version of an answer I wrote before. I think I did a better job of wording things this time.
I believe the general form of what you want is repr() or the __repr__() method of an object.
with regards to __repr__():
Called by the repr() built-in function
and by string conversions (reverse
quotes) to compute the “official”
string representation of an object.
See the docs here: object.repr(self)
stevenha has a great answer to this question. But, if you actually do want to poke around in the namespace dictionaries anyway, you can get all the names for a given value in a particular scope / namespace like this:
def foo1():
x = 5
y = 4
z = x
print names_of1(x, locals())
def names_of1(var, callers_namespace):
return [name for (name, value) in callers_namespace.iteritems() if var is value]
foo1() # prints ['x', 'z']
If you're working with a Python that has stack frame support (most do, CPython does), it isn't required that you pass the locals dict into the names_of function; the function can retrieve that dictionary from its caller's frame itself:
def foo2():
xx = object()
yy = object()
zz = xx
print names_of2(xx)
def names_of2(var):
import inspect
callers_namespace = inspect.currentframe().f_back.f_locals
return [name for (name, value) in callers_namespace.iteritems() if var is value]
foo2() # ['xx', 'zz']
If you're working with a value type that you can assign a name attribute to, you can give it a name, and then use that:
class SomeClass(object):
pass
obj = SomeClass()
obj.name = 'obj'
class NamedInt(int):
__slots__ = ['name']
x = NamedInt(321)
x.name = 'x'
Finally, if you're working with class attributes and you want them to know their names (descriptors are the obvious use case), you can do cool tricks with metaclass programming like they do in the Django ORM and SQLAlchemy declarative-style table definitions:
class AutonamingType(type):
def __init__(cls, name, bases, attrs):
for (attrname, attrvalue) in attrs.iteritems():
if getattr(attrvalue, '__autoname__', False):
attrvalue.name = attrname
super(AutonamingType,cls).__init__(name, bases, attrs)
class NamedDescriptor(object):
__autoname__ = True
name = None
def __get__(self, instance, instance_type):
return self.name
class Foo(object):
__metaclass__ = AutonamingType
bar = NamedDescriptor()
baaz = NamedDescriptor()
lilfoo = Foo()
print lilfoo.bar # prints 'bar'
print lilfoo.baaz # prints 'baaz'
There are three ways to get "the" string representation of an object in python:
1: str()
>>> foo={"a":"z","b":"y"}
>>> str(foo)
"{'a': 'z', 'b': 'y'}"
2: repr()
>>> foo={"a":"z","b":"y"}
>>> repr(foo)
"{'a': 'z', 'b': 'y'}"
3: string interpolation:
>>> foo={"a":"z","b":"y"}
>>> "%s" % (foo,)
"{'a': 'z', 'b': 'y'}"
In this case all three methods generated the same output, the difference is that str() calls dict.__str__(), while repr() calls dict.__repr__(). str() is used on string interpolation, while repr() is used by Python internally on each object in a list or dict when you print the list or dict.
As Tendayi Mawushe mentiones above, string produced by repr isn't necessarily human-readable.
Also, the default implementation of .__str__() is to call .__repr__(), so if the class does not have it's own overrides to .__str__(), the value returned from .__repr__() is used.