Develop Reference

Trigger f-string parse on python string in variable

This question comes from handling jupyter magics, but can be expressed in a more simple way. Given a string s = "the key is {d['key']}" and a dictionary d = {'key': 'val'}, we want to parse the string.
The old method would be .format(), which will raise an error - it doesn't handle dictionary keys.
"the key is {d['key']}".format(d=d) # ERROR
I thought the only way around was to transform the dictionary to an object (explained here or here).
"the key is {d.key}".format(obj(d))
But Martijn explained nicely that you can simply leave out the quotes to get it working:
"the key is {d[key]}".format(d=d)
Still the new method f'string' does handle dictionary keys ain an intuitive python manner:
f"the key is {d['key']}"
It also handles functions - something .format also cannot handle.
f"this means {d['key'].lower()}"
Although we now know that you can do it with .format, I am still wondering about the original question: given s and d, how do you force a f'string' parse of s? I added another example with a function inside the curly brackets, that .format can also not handle and f'string' would be able to solve.
Is there some function .fstring() or method available? What does Python use internally?

String formatting can handle most string dictionary keys just fine, but you need to remove the quotes:
"the key is {d[key]}".format(d=d)
Demo:
>>> d = {'key': 'val'}
>>> "the key is {d[key]}".format(d=d)
'the key is val'
str.format() syntax isn't quite the same thing as Python expression syntax (which is what f-strings mostly support).
From the Format String Syntax documentation:
field_name ::= arg_name ("." attribute_name | "[" element_index "]")*
[...]
element_index ::= digit+ | index_string
index_string ::= <any source character except "]"> +
and
[A]n expression of the form '[index]' does an index lookup using __getitem__()
The syntax is limited, in that it will convert any digit-only strings into an integer, and everything else is always interpreted as a string (though you could use nested {} placeholders to dynamically interpolate a key value from another variable).
If you must support arbitrary expressions, the same way that f-strings do and you do not take template strings from untrusted sources (this part is important), then you could parse out the field name components and then use the eval() function to evaluate the values before you then output the final string:
from string import Formatter
_conversions = {'a': ascii, 'r': repr, 's': str}
def evaluate_template_expressions(template, globals_=None):
if globals_ is None:
globals_ = globals()
result = []
parts = Formatter().parse(template)
for literal_text, field_name, format_spec, conversion in parts:
if literal_text:
result.append(literal_text)
if not field_name:
continue
value = eval(field_name, globals_)
if conversion:
value = _conversions[conversion](value)
if format_spec:
value = format(value, format_spec)
result.append(value)
return ''.join(result)
Now the quotes are accepted:
>>> s = "the key is {d['key']}"
>>> d = {'key': 'val'}
>>> evaluate_template_expressions(s)
'the key is val'
Essentially, you can do the same with eval(f'f{s!r}', globals()), but the above might give you some more control over what expressions you might want to support.

[G]iven s and d, how do you force a f'string' parse of s? Is there some function or method available?
This can be done... using eval. But beware eval!
>>> eval('f' + repr(s))
the key is val
The repr is there to escape any quotes and to wrap s itself with quotes.
If you are aware of which variables to format (d in this case), opt for Martijn's answer of doing str.format. The above solution should be your last resort due to the dangers of eval.

How to pass "random" amount of variables not all of them exist

I have a method to validate input:
def validate_user_input(*args):
for item in args:
if not re.match('^[a-zA-Z0-9_-]+$', item):
And I'm calling it like this:
validate_user_input(var1, var2, ..., var7)
But those are generated from user input, and some of those can be missing. What would be the proper way to do that, without creating tons of if statements?
Variables are assigned from a json input like so, and json input might not have some of the needed properties:
var1 = request.json.get('var1')
I assume they are <class 'NoneType'>
Here's the error: TypeError: expected string or buffer

If your request.json object is a dict or dict-like you can just pass a default value as second argument to get

If I understand correctly you are generating var_ variables by request.json.get('var_') which will either return a string which you want to validate or None if the field was missing.
If this is the case then you can just add a special case to validate_user_input for a None value:
def validate_user_input(*args):
for item in args:
if item is None:
continue #this is acceptable, don't do anything with it
elif not re.match('^[a-zA-Z0-9_-]+$', item):
...
Or it may make more sense to store all of the values you are interested in in a dictionary:
wanted_keys = {'var1','var2','var3'}
## set intersection works in python3
present_keys = wanted_keys & response.json.keys()
## or for python 2 use a basic list comp
#present_keys = [key for key in response.json.keys() if key in wanted_keys]
actual_data = {key: response.json[key] for key in present_keys}
Then you would pass actual_data.values() as the argument list to validate_user_input.

If it really is possible that some var-variables are undefined when you call validate_user_input, why not just initialize them all (e.g. to the empty string '' so that your regex fails) before actually defining them?

Python, how can I initialize a variable during function/method calls

In most interpreted languages (example given in psuedo PHP) I would be able to do something like this
function f($name, &$errors = null) {
if(something_wrong) {
if(!is_null($errors) {
$errors[] = 'some error';
}
return false;
}
}
if(!f('test', $errs = array()) {
print_r($errs);
}
And get a result like
Array
(
[0] => some error
)
However when I try this in Python
def f(name, errors = None):
if something_wrong:
if errors:
errors.append('some error')
return False
if not f('test', errs = []):
print str(errs)
I get an error
TypeError: f() got an unexpected keyword argument 'errs'
Which makes perfect sense in the context of Python thinking I am trying to set a specific argument, and not create a new variable altogether.
However if I try
if not f('test', (errs = [])):
print str(errs)
I get
f('test', (errs = []))
^
SyntaxError: invalid syntax
Because I think now it assumes I am trying to create a tuple.
Is there any way to keep this code on one line, or do I absolutely have to initialise the variable on it's own before passing it to the function/method? From what I can tell the only solution is this
errs = []
if not f('test', errs):
print str(errs)

It should be called as f(name, errors = []) or f(name, []). If you want to init with attribute name, python requires the variable key you given is same with any attribute name you declared in the function definition.

"In Python, assignment is a statement, not an expression, and can
therefore not be used inside an arbitrary expression."
see http://effbot.org/pyfaq/why-can-t-i-use-an-assignment-in-an-expression.htm

Use an object (e.g., an Enum) as a key in **kwargs?

Very simple question from a Python newbie:
My understanding is that the keys in a dict are able to be just about any immutable data type. Is it possible to pass an immutable object (e.g., a member of an enum class) as a key in the **kwargs dictionary for a function or a class? I have tried it and the answer seems to be "no":
from enum import Enum
class MyEnum(Enum):
X= 'X'
Y= 'Y'
def func(*args,**kwargs):
pass
func(MyEnum.X = 1)
Output:
"SyntaxError: keyword can't be an expression"
However, there may be something I am missing.
EDIT: Note that I am not trying to make the key equal to MyEnum.X.value (which is a string in this case); I want the key to be the actual Enum object, e.g. MyEnum.X.

You're doing:
func(MyEnum.X = 1)
Here, the problem is MyEnum.X = 1 -- Your keyword (MyEnum.X) is actually an expression (getattr(MyEnum, 'X')), and expressions can't be used as keywords in function calls. In fact, only identifiers can be used as keywords.
To get your call to work, you'll need to use dictionary unpacking like this:
func(**{MyEnum.X.name: 1})
Note, to get the name of the attribute, I needed to do MyEnum.X.name or MyEnum.X.value, depending on how you set up your enum -- In your case, I they are the same thing.
>>> from enum import Enum
>>> class Foo(Enum):
... X = 'X'
...
>>> Foo.X.value
'X'
>>> Foo.X.name
'X'

This won't work, because of the way keyword arguments are being processed. The documentation says:
[...] Next, for each keyword argument, the identifier is used to determine the corresponding slot (if the identifier is the same as the first formal parameter name, the first slot is used, and so on) [...]
So there must be a way to match the key from the dictionary to the formal parameter name. The exception:
keywords must be strings
when you try to pass something that's not a string:
func(**{MyEnum.X: 1})
suggest the simplest case is required: keys must be strings.
A possible workaround is to make implicit things explicit: just create a class that contains all the necessary information you want to pass in its attributes and pass it. The code will surely be more readable.

The answer to my original question is indeed "no". However, thanks to the input from mgilson and BartoszKP and others, the following work around I came up with is not a bad solution, and solves my current problem. I offer it for others to look at who are trying to do something similar:
from enum import Enum
class MyEnum(Enum):
X= 'X'
Y= 'Y'
def func(*args,**kwargs):
#replace kwargs with kwargsNew
kwargsNew = {}
for kwkey, kwvalue in kwargs.items():
try: kwargsNew[MyEnum(kwkey)] = kwvalue
except ValueError: kwargsNew[kwkey] = kwvalue
doStuffWithKwargs(kwargsNew)
def doStuffWithKwargs(k):
for K in k:
print(K)
#Pass the name X or Y as the key;
#all other keys not found in `MyEnum` are treated normally
func(X = 1, Y = 2, Z = 3)
Output:
Z
MyEnum.X
MyEnum.Y
(no errors)

Do you actually want to create an instnace of MyEnum?
myenum = MyEnum()
func(myenum.X = 1)

One alternative I have found is to pass a dict into *args instead of **kwargs, or to assign a dict to kwargs[0] directly:
func({MyEnum.X: 1})
func(kwargs = {MyEnum.X: 1})
(No errors produced)
However, I really don't like either of these methods.
EDIT: See my second answer for a much better solution.

Passing keyword arguments with a hyphen

I have a keyword argument function:
def f1(**kw):
for key,val in kw.iteritems():
print "key=%s val=%s" % (key,val)
f1(Attr1 = "Val1", Attr2 = "Val2") # works fine.
f1(Attr1-SubAttr = "Val1", Attr2 = "Val2") # complains about keyword being an expression.
f1("Attr1-SubAttr" = "Val1", Attr2 = "Val2") # doesn't work either.
How do I pass in keywords with a hyphen? I don't have control over these keywords since I am parsing these from an existing legacy database.

Keyword arguments must be valid Python identifiers; these don't allow for - as that's reserved for subtraction.
You can pass in arbitrary strings using the **kwargs variable keyword argument syntax instead:
f1(**{"Attr1-SubAttr": "Val1", "Attr2": "Val2"})

To make the above code to work in python 3 iteritems() replaced by items().