Develop Reference

How to substitute value of variables in Python expression, but not evaluate the expression?

I have a Python expression that looks like the following:
var1 = 'GOOGLE'
var2 = '5'
expr = 'df[df[var1]>=var2]'
In my workspace var1 and var2 are well defined so I can evaluate expr as follows:
eval(expr)
However, I want to pass this expr (as string) to another function with values of var1 and var2 substituted in it. I do not want to pass the variables var1 and var2, as I can have any number of variables, not just two. How do I accomplish this?

You can simply use Python f-string as demonstrated below
expr = f'df[df[{var1}] >= {var2}]'

You can parse the expression with ast.parse and use a subclass of ast.NodeTransformer to convert Name nodes to the corresponding values as Constant nodes, and then convert the AST back to code with ast.unparse:
import ast
var1 = 'GOOGLE'
var2 = '5'
expr = 'df[df[var1]>=var2]'
class NamesToConstants(ast.NodeTransformer):
def visit_Name(self, node):
if node.id in globals(): # feel free to use your own dict instead of globals()
value = globals()[node.id]
try: # convert value to integer if viable
value = int(value)
except:
pass
return ast.Constant(value=value)
return node
tree = ast.parse(expr)
NamesToConstants().visit(tree)
print(ast.unparse(tree))
This outputs:
df[df['GOOGLE'] >= 5]
ast.unparse requires Python 3.10 or later. If you're using an earlier version, you can use astunparse.unparse from the astunparse package instead.
Demo: https://trinket.io/python3/18cc1182d0

Python type hints - how to handle a variable that changes type

I am adding type hints to a code base. Part of the code looks like this
var1 = get_my_int() # var1 is an int
...
if var1 !=0:
var1 = hex(var1) # var1 is now a string
output = f"something {var1}"
mypy will complain
error: Incompatible types in assignment (expression has type "str", variable has type "int")
The easiest solution is to create a new variable. For example:
if var1 !=0:
output = f"something {var1}"
else:
var1_hex = hex(var1) # var1_hex is a string, var1 is an int
output = f"something {var1_hex}"
What is the cleanest solution?

I would also convert 0 to a string for the purpose of including in the f-string, so that you can use the same assignment to output in both cases.
var1_str = hex(var1) if var1 else "0"
output = f'something {var1_str}'
This assumes that "0x0" isn't suitable for some reason, precluding the simplest solution
output = f'something {hex(var1)}'
For that matter, you don't strictly need var1_str at all:
output = f'something {hex(var1) if var1 else "0"}'

How to avoid hard-coding tons of dict key strings in Python

May be it is a common question, but I am new in python!
I am working on some libraries that handle json queries and asked myself if there is some best practice to avoid hard-coding tons of json keys like:
# just an example..
var1 = response.json()["ACRI"]["message"]["title"]
var2 = response.json()["ACRI"]["message"]["content"]
var3 = response.json()["ACRI"]["message"]["meta"]["timestamp"]
when I saw it, it didn't likes me and I created a class with constants like:
class _const:
class headers:
X_Auth_Token = "X-Auth-Token"
X_Username = 'X-Username'
X_Password = 'X-Password'
class params:
ID = "DocumentID"
Symbols = "Symbols"
ACRICode = "ACRICode"rketSegmentID"
entries = "entries"
depth = "depth"
date = "date"
meta = "meta"
timestamp = "timestamp"
message = "message"
# ...
Is that a good practice? Is there something I don't understanding about python compiler?
Thanks
edit: I'm facing performance and/or memory consumption best practices especially

The performance should be least of your concerns, the maintainability is more important. A good reason to use a symbolic name for a string is that you cannot typo it that easily then, c.f.
KEY_SYMBOLS = "Symbols"
foo[KEY_SYMOBLS]
vs
foo["Symobls"]
An IDE or a linter can find the former typo and highlight it even before running the code, but not so likely the latter one.
When it comes to performance the most performant Python code is the one that does not store strings in variables. Upon compilation the distinct uses of the same string within the same module are merged to reference just one string constant:
>>> 'foo' is 'foo'
True
Still, the point raised in Chepner's answer remains: you need not resolve subdictionaries from the root unnecessarily.

Your initial strategy follows Zen of Python very well. It is very readable, and anyone who uses Python will understand it right away. Your second example, however, is overly complicated, and difficult to understand.
That said, it is possible to parse json so that it becomes an object with attributes instead of a dict with keys. Your code would then look something like this:
import json
from collections import namedtuple
x = json.loads(response, object_hook=lambda d: namedtuple('X', d.keys())(*d.values()))
var1 = x.ACRI.message.title
var2 = x.ACRI.message.content
var3 = x.ACRI.message.meta.timestamp
Alternatively, define a simple class that takes care of it:
import json
class x(object):
def __init__(self, j):
self.__dict__ = json.loads(j)
var1 = x.ACRI.message.title
var2 = x.ACRI.message.content
var3 = x.ACRI.message.meta.timestamp

Each nested dict is a dict you can store a reference to.
resp = response.json()["ACRI"]["message"]
var1 = resp["title"]
var2 = resp["content"]
var3 = resp["meta"]["timestamp"]

If reading JSON data from a file, use the below code snippet.
with open("config.json") as json_data_file:
json_text = json_data_file.read()
json_data = json.loads(json_text, object_hook=lambda d: namedtuple('X', d.keys())(*d.values()))
# Access data using below format
var1 = json_data.key1.subkey1
var2 = json_data.key2.subkey2

Python Creating Variable Names from Strings and other Variables [duplicate]

This question already has answers here:
How do I create variable variables?
(17 answers)
Closed 5 years ago.
I am currently writing my first script in python and have functions that are all similar, except for the fact that one number in each variables name changes.
For example like this:
def Function1():
global Var1
Var1 = A1
global Var2
Var2 = B1
def Function2():
global Var1
Var1 = A2
global Var2
Var2 = B2
The variables Ax, Bx and so on are read from a file. Then, if I click a tkinter-Button, one of these functions is activated and the global variables Var1 and Var2 are set to either A1 and B1 or A2 and B2 and so on...
Since this is just copy and pasting the functions and then manually changing the number in the variable name that is read from the file, I thought there has to be a way to make this easier. Something like this:
def Function1():
FUNCTIONNUMBER = 1
global Var1
Var1 = ("A" + FUNCTIONNUMBER)
global Var2
Var2 = ("B" + FUNCTIONNUMEBR)
In other words, something that makes it possible to set up a variable name from a constant part ("A") and a changing part ("1" or "2" or "3"...)
I´ve already searched for this question, but didn´t find anything that helps me with my specific case, since I don´t want to create variables out of nowhere or I don´t want to write all the variable names in a dictionary. I just want to make python think it reads a variable name that consists of different parts.
Thank you in advance!

I think what you're looking for is a list. let me show you and example
some_list = ["hey", "hows", "it", "going"]
print(some_list[0])
print(some_list[2])
This output would be:
hey
it
a list is just that, a list of numbers/strings/function/other lists, and you access them by specifying which position in the list you want (starting at 0)

Python: modify global var from a function argument [duplicate]

This question already has answers here:
How do I create variable variables?
(17 answers)
How do I get a result (output) from a function? How can I use the result later?
(4 answers)
Closed 7 months ago.
I'd like to create a function that will modify an initialized global variable based on the argument passed to it, but I get a SyntaxError: name 'arg' is local and global. I have seen other methods to accomplish this, using globals() or creating a simple func inside myFunc to "trick" Python. Another approach would be to create if statements inside myFunc to explicitly assign the corresponding global variables, but that seems overly verbose.
Why does this occur, and what would be the most efficient/elegant/Pythonic way to accomplish this?
Given:
var1 = 1
var2 = 2
var3 = 3
def myFunc(arg):
global arg
arg = 10
myFunc(var1) # each of these should print to 10
myFunc(var2)
myFunc(var3)

You can use globals() to access the variables and assign new values from within myFunc()
var1 = 1
var2 = 2
def myFunc(varname):
globals()[varname] = 10
print(var1, var2)
myFunc("var1")
myFunc("var2")
print(var1, var2)
Will output:
1, 2
10, 10

In python a variable is a name for an object. When you call a function, and pass it an argument you're passing the object associated with the variable, not the name. So for example when you call wash(mydog), you're saying "wash the object known as mydog". Keep in mind, that the same object could have more than one name, for example spot = mydog = best_dog_ever = new_dog(). The function doesn't know which name was used to pass it the object, and even if it did, what if the name used was not the one in the global scope, you'd have to have some way of saying this function only takes global variables as arguments.
I hope that helps explain why you're getting a syntax error, but you can still accomplish pretty much the same thing at least two ways. The first is to simply assign the return value to the variable you're trying to change, for example:
var1 = 1
var2 = 2
def addone(a):
return a + 1
def main():
global var1, var2
var1 = addone(var1)
var2 = addone(var2)
print var1, var2
main()
print var1, var2
The second is to use a more object oriented approach, something like this:
class GlobalValue(object):
def __init__(self, value):
self.value = value
var1 = GlobalValue(1)
var2 = GlobalValue(2)
def addone(a):
a.value += 1
print var1.value, var2.value
addone(var1)
addone(var2)
print var1.value, var2.value
Or even better:
class GlobalValue(object):
def __init__(self, value):
self.value = value
def addone(self):
self.value += 1
var1 = GlobalValue(1)
var2 = GlobalValue(2)
print var1.value, var2.value
var1.addone()
var2.addone()
print var1.value, var2.value

Why does this occur
Because the global variable that you want to use has the same name as the parameter, arg. In Python, parameters are local variables, and a variable can only be local or global, not both.
It appears as though you expected to be able to use the contents of var to, somehow, specify which existing global variable to modify. It does not work like that. First off, variables don't contain other variables; they contain values. The name of a variable isn't a value. Again, parameters are local variables - and calling a function assigns to those variables. It assigns a value. (Keep in mind that you could just as easily call the function without a variable for the argument: myFunc(3). Should this cause 3 to become equal to 10 somehow?)
Second, even if you passed, for example, a string, you would have to do more work in order to access the corresponding global variable. It can be done, but please do not.
and what would be the most efficient/elegant/Pythonic way to accomplish this?
The Pythonic way is don't. If you want your function to communicate information out when it is called, return a value:
def myFunc():
return 10
var1 = myFunc()
var2 = myFunc()
var3 = myFunc()
The simplest way to fix the error is to just rename the global variable. However, this does not fix the apparent intent of the code:
var1 = 1
var2 = 2
var3 = 3
def myFunc(arg):
global g
g = 10
# var1, var2 and var3 will NOT CHANGE
# Instead, the new global variable g is created, and assigned a value of 10,
# three times in a row.
myFunc(var1)
myFunc(var2)
myFunc(var3)