Hello I am currently doing a project were I need up to 30.000 variables, which will be created dynamically. My problem however is accessing said variables dynamically , storing them in an array and accessing them like this works but I'd like to access them by name only. My code looks like:
NG=10
for i in range(1, NG+1 ):
globals()[f"u_{i}"] = i
print(u_{i})
Declaring variables like this works and they can be accessed by typing u_1, but the above print statement breaks the code.
Is there an option to access a variable similar to this in python?
You can access it the same way you set it:
globals()[f"u_{i}"]
Except I highly recommend you NOT to use global variables. You can use a dictionary; eg.
data = {}
data["some_key"] = 123
print(data["some_key"])
This will work the same way as does with global variables, except not having the pain of global variables.
Using a Dictionary would be the best option if you ask me. Just to give an example of a dummy assignment:
import random
a={} # the dictionary
random.seed(5)
for i in range(30000):
a['u'+str(i+1)]=random.random() # Or whatever value you want to put in the variable
print(a['u1']) # First variable and so on...
print(a['u2'])
Related
Can you run a for loop over the names of multiple subsets?
For instance, I now have subsets dfVC1 up until dfVC20 and I would like to do something like:
for x in range(20):
print(dfVC[x])
I get this doesn't work... but wonder if there is a way to do this.
I'm going to assume your 'subsets' in this case are variables, named dbVC0, dbVC1, etc. Then, your problem is that you want to print all of them by number, but since they're variables, you can't.
One way to solve this would be to change how the 'subsets' are declared. Instead of
dfVC0 = ...
dfVC1 = ...
you could make one dfVC variable that's a dict, that holds all the others at their proper indices.
dfVC = {}
dfVC[0] = ...
dfVC[1] = ...
which would then allow you to access the various dbVC subsets in the way you're currently trying to.
But changing such a large part of the program isn't always possible. What you might be able to do instead is to figure out which object the dfVCs are attached to, and grab them by string.
If they're in the local namespace (i.e. were declared in the same function as you're currently executing in), you can call the built-in locals() to get a dict that you can then try to find your key in:
for x in range(20):
sname = f'dfVC{x}'
print(locals()[sname])
globals() can be used similarly, if your 'subsets' are in the global scope (i.e. declared outside of the current function).
And if your dfVC variables are attached to a class or module (or something else that behaves like a namespace), you can retrieve them using the built-in getattr() function:
for x in range(20):
sname = f'dfVC{x}'
print(getattr(self, sname)) # replace self with whichever object has the dbVC attached to it
I have 4 variables in my python code which look like this:
dev_login=os.environ['DEV_LOGIN']
dev_password=os.environ['DEV_PASSWORD']
prod_login=os.environ['PROD_LOGIN']
prod_password=os.environ['PROD_PASSWORD']
And I have a function which receives one of two environments as an attribute like:
somefunction(env)
in somefunction I need to use 2 of the above variables depending the env attribute. So if I call somefunction(dev) then the login and password variables in the function should be assigned like this:
login=dev_login
password=dev_password
using if isn't the right way for me, as I may have 10+ envs, and don't want to add some code when new environment is added.
Tried this way but it didn't work:
def somefunctions(env):
login=+env+"_login"
def some_func(input):
login = '{}_LOGIN'.format(input.upper())
password = '{}_PASSWORD'.format(input.upper())
print(login, password)
some_func('env')
some_func('dev')
>> ENV_LOGIN ENV_PASSWORD
>> DEV_LOGIN DEV_PASSWORD
You can then insert the variables as needed:
dev_login=os.environ[login] # 'DEV_LOGIN'
dev_password=os.environ[password] # 'DEV_PASSWORD'
I would like to use a function's parameter to create dynamic names of dataframes and/or objects in Python. I have about 40 different names so it would be really elegant to do this in a function. Is there a way to do this or do I need to do this via 'dict'? I read that 'exec' is dangerous (not that I could get this to work). SAS has this feature for their macros which is where I am coming from. Here is an example of what I am trying to do (using '#' for illustrative purposes):
def TrainModels (mtype):
model_#mtype = ExtraTreesClassifier()
model_#mtype.fit(X_#mtype, Y_#mtype)
TrainModels ('FirstModel')
TrainModels ('SecondModel')
You could use a dictionary for this:
models = {}
def TrainModels (mtype):
models[mtype] = ExtraTreesClassifier()
models[mtype].fit()
First of all, any name you define within your TrainModels function will be local to that function, so won't be accessible in the rest of your program. So you have to define a global name.
Everything in Python is a dictionary, including the global namespace. You can define a new global name dynamically as follows:
my_name = 'foo'
globals()[my_name] = 'bar'
This is terrible and you should never do it. It adds too much indirection to your code. When someone else (or yourself in 3 months when the code is no longer fresh in your mind) reads the code and see 'foo' used elsewhere, they'll have a hard time figuring out where it came from. Code analysis tools will not be able to help you.
I would use a dict as Milkboat suggested.
How can I delete all variables in jupyter that are defined / created after a certain cell number?
My specific use case is that I will load some variables in the first few cells and I want to keep those variables, while I will experiment on the notebook.
I do not want to keep restarting or reseting the notebook as it takes time to load those variables. Instead I would like to "restart" from a certain cell.
If you can afford the memory and duplicate the variables, store "backup" variables using something like copy.copy() or copy.deepcopy() and create a cell were you reallocate original values to your variables from the backups.
You'll have run this cell to restore your values.
see edit details below
For illustration:
Store original values
from copy import deepcopy
bckp_a = deepcopy(var_a)
bckp_b = deepcopy(var_b)
dir_bckp = deepcopy(dir()) # store defined variables defined at this point
Do you stuff
var_a = some_func(var_a)
var_b = some_other_func(var_a)
var_c = some_value
Restore original values for preserved variables
var_a = deepcopy(bckp_a)
var_b = deepcopy(bckp_b)
Delete newly created variables
for var in dir():
if var == 'dir_bckp': # Note that it is a string
continue # Preserve dir_bckp, very important.
elif var not in dir_bckp:
del globals()[var] # Delete newly defined variables
>>> var_c
NameError: name 'var_c' is not defined
EDIT:
If you absolutely need to delete created variables, you can try a trick with dir() and globals(). But this is probably not good practice, so be cautious.
See changes above.
Note that there is also the option of creating a restore point using Pickle, but I'm not sure of the performance if some variables take time to load.
I have been told that using exec is a Very Bad Thing.
However, I'm new to python and trying to figure out how to dynamically create a bunch of global variables (I'm aware that this is also supposed to be a Bad Thing, but let's burn one bridge at a time, shall we?).
What this is doing: get a list of the current variables that need to be created (currently sitting in a CSV), get the unique ID's within that list, then create the necessary objects by appending the ID to the name and reading the content of another CSV into it.
import pandas as pd
def importtest():
ilist = pd.read_csv('Z:/fakepath/ID.csv')
for i in range(0, len(ilist['ID'].unique())):
tempID = ilist['ID'].unique()[i]
exec("variable%s = pd.read_csv('%s')" % (
str(tempID), 'Z:/fakepath/'+str(tempID)+'.csv'), globals())
i = i + 1
Is there another/better way to dynamically create/update the variables I need so they show up in the global scope?
String keys in globals() dictionary correspond to variable names, so, you don't need to use exec, you can write variable to globals hash directly:
globals()["variable" + str(tempID)] = pd.read_csv('Z:/fakepath/'+str(tempID)+'.csv')