How can I delete all variables in jupyter that are defined / created after a certain cell number?
My specific use case is that I will load some variables in the first few cells and I want to keep those variables, while I will experiment on the notebook.
I do not want to keep restarting or reseting the notebook as it takes time to load those variables. Instead I would like to "restart" from a certain cell.
If you can afford the memory and duplicate the variables, store "backup" variables using something like copy.copy() or copy.deepcopy() and create a cell were you reallocate original values to your variables from the backups.
You'll have run this cell to restore your values.
see edit details below
For illustration:
Store original values
from copy import deepcopy
bckp_a = deepcopy(var_a)
bckp_b = deepcopy(var_b)
dir_bckp = deepcopy(dir()) # store defined variables defined at this point
Do you stuff
var_a = some_func(var_a)
var_b = some_other_func(var_a)
var_c = some_value
Restore original values for preserved variables
var_a = deepcopy(bckp_a)
var_b = deepcopy(bckp_b)
Delete newly created variables
for var in dir():
if var == 'dir_bckp': # Note that it is a string
continue # Preserve dir_bckp, very important.
elif var not in dir_bckp:
del globals()[var] # Delete newly defined variables
>>> var_c
NameError: name 'var_c' is not defined
EDIT:
If you absolutely need to delete created variables, you can try a trick with dir() and globals(). But this is probably not good practice, so be cautious.
See changes above.
Note that there is also the option of creating a restore point using Pickle, but I'm not sure of the performance if some variables take time to load.
Related
I want to iterate over a list, and then pass that variable to another Python file, witch writes that text.
forloop.py:
class Main:
def list():
list = ["a","b","c","d","e","f","g"]
for i in list:
print_this_variable = i
That iterates over the list, now i want to print the results in a separate file.
print.py:
from forloop import *
print(print_this_variable)
Thanks for the help.
You can't, the way you've configured things. The variable print_this_variable is local to list and won't be available outside of that method.
Here's one way to structure things (there are a variety of other ways, but your question isn't very clear about what you're actually trying to accomplish):
First, note that list is the name of the Python list data type -- you shouldn't use it as a name for functions or variables. Second, you shouldn't name variables the same as functions, because this will mask the function name and will probably bite you at some point.
So, in forloop.py, let's do this:
class Main:
def example_function(self):
data = ["a","b","c","d","e","f","g"]
for i in data:
self.print_this_variable = i
That makes print_this_variable an instance variable for Main objects.
In print.py, we could write:
import forloop
# We need to create a Main object
m = forloop.Main()
# The `print_this_variable` attribute isn't available until
# after we # call the `example_function` method.
m.example_function()
# Now we can ask for the instance attribute
print(m.print_this_variable)
Hello I am currently doing a project were I need up to 30.000 variables, which will be created dynamically. My problem however is accessing said variables dynamically , storing them in an array and accessing them like this works but I'd like to access them by name only. My code looks like:
NG=10
for i in range(1, NG+1 ):
globals()[f"u_{i}"] = i
print(u_{i})
Declaring variables like this works and they can be accessed by typing u_1, but the above print statement breaks the code.
Is there an option to access a variable similar to this in python?
You can access it the same way you set it:
globals()[f"u_{i}"]
Except I highly recommend you NOT to use global variables. You can use a dictionary; eg.
data = {}
data["some_key"] = 123
print(data["some_key"])
This will work the same way as does with global variables, except not having the pain of global variables.
Using a Dictionary would be the best option if you ask me. Just to give an example of a dummy assignment:
import random
a={} # the dictionary
random.seed(5)
for i in range(30000):
a['u'+str(i+1)]=random.random() # Or whatever value you want to put in the variable
print(a['u1']) # First variable and so on...
print(a['u2'])
Can you run a for loop over the names of multiple subsets?
For instance, I now have subsets dfVC1 up until dfVC20 and I would like to do something like:
for x in range(20):
print(dfVC[x])
I get this doesn't work... but wonder if there is a way to do this.
I'm going to assume your 'subsets' in this case are variables, named dbVC0, dbVC1, etc. Then, your problem is that you want to print all of them by number, but since they're variables, you can't.
One way to solve this would be to change how the 'subsets' are declared. Instead of
dfVC0 = ...
dfVC1 = ...
you could make one dfVC variable that's a dict, that holds all the others at their proper indices.
dfVC = {}
dfVC[0] = ...
dfVC[1] = ...
which would then allow you to access the various dbVC subsets in the way you're currently trying to.
But changing such a large part of the program isn't always possible. What you might be able to do instead is to figure out which object the dfVCs are attached to, and grab them by string.
If they're in the local namespace (i.e. were declared in the same function as you're currently executing in), you can call the built-in locals() to get a dict that you can then try to find your key in:
for x in range(20):
sname = f'dfVC{x}'
print(locals()[sname])
globals() can be used similarly, if your 'subsets' are in the global scope (i.e. declared outside of the current function).
And if your dfVC variables are attached to a class or module (or something else that behaves like a namespace), you can retrieve them using the built-in getattr() function:
for x in range(20):
sname = f'dfVC{x}'
print(getattr(self, sname)) # replace self with whichever object has the dbVC attached to it
I have found some vaguely related questions to this question, but not any clean and specific solution for CPython. And I assume that a "valid" solution is interpreter specific.
First the things I think I understand:
locals() gives a non-modifiable dictionary.
A function may (and indeed does) use some kind of optimization to access its local variables
frame.f_locals gives a locals() like dictionary, but less prone to hackish things through exec. Or at least I have been less able to do hackish undocumented things like the locals()['var'] = value ; exec ""
exec is capable to do weird things to the local variables, but it is not reliable --e.g. I read somewhere that it doesn't work in Python 3. Haven't tested.
So I understand that, given those limitations, it will never be safe to add extra variables to the locals, because it breaks the interpreter structure.
However, it should be possible to change a variable already existing, isn't it?
Things that I considered
In a function f, one can access the f.func_code.co_nlocals and f.func_code.co_varnames.
In a frame, the variables can be accessed / checked / read through the frame.f_locals. This is in the use case of setting a tracer through sys.settrace.
One can easily access the function in which a frame is --cosidering the use case of setting a trace and using it to "do things" in with the local variables given a certain trigger or whatever.
The variables should be somewhere, preferably writeable... but I am not capable of finding it. Even if it is an array (for interpreter efficient access), or I need some extra C-specific wiring, I am ready to commit to it.
How can I achieve that modification of variables from a tracer function or from a decorated wrapped function or something like that?
A full solution will be of course appreciated, but even some pointers will help me greatly, because I'm stuck here with lots of non writeable dictionaries :-/
Edit: Hackish exec is doing things like this or this
It exists an undocumented C-API call for doing things like that:
PyFrame_LocalsToFast
There is some more discussion in this PyDev blog post. The basic idea seems to be:
import ctypes
...
frame.f_locals.update({
'a': 'newvalue',
'b': other_local_value,
})
ctypes.pythonapi.PyFrame_LocalsToFast(
ctypes.py_object(frame), ctypes.c_int(0))
I have yet to test if this works as expected.
Note that there might be some way to access the Fast directly, to avoid an indirection if the requirements is only modification of existing variable. But, as this seems to be mostly non-documented API, source code is the documentation resource.
Based on the notes from MariusSiuram, I wrote a recipe that show the behavior.
The conclusions are:
we can modify an existing variable
we can delete an existing variable
we can NOT add a new variable.
So, here is the code:
import inspect
import ctypes
def parent():
a = 1
z = 'foo'
print('- Trying to add a new variable ---------------')
hack(case=0) # just try to add a new variable 'b'
print(a)
print(z)
assert a == 1
assert z == 'foo'
try:
print (b)
assert False # never is going to reach this point
except NameError, why:
print("ok, global name 'b' is not defined")
print('- Trying to remove an existing variable ------')
hack(case=1)
print(a)
assert a == 2
try:
print (z)
except NameError, why:
print("ok, we've removed the 'z' var")
print('- Trying to update an existing variable ------')
hack(case=2)
print(a)
assert a == 3
def hack(case=0):
frame = inspect.stack()[1][0]
if case == 0:
frame.f_locals['b'] = "don't work"
elif case == 1:
frame.f_locals.pop('z')
frame.f_locals['a'] += 1
else:
frame.f_locals['a'] += 1
# passing c_int(1) will remove and update variables as well
# passing c_int(0) will only update
ctypes.pythonapi.PyFrame_LocalsToFast(
ctypes.py_object(frame),
ctypes.c_int(1))
if __name__ == '__main__':
parent()
The output would be like:
- Trying to add a new variable ---------------
1
foo
ok, global name 'b' is not defined
- Trying to remove an existing variable ------
2
foo
- Trying to update an existing variable ------
3
I have been told that using exec is a Very Bad Thing.
However, I'm new to python and trying to figure out how to dynamically create a bunch of global variables (I'm aware that this is also supposed to be a Bad Thing, but let's burn one bridge at a time, shall we?).
What this is doing: get a list of the current variables that need to be created (currently sitting in a CSV), get the unique ID's within that list, then create the necessary objects by appending the ID to the name and reading the content of another CSV into it.
import pandas as pd
def importtest():
ilist = pd.read_csv('Z:/fakepath/ID.csv')
for i in range(0, len(ilist['ID'].unique())):
tempID = ilist['ID'].unique()[i]
exec("variable%s = pd.read_csv('%s')" % (
str(tempID), 'Z:/fakepath/'+str(tempID)+'.csv'), globals())
i = i + 1
Is there another/better way to dynamically create/update the variables I need so they show up in the global scope?
String keys in globals() dictionary correspond to variable names, so, you don't need to use exec, you can write variable to globals hash directly:
globals()["variable" + str(tempID)] = pd.read_csv('Z:/fakepath/'+str(tempID)+'.csv')