I have been trying to read python files and print its variables for a while now. Is it possible to view and print the variables of another file without importing or running it? Everything I have already searched only explains how to import or use execfile, etc... (disclaimer, I am probably dumb)
This is what I have so far:
for vars in dir():
print(vars)
Now, this works perfectly fine for the file it is being run from, but when I try this code:
for vars in file:
print(vars)
(file is simply path.read())
Well, it gives me every character of the file on a new line. I have no idea if my loop is correct at all. Will I have to write something that will manually find each variable, then add it into a list?
Use ast.parse to parse the code, recursively traverse the nodes by iterating through those with a body attribute (which are code blocks), look for Assign objects and get their targets (which are the variables being assigned with values, which are what you're looking for) and get their id attribute if they are Name objects.
Try the following code after replacing file.py with the file name of the python script you want to parse.
import ast
import _ast
def get_variables(node):
variables = set()
if hasattr(node, 'body'):
for subnode in node.body:
variables |= get_variables(subnode)
elif isinstance(node, _ast.Assign):
for name in node.targets:
if isinstance(name, _ast.Name):
variables.add(name.id)
return variables
print(get_variables(ast.parse(open('file.py').read())))
No ... and yes.
The question is whether the "variables" are constants or true variables.
Python runs a garbage collector. It will create the variables when you run/import a module. These variables will have scope based on how they are used. Once they are no longer in use, the garbage collector will remove the objects from memory.
If the variable is assigned a fixed value (e.g. i = 1) then you can simply read the file in as if it is a text file - since it is a text file. If you want to change those variables, you can simply write it out as a text file. It will be incumbent on you to trace the variables in the text, exactly as with any text matching.
If the variables are generated as part of the code, then no (e.g. it generates a list of file in a directory and assigns to a variable). You would need to either import the module, or change the module so that it exports the outputs to a separate file - e.g. csv - and then you can read the data file.
First of all you are not dumb, its just not many people need to do this.
Also, I cant think of a way without moving the variables to a septate json or pickle file then loading it in later in both programs using json.load(filename) or something like that. But that only works if it isn't being changed while the other program is assessing it.
Related
I hope the question phrasing is meaningful. What I am wanting to do is change a flat variable's value in a file, and have the files which have imported that file see the updated value. It appears that I can do this. For example:
#settings.py
VARIABLE = 1
def change_variable():
global VARIABLE
VARIABLE = 2
and
#main.py
import settings
print(settings.VARIABLE)
settings.change_variable()
print(settings.VARIABLE)
which outputs:
1
2
As desired. Although I was a little surprised since I thought maybe the value of settings.VARIABLE would be fixed after settings was imported. I would like to know whether I can rely on this behaviour. My question is thus also, when in general will the values from an imported file be "updated" or "re-evaluated" from the perspective of the importing file? How does it work behind the scenes?
I could of course just make a class. But I don't like the idea of settings, or any config, being an object. I prefer it flat. But I want the option to change the settings after import based on user cli input.
Once the file settings.py is imported, python is done looking at the file. It now has a module loaded in memory, and if it is imported somewhere else, that module will be loaded there. The file is never looked at again after the first import.
Your function changed the value of VARIABLE in that module. You can depend on it being your new value unless you change it again.
I am trying to write some kind of wrapper for testing python module.
This wrapper simulates an environment and then starts other python module via execfile.
There are many python modules to be tested (>200).
Inside those modules there are some hard-coded variables that contain some absolute file-paths, that are not existing in my simulated environment (I also cannot create them). Those file-paths are paths to option files, that the script will read in. There is always exactly one option-file for each module and the file-path to thisoption files is always saved in the same global variable (What I mean: The variable name is the same in each module: optionFile).
optionFile = "Path to Option file"
My thought was, that I could maybe pre-set this global variable "optionFile" with an existing path before I execute the test-module. But of course this alone won't help, since the executed module will just overwrite "optionFile" with the hard-coded value when it is executed.
I wondered if there might be a way to overwrite the __setattr__ function of the globals-object, so that it will do nothing for certain variable names, but I was not successful with my tries. Do you think this could work and have any suggestions?
Based on the first impressions we got here it seems to be not possible to alter the __setattr__ of the globals object (though I don't understand why not...)
So the answer seems to be "No".
EDIT:
The reason, why this does not work here is, that there is no global "globals "-object. Instead each module has its "personal" namespace with its own global variables. This namespace is created once the module is loaded. It is for sure possible to alter that global namespace - but only after the module has already been loaded (which does not help in my application scenario). Thanks to BrenBarn for the clarification.
END OF EDIT
A workaround for my specific described problem would be to alter Python's built-in open-function instead.
# Keep original pointer to the actual Open-function
realOpen = open
# Overwrite the name of the Open-function to implement own logic
def open(filename, mode='r'):
if filename.endswith(".opt"):
print "Rerouting opening command"
realOpen("myCentralOptionFile.opt","r")
else:
realOpen(filename,mode)
Attention: This workaround has nothing to do with the title anymore.
I already use this function to change some string to class object.
But now I have defined a new module. How can I implement the same functionality?
def str2class(str):
return getattr(sys.modules[__name__], str)
I want to think some example, but it is hard to think. Anyway, the main problem is maybe the file path problem.
If you really need an example, the GitHub code is here.
The Chain.py file needs to perform an auto action mechanism. Now it fails.
New approach:
Now I put all files under one filefold, and it works, but if I use the modules concept, it fails. So if the problem is in a module file, how can I change the string object to relative class object?
Thanks for your help.
You can do this by accessing the namespace of the module directly:
import module
f = module.__dict__["func_name"]
# f is now a function and can be called:
f()
One of the greatest things about Python is that the internals are accessible to you, and that they fit the language paradigm. A name (of a variable, class, function, whatever) in a namespace is actually just a key in a dictionary that maps to that name's value.
If you're interested in what other language internals you can play with, try running dir() on things. You'd be surprised by the number of hidden methods available on most of the objects.
You probably should write this function like this:
def str2class(s):
return globals()[s]
It's really clearer and works even if __name__ is set to __main__.
I am using a 3rd party application's Python integration where Python code is stored internally in binary files. So I want to separate the code from it. I could create a module but the code I need to include doesn't function on its own, as it includes locals only valid in that context, etc.
I am basically looking for a copy-paste solution at compile-time.
Is there any way to do this in Python?
Yes, it's called exec.
Your code would look like this: exec your_code_str. The code is then executed in the current scope.
You could also specify a different scope: exec your_code_str in globals(), locals() (those values are the default, so only specify them if they differ)
Note that you can also pass a file object, so if the python code is only a single blob of text right at the end of the binary file you can just seek to the beginning of the code and then pass the file object.
I know this does not sound Pythonic, but bear with me for a second.
I am writing a module that depends on some external closed-source module. That module needs to get instantiated to be used (using module.create()).
My module attempts to figure out if my user already loaded that module (easy to do), but then needs to figure out if the module was instantiated. I understand that checking out the type() of each variable can tell me this, but I am not sure how I can get the names of variables defined by the main program. The reason for this is that when one instantiates the model, they also set a bunch of parameters that I do not want to overwrite for any reason.
My attempts so far involved using sys._getframe().f_globals and iterating through the elements, but in my testing it doesn't work. If I instantiate the module as modInst and then call the function in my module, it fails to show the modInst variable. Is there another solution to this? Sample code provided below.
import sys
if moduleName not in sys.modules:
import moduleName
modInst = moduleName.create()
else:
globalVars = sys._getframe().f_globals
for key, value in globalVars:
if value == "Module Name Instance":
return key
return moduleName.create()
EDIT: Sample code included.
Looks like your code assumes that the .create() function was called, if at all, by the immediate/direct caller of your function (which you show only partially, making it pretty hard to be sure about what's going on) and the results placed in a global variable (of the module where the caller of your function resides). It all seems pretty fragile. Doesn't that third-party module have some global variables of its own that are affected by whether the module's create has been called or not? I imagine it would -- where else is it keeping the state-changes resulting from executing the create -- and I would explore that.
To address a specific issue you raise,
I am not sure how I can get the names
of variables defined by the main
program
that's easy -- the main program is found, as a module, in sys.modules['__main__'], so just use vars(sys.modules['__main__']) to get the global dictionary of the main program (the variable names are the keys in that dictionary, along of course with names of functions, classes, etc -- the module, like any other module, has exactly one top-level/global namespace, not one for variables, a separate one for functions, etc).
Suppose the external closed-sourced module is called extmod.
Create my_extmod.py:
import extmod
INSTANTIATED=False
def create(*args,**kw):
global INSTANTIATED
INSTANTIATED=True
return extmod.create(*args,**kw)
Then require your users to import my_extmod instead of extmod directly.
To test if the create function has been called, just check the value of extmod.INSTANTIATED.
Edit: If you open up an IPython session and type import extmod, then type
extmod.[TAB], then you'll see all the top-level variables in the extmod namespace. This might help you find some parameter that changes when extmod.create is called.
Barring that, and barring the possibility of training users to import my_extmod, then perhaps you could use something like the function below. find_extmod_instance searches through all modules in sys.modules.
def find_instance(cls):
for modname in sys.modules:
module=sys.modules[modname]
for value in vars(module).values():
if isinstance(value,cls):
return value
x=find_instance(extmod.ExtmodClass) or extmod.create()