Develop Reference

Viewing Variables of Another Python File Without Importing/Running

I have been trying to read python files and print its variables for a while now. Is it possible to view and print the variables of another file without importing or running it? Everything I have already searched only explains how to import or use execfile, etc... (disclaimer, I am probably dumb)
This is what I have so far:
for vars in dir():
print(vars)
Now, this works perfectly fine for the file it is being run from, but when I try this code:
for vars in file:
print(vars)
(file is simply path.read())
Well, it gives me every character of the file on a new line. I have no idea if my loop is correct at all. Will I have to write something that will manually find each variable, then add it into a list?

Use ast.parse to parse the code, recursively traverse the nodes by iterating through those with a body attribute (which are code blocks), look for Assign objects and get their targets (which are the variables being assigned with values, which are what you're looking for) and get their id attribute if they are Name objects.
Try the following code after replacing file.py with the file name of the python script you want to parse.
import ast
import _ast
def get_variables(node):
variables = set()
if hasattr(node, 'body'):
for subnode in node.body:
variables |= get_variables(subnode)
elif isinstance(node, _ast.Assign):
for name in node.targets:
if isinstance(name, _ast.Name):
variables.add(name.id)
return variables
print(get_variables(ast.parse(open('file.py').read())))

No ... and yes.
The question is whether the "variables" are constants or true variables.
Python runs a garbage collector. It will create the variables when you run/import a module. These variables will have scope based on how they are used. Once they are no longer in use, the garbage collector will remove the objects from memory.
If the variable is assigned a fixed value (e.g. i = 1) then you can simply read the file in as if it is a text file - since it is a text file. If you want to change those variables, you can simply write it out as a text file. It will be incumbent on you to trace the variables in the text, exactly as with any text matching.
If the variables are generated as part of the code, then no (e.g. it generates a list of file in a directory and assigns to a variable). You would need to either import the module, or change the module so that it exports the outputs to a separate file - e.g. csv - and then you can read the data file.

First of all you are not dumb, its just not many people need to do this.
Also, I cant think of a way without moving the variables to a septate json or pickle file then loading it in later in both programs using json.load(filename) or something like that. But that only works if it isn't being changed while the other program is assessing it.

Prevent global variables of called submodules from being altered without changing the submodul itself?

I am trying to write some kind of wrapper for testing python module.
This wrapper simulates an environment and then starts other python module via execfile.
There are many python modules to be tested (>200).
Inside those modules there are some hard-coded variables that contain some absolute file-paths, that are not existing in my simulated environment (I also cannot create them). Those file-paths are paths to option files, that the script will read in. There is always exactly one option-file for each module and the file-path to thisoption files is always saved in the same global variable (What I mean: The variable name is the same in each module: optionFile).
optionFile = "Path to Option file"
My thought was, that I could maybe pre-set this global variable "optionFile" with an existing path before I execute the test-module. But of course this alone won't help, since the executed module will just overwrite "optionFile" with the hard-coded value when it is executed.
I wondered if there might be a way to overwrite the __setattr__ function of the globals-object, so that it will do nothing for certain variable names, but I was not successful with my tries. Do you think this could work and have any suggestions?

Based on the first impressions we got here it seems to be not possible to alter the __setattr__ of the globals object (though I don't understand why not...)
So the answer seems to be "No".
EDIT:
The reason, why this does not work here is, that there is no global "globals "-object. Instead each module has its "personal" namespace with its own global variables. This namespace is created once the module is loaded. It is for sure possible to alter that global namespace - but only after the module has already been loaded (which does not help in my application scenario). Thanks to BrenBarn for the clarification.
END OF EDIT
A workaround for my specific described problem would be to alter Python's built-in open-function instead.
# Keep original pointer to the actual Open-function
realOpen = open
# Overwrite the name of the Open-function to implement own logic
def open(filename, mode='r'):
if filename.endswith(".opt"):
print "Rerouting opening command"
realOpen("myCentralOptionFile.opt","r")
else:
realOpen(filename,mode)
Attention: This workaround has nothing to do with the title anymore.

Temporarily modifying `builtins` while splitting a django settings file

I'm starting a django 1.10 project and would like to split the settings file. I was unsatisfied with any existing solutions.
I do not want to be able to override string/boolean/dict settings from one file in another. Each string/boolean/dict setting should be set in only one place. This makes it easy to keep track of where things are defined.
I do not want to have to manually extend tuple/list settings, e.g. INSTALLED_APPS += (test_app). This seems to be messy and requires me to keep track of whether a list or tuple was used in the other file.
I do not want to have to import os and define BASE_DIR in multiple files. DRY.
My solution, having looked at many others, is to replace settings.py with a directory containing local_settings.py, common_settings.py and __init__.py.
In __init__.py, I import os and calculate BASE_DIR. I then
import builtins
builtins.BASE_DIR = BASE_DIR
builtins.os = os
from .common_settings import *
from . import local_settings
# At this point both modules have run and we no longer need to be messing
# with the builtins namespace.
del builtins.BASE_DIR
del builtins.os
del builtins
I then loop over dir(local_settings) and mess with globals() to achieve the first two requirements (I can post the whole thing if requested but I'm interested in my use of builtins).
Is this use of builtins too evil? What can break it. Obviously if either identifier clashs with an attribute of a later version of builtins, then this code would break python. If a function that uses either of these identifiers ended up in one of the settings files and it was later called, then that would break.
I don't see either of those happening though. Is there a potential problem that I'm not seeing?

The main problem with modifying builtins in this way is that it adds non-local reasoning to your code for no good reason. The behavior of the common/local settings modules now implicitly depends on what happens in the module that imports them. That's bad.
Essentially, you need to get rid of your requirement #3.
Importing os in each module isn't "repeating yourself" because each module imports os into its own namespace. That's just how Python works.
You're right to want to only define BASE_DIR once, but the correct way to do this is to define the variable in one module (say basedir.py) and then explicitly import that variable (from basedir import BASE_DIR) into each module that uses it.

Accessing a Python global variable across files

I have three python files in a project:
'''lib.py
library file that first defines and also accesses a boolean flag'''
vflag = False
...
def lib_method()
global vflag
if not vflag:
do_something_here
'''app.py
main application that sets the boolean flag defined in lib.py'''
import lib
lib.vflag = method_that_sets_or_resets_vflag()
'''processor.py
does some processing and needs to access the bool flag'''
from lib import *
...
def processor_method()
global vflag
...
if vflag:
do_something_here
I am able to set/ reset the flag at app.py, but processor method is not able to get the correct value for this boolean variable.
It only gets whatever is set at the beginning of lib.py(not what is set by app.py). The requirement is to access the value set at runtime by app.py, NOT the value to which it was initialized by lib.py
I am importing the lib file in different ways at app.py and processor.py. Does that make a difference ?
It may be a fundamental mistake, so I will appreciate if some one can point me to a specific knowledge base.
Also, it is messing with my understanding of global variables. If I define a variable to be 'global', does that mean the variable stays to be a global variable in all the files that import this variable(or the file containing the variable definition)

When you use from lib import * in processor.py, you are getting a snapshot of what's going on in lib.py at that moment. The lib.py file is executed, and all of the functions and variables are copied and stored in the namespace of processor.py. You are not storing references to the original vflag from lib.py - you're storing an entirely new copy. So if you change vflag in lib.py, then processor.py will never know about it.
The better practice is to always use import lib, and just access the variable with lib.vflag. You don't even need to use the global keyword.

The best practice for this situation would be to pass vflag into the functions you need as a parameter and to not use global variables.

How can I figure out in my module if the main program uses a specific variable?

I know this does not sound Pythonic, but bear with me for a second.
I am writing a module that depends on some external closed-source module. That module needs to get instantiated to be used (using module.create()).
My module attempts to figure out if my user already loaded that module (easy to do), but then needs to figure out if the module was instantiated. I understand that checking out the type() of each variable can tell me this, but I am not sure how I can get the names of variables defined by the main program. The reason for this is that when one instantiates the model, they also set a bunch of parameters that I do not want to overwrite for any reason.
My attempts so far involved using sys._getframe().f_globals and iterating through the elements, but in my testing it doesn't work. If I instantiate the module as modInst and then call the function in my module, it fails to show the modInst variable. Is there another solution to this? Sample code provided below.
import sys
if moduleName not in sys.modules:
import moduleName
modInst = moduleName.create()
else:
globalVars = sys._getframe().f_globals
for key, value in globalVars:
if value == "Module Name Instance":
return key
return moduleName.create()
EDIT: Sample code included.

Looks like your code assumes that the .create() function was called, if at all, by the immediate/direct caller of your function (which you show only partially, making it pretty hard to be sure about what's going on) and the results placed in a global variable (of the module where the caller of your function resides). It all seems pretty fragile. Doesn't that third-party module have some global variables of its own that are affected by whether the module's create has been called or not? I imagine it would -- where else is it keeping the state-changes resulting from executing the create -- and I would explore that.
To address a specific issue you raise,
I am not sure how I can get the names
of variables defined by the main
program
that's easy -- the main program is found, as a module, in sys.modules['__main__'], so just use vars(sys.modules['__main__']) to get the global dictionary of the main program (the variable names are the keys in that dictionary, along of course with names of functions, classes, etc -- the module, like any other module, has exactly one top-level/global namespace, not one for variables, a separate one for functions, etc).

Suppose the external closed-sourced module is called extmod.
Create my_extmod.py:
import extmod
INSTANTIATED=False
def create(*args,**kw):
global INSTANTIATED
INSTANTIATED=True
return extmod.create(*args,**kw)
Then require your users to import my_extmod instead of extmod directly.
To test if the create function has been called, just check the value of extmod.INSTANTIATED.
Edit: If you open up an IPython session and type import extmod, then type
extmod.[TAB], then you'll see all the top-level variables in the extmod namespace. This might help you find some parameter that changes when extmod.create is called.
Barring that, and barring the possibility of training users to import my_extmod, then perhaps you could use something like the function below. find_extmod_instance searches through all modules in sys.modules.
def find_instance(cls):
for modname in sys.modules:
module=sys.modules[modname]
for value in vars(module).values():
if isinstance(value,cls):
return value
x=find_instance(extmod.ExtmodClass) or extmod.create()