I inherited some large and unwieldy python code. In one file its using a list of commands imported from another file. Looking at it with pdb this commands variable ends up in the global namespace. However there's another file that doesn't look like its even being used that also has a commands variable in it and for some reason on certain machines that variable is used instead.
My question is, is there a way in pdb or just code to show the source of the commands variable? I'm hoping for some concrete evidence that shows it's pointing to that file for some reason.
It's a nice demonstration on the dangers of global variables I guess, and I can clean up the code but I'd like to fully understand it first.
To get the module of the commands object, you could try:
import inspect
inspect.getmodule(commands)
Related
I am a beginner in Python, and I am trying to learn by making a simple game. I started by having everything in one big file (let's call it main.py), but it is getting to the point where it has so many classes and functions that I would like to split this code into more manageable components.
I have some experience with LaTeX (although certainly not an expert either) and, in LaTeX there is a function called \input which allows one to write part of the code in a different file. For example, if I have files main.tex and sub.tex which look like:
main.tex:
Some code here.
\input{sub}
Lastly, some other stuff.
and
sub.tex:
Some more code here
then, when I execute main.tex, it will execute:
Some code here.
Some more code here
Lastly, some other stuff.
I wonder, is there a similar thing in Python?
Note 1: From what I have seen, the most commonly suggested way to go about splitting your code is to use modules. I have found this a bit uncomfortable for a few reasons, which I will list below (of course, I understand that I find them uncomfortable because I am a inexperienced, and not because this is the wrong way to do things).
Reasons why I find modules uncomfortable:
My main.py file imports some other modules, like Pygame, which need to be imported into all the new modules I create. If for some reason I wanted to import a new module into main.py later in the process I would then need to import it on every other module I create.
My main.py file has some global variables that are used in the different classes; for example, I have a global variable CITY_SIZE that controls the size of all City instances on the screen. Naturally, CITY_SIZE is used in the definition of the class City. If I were to move the class City to a module classes.py, then I need to define CITY_SIZE on classes.py as well, and if I ever wanted to change the value of CITY_SIZE I would need to change its value on classes.py as well.
Again, suppose that I add a classes.py module where I store all my classes, like City. Then in main.py I need to write classes.City in my code instead of City. I understand this can be overcome by using from classes import City but then I need to add a line of code every time I add a new class to classes.py.
Note 2: I would very much appreciate any comments about how to use modules comfortably in Python, but please note that because this is not my question I would not be able to accept those as valid answers (but, again, they would be appreciated!).
If you have all of your modules in the same directory, you can simply use:
import <name of submodule without .py>
For example, if a submodule file was named sub.py, you would import it like this:
import sub
I'm wondering, if Python offers something similar to the package keyword in Perl. This keyword in effect creates a labeled namespace just anywhere in the code.
As far as I know, similar namespacing in Python is only possible by putting that code into a file and import it. But what if I have the code in a variable (e.g. read from some configuration file of my script)?
So in other words: Is there a way to eval Python code within an arbitrary namespace? In Perl I would just add
package my_pack;
at the beginning of that code and then eval it (within a namespace called my_pack)
Thanks for any help.
No, Perl's and Python's module systems work very differently. It is not possible to explicitly declare a specific Python module.
For a Python eval() or exec() that should execute the code within the context of a particular module, consider which aspects define this module for your purposes – the important aspect is likely that module's global variables. You can provide these explicitly, and capture the current environment via the globals() function. The environment is just a dict, which you can copy if you want to avoid modifications of the module's environment.
I use a lot of Jupyter notebooks for my work to design systems. I find myself doing the same things to set up my environment for each notebook I start, soo I figured I would make a function that does all that for me. It should import a list of modules into the desired namespace. For instance, say this function exists at: my_package.setup_workspace:
def imports():
import numpy as np
I would then like to be able to setup my workspace on a new jupyter notebook with:
import my_package
my_package.setup_workspace
np.array(["I wish this could work!"])
But when I do that, python reports that it can't find np, so the imports() function wasn't able to properly import numpy into the new jupyter workspace.
When you do an import in a function you import things in the local namespace of the function, not the module global one. Unless you declare that the variable are global, but then they need to already exit in outer scope. This allow you to reuse variable name across function without "contaminating" outer scope.
You have a couple ways of going forward:
First self contained way see this twitter thread and this solution, to define your own magic. Then you can then do %imports in a notebook to replace the current cell by a bunch of imports. Advantage is you can pass options, but it works only in IPython.
Second, simply remove the function and do all your setup at the top level of my_package.py, then do from my_package import *. import * are frowned upon in the Python world, but if it's just for you, you should be safe.
Third, look at what %run -i in IPython can do, it's hackish but should work, as it can execute and external scripts in the current namespace.
There is a couple of other crazy way you can investigate once you want to dive deep into how python is working: You can, for example actually manually extract setup_workspace's __code__ object; modify it and re-exec it to not create a local scope, but that's probably not what you'd looking for. It is possible though, but may not be portable across Python versions.
I want a separate Python code where I can define default .py files that have to be created at the start of a project, depending on what models I want. So when I start a new project, I don't have to copy the code from a different project and adjust class names, etc. So for instance, I want to automatically create a model_1.py as:
class Model1(object):
code
and a model_2.py as:
class Model2(object):
code
I want these to be created from another file, where I define which models have to be created. So for instance:
models = ['Model1', 'Model2']
Is it possible to have the class name as a variable? So something like:
class models[0]()
Moreover, is something similar possible for the import part? So
from model_type_x.test import *
where model_type_x is a variable?
What other possibilities are there? Let Python create a text file and turn this into a .py file?
You need this module named cookiecutter. You can have templates for your project and have them configured with a prompt to create your project
First of all, python file are simply text files. You just have to save them with a .py extension.
What you're trying to achieve is more or less out of the scope of python. Python by itself doesn't generate code. If you want to generate code, you can use templates in any language that you like. It doesn't really matter much since the code isn't going to get executed.
Class names and import names cannot be variables. These are syntax sugar allowing you to define types or import code.
If you want to import using a variable name, you can import modules as such:
__import__(module_name)
Where module_name is a variable so you can import modules at runtime with this if you can guess how they are called or going to be imported... Even though it's possible to do that, I do not recommend using this method as it's pretty ugly and pretty much useless to do that that way since we usually know beforehand what we're importing. You can always use the "*" but that's also not a particularly good idea because some things inside a module won't get exported and it's usually better to explicitly tell what you're importing.
class models[0]()
This is clearly not possible, the keyword class is used to define a type. What you can do on the other hand is this:
locals()[models[0]] = type(models[0], parent_tuple, attributes)
But accessing locals() to define a local variable using the type constructor to define a new type manually instead of using the class keyword that makes things much more easily to read... There's just no point to do otherwise.
The real question here is... What are you trying to achieve? Chances are that you're not looking for the right solution to a problem you don't have.
I'm maintaining a dictionary and that is loaded inside the config file. The dictionary is loaded from a JSON file.
In config.py
name_dict = json.load(open(dict_file))
I'm importing this config file in several other scripts(file1.py, file2.py,...,filen.py) using
import config
statement. My question is when will the config.py script be executed ? I'm sure it wont be executed for every import call that is made inside my multiple scripts. But, what exactly happens when an import statement is called.
The top-level code in a module is executed once, the first time you import it. After that, the module object will be found in sys.modules, and the code will not be re-executed to re-generate it.
There are a few exceptions to this:
reload, obviously.
Accidentally importing the same module under two different names (e.g., if the module is in a package, and you've got some directory in the middle of the package in sys.path, you could end up with mypackage.mymodule and mymodule being two copies of the same thing, in which case the code gets run twice).
Installing import hooks/custom imported that replace the standard behavior.
Explicitly monkeying with sys.modules.
Directly calling functions out of imp/importlib or the like.
Certain cases with multiprocessing (and modules that use it indirectly, like concurrent.futures).
For Python 3.1 and later, this is all described in detail under The import system. In particular, look at the Searching section. (The multiprocessing-specific cases are described for that module.)
For earlier versions of Python, you pretty much have to infer the behavior from a variety of different sources and either reading the code or experimenting. However, the well-documented new behavior is intended to work like the old behavior except in specifically described ways, so you can usually get away with reading the 3.x docs even for 2.x.
Note that in general, you don't want to rely on whether top-level code in the module is run once or multiple times. For example, given a top-level function definition, as long as you never compare function objects, or rebind any globals that it (meaning the definition itself, not just the body) depends on, it doesn't make any difference. However, there are some exceptions to that, and loading start-time config files is a perfect example of an exception.