I have a Python library I wrote which has been acting up on me. I have a set of variables which change the way the library works. In testing it all worked fine, but when I python lib.py install the variables have no effect on the library. I broke this down to the simplest example possible:
Library:
##lib.py
config="Original"
def run():
print config
Script:
import lib
lib.config="New"
lib.run()
print lib.config
If you place the library in the same directory as the script and run it the output is:
New
New
But if you install the library and then try the script using the library from the dist-packages the output is:
Original
New
Could someone explain what is going on? I'm a bit confused and terribly interested in the happenings and reason. Additionally am I doing programatical configuration totally wrong?
Edit
It turns out the problem is the init.py file. It's basically like importing a library that just imports another library. When you import an installed module it looks at the folder lib and the file init.py. init.py is just a one liner from lib import *. It simply pretends to be the actual library, but that causes an odd problem if you use a global variable. A simulated example of what is essentially going on:
##init.py
from lib import *
Script:
import init
init.config = 'New'
init.run()
print init.config
Output:
Original
New
The function run() looks for config in lib.py, but print init.config looks for it in init.py. Thanks for the help everyone. Fix is to change the way the module installs (no init.py). Eventually, I hope to remove all the global variables, but for the time being everything works perfect.
What you describe would be inconsistent with how Python works (read, if you like, "I don't believe you did precisely that and got precisely that result").
If you go importing lib from different places or in different ways, though, you could be ending up with two copies of it, either two copies of one of the modules or one of the current-directory lib and the other the installed lib. If you are getting this "Original"/"New" behaviour, that seems to me the most likely reason.
Related
I have a project which worked perfectly fine, but I wanted to separate some of the code into a separate project (with its own packages). I have successfully done that, but now in that newly separated package I need to explicitly import local modules using from . import <module> instead of import <module> - the latter of which had worked before separating the modules into their own package.
The new imports (where I include the dot representing the local dir) works but it seems like it should also just work without having to specify the module locations since the files are in the same directory as the module that is trying to import them. Am I missing something here?
Artificial example:
|-root
|--ProjectA
|---app.py
|-root
|--ProjectB
|---ProjectB
|----__init__.py
|----libA.py
|----libB.py
In app.py I am able to import a module from ProjectB (I have adjusted PYTHONPATH to include /root/ProjectB). So the following works just fine as I would expect:
from ProjectB import libA.py
But the entire thing fails because in libA.py the following does not work:
import libB
Instead I have to use:
from . import libB
I have tried to wrap my head around how python imports work across packages and I thought I had sorted it in my head, but apparently I am missing something. Unless this is the way it is supposed to work? (packages need to explicitly indicate that a module is local). But I thought python would always implicitly add the path of the current module that is doing the importing.
Any advice (or even just links to tutorials that explain this - I have watched a lot but they all tend to focus on different aspects of importing than this) is greatly appreciated.
I am testing my own package, but I am struggling to import it. My package file structure is as follows:
(You can also alternatively view my Github repository here)
In PyAdventures is my init.py with the content of
name="pyadventures"
So my question is, when I import it in another python file, how come it doesn't work?
I run:
import pyadventures
But I get the following error:
No module named pyadventures
It'd be great if you could answer my question!
It's important to note that the package is in my Python package directory, not the test one I showed
New discovery! In my PyAdventures folder (the one Python actually uses) the folder only has a few files, not the ones in the screenshot above.
You can install this with pip install pyadventures
Ah, like others remark in comments and answer: The camelcase might be the problem. (Why not name it just all in lower case: pyadventures? - Else users will be as confused as its developer now :D .)
Before, I thought, it might be a problem that you want to use your module without having installed it (locally). And my following answer is for this case (using a not installed package from a local folder):
In the docs you can read:
The variable sys.path is a list of strings that determines the
interpreter’s search path for modules. It is initialized to a default
path taken from the environment variable PYTHONPATH, or from a
built-in default if PYTHONPATH is not set. You can modify it using
standard list operations:
import sys
sys.path.append('/ufs/guido/lib/python')
thus, before import do:
import sys
sys.path.append('/absolute/or/relative/path/to/your/module/pyadventures')
# Now, your module is "visible" for the module loader
import pyadventures
Alternatively, you could just place your pyadventures module folder locally in your working directory where you start python and then just do
import pyadventures
However, it is much easier to manage to keep only one version in one place and refer to this from other places/scripts. (Else you have multiple copies, and have difficulties to track changes on the multiple copies).
As far as I know, your __init__.py doesn't need to contain anything. It works if it is empty. It is there just to mark your directory as a module.
Simple. Even though the package listed on pip is namd pyadventures, in your code the directory is called PyAdventures. So that's what python knows it as. I ran import PyAdventures and it worked fine.
I'm working through Learn Python the Hard way, and I'm currently on exercise 46 where you learn to create packages. I've created a basic package that does a few calculations, and uses a few different modules.
I've gotten the package to install in my python2.7 site packages, but I can't seem to run the module from my site packages after the fact. I'm wondering if the path that python is searching is different, because of the following:
After the install, I see this message Copying story-0.1-py2.7.egg to /usr/local/lib/python2.7/site-packages
However, when I try to run the module, I see this message /usr/local/Cellar/python/2.7.12_1/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: can't open file 'story.py': [Errno 2] No such file or directory
Sorry if this makes absolutely no sense, I'm very new to the fantastical world of programing.
When you install a package, you access the internals differently. You can no longer just call python story.py.
To access the functions in story.py you need to import the module at the top of another python file, or in the interpreter.
If story.py contained
def my_test_function(blah):
print blah
You would use this function in another file by the following (after installing the module, as you've already done)
import story
story.my_test_function("Hello!")
By importing the module, you get access to all the functions and classes inside it by typing module_name.function_name. You could also just import the function you wanted to use directly, which would look something like
from story import my_test_function
my_test_function("Hello!")
I have two packages in the same namespace with tests, the structure is like this:
Package-1/namespace/__init__.py
Package-1/namespace/module1/__init__.py
Package-2/namespace/__init__.py
Package-2/namespace/module2/__init__.py
Package-2/tests/foo_test.py
Package 1 is properly installed using pip. Now Package 2 depends on Package 1, so foo_test contains the line
import namespace.module1
When I try to run nosetests ./tests/foo_test.py from the directory Package-2 I get an import error because Python complains that it did not find namespace.module1. I am quite sure that the problem is that it tries to search the path Package-2/namespace for the directory module1 which is of course not present. I would like Python to load module1 from the installed packages and not search for it in the current path.
I am not entirely sure, but I think that here a similar issue is explained but to my understanding this should have been fixed. Does anybody have an idea how to work around this? Or am I supposed to structure things differently?
There is a solution for this, at least in Python 3.3+. The __init__.py files that are present in Package-i/namespace/ have to be deleted. I don't understand exactly why that solves the problem but it works in my case...
I have this structure of a python project:
RF
\__init__.py
----tools
--------\__init__.py
--------drawtools.py
----examples
--------\__init__.py
--------something.py
All __init__.py are left blank. Now, in "something.py" I type:
from RF.tools.drawtools import *
And I get:
ImportError: No module named RF.tools.drawtools
What's the correct program structure? Do I have to put something in the init files?
I notice that if "something.py" is in the top directory it works. The strange thing is that PyCharm, the IDE I'm using, seems to recognize the import and give me code completion.
I heard something about setting PYTHONPATH but as this project must be shared in a team I'd prefer to keep things as simple as possible (you copy the project from one to one and run it without any annoying importError).
I think that it would work with a relative import, such as
from .. import drawtools