We are having some annoying problems with Eclipse, which apparently have been always there, on Windows 7 + a custom python installation + Eclipse 3.7 and the latest Pydev.
The problem is basically that if when setting up the python interpreter you add all the libraries too, then Eclipse or Pydev or who knows what deletes all these entries from the global easy_install.pth file, which are also configured in Pydev.
This means that within Eclipse everything works fine, but anything outside of it can't possibly work correctly.
Now I find it hard to believe that noone noticed, so I would think that we're doing something wrong.
I also filled a bug which didn't get much attention yet:
https://sourceforge.net/tracker/?func=detail&aid=3446052&group_id=85796&atid=577329
Any idea of what and why it could be?
Already being able to know exactly who and when modify that file would help a little bit, any suggested tool?
EDIT:
I'm trying with Process Monitor as suggested below.
So I fire up Eclipse, I run a simple Python command and I get something like this happened the file.
"File Time","Total Events","Opens","Closes","Reads","Writes","Read Bytes","Write Bytes","Get ACL","Set ACL","Other","Path"
"0.0000306","4","1","1","0","0","0","0","0","0","2","C:\python25\Lib\site-packages\easy-install.pth"
So basically it was open:1, closed:1, other:2.
And the file was actually modified.
My impression is that Eclipse just stupidly overwrites the file, otherwise there should be also some read operations, right?
That, however, doesn't help me that much, I can only see Eclipse as the process modifying the file, not much else.
Ok, I must say I haven't been able to reproduce this so far in PyDev... do you know if that's only happening in your machine (or some other colleague also has the same problem)?
To try to reproduce it, can you give more details on your toolchain: what's your python version and what's your setuptools version? Which packages you have installed? (my feeling is that the culprit is not really Eclipse/PyDev, but some python package that's doing that).
Just to note, PyDev simply doesn't do anything related to .pth files -- all it does is launching your python script with the PYTHONPATH setup in the environment variables for the launched process (so, it's very unlikely that PyDev is writing that file -- simply because it doesn't know anything related to .pth files).
One thing that could help in getting to the root of it: you can try making that file unwritable (you should be able to change its permissions so that you can't modify it), then, try to reproduce and see if you get some error somewhere (because if someone tries to write it, there'll be an exception because you don't get permissions to do so).
Since you are on windows, why not use process monitor? Turn it on, filter for your file(s), then install. It'll log who touches the file, and in what sequence. That should find your culprit.
I haven't really found the fix for the real problem, but at least I found a workardound.
What I do now is to store in a text file all the egg names that should be in the sys.path at run-time.
Before doing anything else I take this list, join these paths with the correct site-packages path, and add what was missing to the sys.path.
It seems to work, but if I remove some libraries (like pyqt) even if it's they are added I still get som errors, so there is still something missing...
import pkg_resources
import sys
from distutils.sysconfig import get_python_lib
from os import path
from psi.devsonly.utils import filename_to_list
#TODO: there appears to be an issue with QT removing the entries
def update_sys_path():
"""Make sure that the sys path contains all the entries needed
"""
site_packages = get_python_lib()
req = pkg_resources.Requirement.parse('psi.devsonly')
wea = pkg_resources.resource_filename(req, 'windows_easy_install.pth')
ppack_list = filename_to_list(wea)
for p in ppack_list:
full_name = path.join(site_packages, p)
# the lower() is necessary or the entries are not found
if full_name.lower() not in sys.path:
print("adding to the path the entry %s" % full_name)
sys.path.insert(0, full_name)
else:
print("%s already present in the path" % full_name)
Related
I have a question regarding simple imports that I cannot get my head around.
Take a look at the attached screenshot to see my project layout.
The file somefile.py imports the class SayHello from a file called someclass.py and calls it. someotherfile.py does the exact same thing. They both use from someclass import SayHello.
In Pycharm both files run. However, From the command line or from VSCode somefile.py runs, but someotherfile.py errors out with the following error:
ModuleNotFound: No module named 'someclass'.
I believe it has something to do with PYTHONPATH/environment variables or something like that, but every explanation I have read has confused me thus far (Even this one which I thought was going to set me strait Relative imports for the billionth time).
Can someone explain in simple terms what is happening here? What is Pycharm doing by default that other editors are not such that my imported modules are found? How can I make someotherfile.py work in VSCode without modifying it?
Thanks in advance!
Pycharm adds your project directory into python paths by default. See configuration of "Pycharm run" that you execute and you shall see few checkboxes like
If those checked Pycharm creates PYTHONPATH environment variable for you that instructs Python where to look for someclass module.
You will have to configure VSCode to define PYTHONPATH environemnt variable for python command you run and include your root project directory path on it.
TLDR: Mess with the line starting with ./automated pointing it to various directories in your project until it works haha.
Long rambling answer: Alright now that I am not in a frenzy from trying to figure this out and it has been a day, I feel like I can make a conherint response (lets see if that is true).
So my original answer was an attempt to simplify my problem into what I thought was the issue due to a ModuleNotFound error I was getting. I had been trying to get unittests in Python inside of Visual Studio code to work (hint hint just use Pycharm it just works out of the box in there), but the integrated testing feature or whatever could not find my tests giving ModuleNotFound as the reason.
The solution to my problem actually just concerned the line ./automated-software-testsing-with-python/blog.
In the below screenshot the relevant part is ./automated-software-testing-with-python/blog.
This is it when it is correctly configured (My tests work Woo hoo!)
Now you can go ahead and read the official documentation for what this line is doing, but I am convinced that the documentation is wrong. Due to how my code is structured and how my files are named, what the documentation says its looking for definitely does not exist. But that is another can of worms...
I actually got it to work by accident. Originally when you go though the wizard to set up what's in that screenshot above, it defaulted to ./automated-software-testing-with-python which did not work. I then manually edited it to something else that was not the below screenshot hoping to get it to work.
It was only when I pointed it to that blog directory on accident (thinking I was in a different file due to trying to debug this for hours and hours in a blind rage) that it ended up working.
I did a bunch of PYTHONPATH manipulation and Environment Variable mumbo jumbo, and I originally thought that that had an effect, but when I cloned my repot to my other machine (which did not have any of that Environment Variable PYTHONPATH stuff going on) it worked too (again, provided that the line in question pointed to blog.
Ok hopefully that might help somone someday, so I'll end it there. Not to end on a bitter sounding zinger, but I will never cease be amazed by how doing such simple things such as configuring the most basic unit test can be so difficult in our profession haha. Well now I can start working. Thanks for the help to those who answered!
I just have a quick question about an error I've been getting when I try to import:
from psychopy import gui
So if I try to import this code in one program I have no problems, however, if I try to import it in another I get:
"ImportError: cannot import name gui"
Does anyone know why this might be happening? Why does it work for one problem, but not the other? Also, I feel like it wasn't doing this before, and it just started suddenly. Any advice would be greatly appreciated.
UPDATE: I think Jon's answer is the correct one. If I was right, you should get an error "no module named psychopy".
Given that you tagged this question with the psychopy tag, my guess is that it works if you run it from the psychopy app and that it doesn't work if you run it from another editor or command line.
The reason is that psychopy is currently shipped as a bundle that comes with it's own python and a lot of modules/dependencies, including psychopy. The system is not made aware of these modules via the PYTHONPATH.
You can make them available system-wide by either (1) following the steps outlined here or (2) use the conda based installation described in this post in the psychopy-dev list. The latter feature is still work in progress but will probably eventually mature to be the default install option.
I think the other answers are wrong ;-)
I think if you had a different virtual environment or installation then the error in your code would indicate "No module named psychopy"
The fact that it finds something called psychopy but no sub-module called gui is a different problem. Often this occurs if you have a folder or file called psychopy next to you current working directory (eg. next to where you launch the script). Then Python thinks that's the psychopy module but can't find gui within it.
So, do you have a folder called psychopy? Rename it psychopyStuff.
I think you are using different virtual environments for both the projects, and so the package is installed in one virtualenv, and not in the other.
To verify this is the case, do a pip freeze in both the projects and compare the results.
If there is a single environment, the output will be same, otherwise the outputs will be different amongst the two.
I am working on an exercise where I need to import a python module called ex25. However, I am getting the following error:
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
import ex25
ImportError: No module named ex25
I already added all possible paths to my system path variables, so I'm not sure why it won't let me import.
The only problem is that, despite what your question and your first comment say, you didn't add the path containing ex25.py to your system paths.
As evidenced by the fact that, once you actually did what you claimed, everything worked fine.
For anyone with a similar problem in the future, just add this above the import:
import sys
print(sys.path)
(If the parentheses look weird, they're just to make the same code work for both Python 2.x and 3.x.)
Now, when you run it, you'll see a list of paths. Is the path to ex25.py in that list? If not, that's your problem. If you don't know how to fix that problem, you have something specific to ask on SO.
Note that "." (that is, the current working directory) is on sys.path. So, running from the same directory as ex25.py will of course fix the problem. Other things on sys.path include:
Your system-wide and user-specific (or, if you're using virtualenv, env-specific) site-packages and dist-packages directories, and anything added to that list by the site module. This is how Python packages that you install become usable in all of your scripts.
Note that if you just download a file and put it somewhere, rather than following its installation instructions or, e.g., using pip on it, it will not be installed.
Any directories that were on the PYTHONPATH environment variable.
Note that this is PYTHONPATH, not PATH. Changing PATH (or Path) in the Windows control panel won't affect this, any more than painting all monks yellow affects monkey or chipmunks.
Also note that changing things in the Windows control panel often does not affect any currently-open cmd.exe ("DOS prompt") windows, only future ones that you create.
Any directories that you explicitly append to sys.path in the script before calling import.
The exact details are more complicated than this, but you really don't want to learn them—at some point you should learn how they work in Python 3.3 and later, but really, nobody wants to know how they worked in 2.7.
I recently upgraded my Python 2.7.1 installation to 2.7.2 using the .msi installer and the process seemed to go OK. However afterwards I happened to be looking at what the default values were for sys.path and noticed the entry for 'C:\Windows\system32\python27.zip'. Wondering exactly what was in it, I decided to try opening it with a Zip utility to check out its contents — however I soon discovered that the file wasn't there (although there is a python27.dll).
Anyhow I'm now wondering if something is wrong. Several existing Python programs I frequently use all seem to work without problems, so I'm not sure whether the installation is messed up or not (or how to fix it, if it is).
Update
I'm aware of and have read PEP273, so know about .zip file modules. That's not what I'm asking about. What I want to know is the fact that there is no python27.zip installed on my system even though it's referred to in my sys.path a problem? I've never encountered problems importing standard Python libraries, which is what I would expect not having one would affect.
From PEP 273 -- Import Modules from Zip Archives:
Just as sys.path currently has default directory names, a default zip
archive name is added too. Otherwise there is no way to import all
Python library files from an archive.
...
The problem is what the name should be. The name should be linked
with the Python version, so the Python executable can correctly find
its corresponding libraries even when there are multiple Python
versions on the same machine.
We add one name to sys.path. On Unix, the directory is sys.prefix +
"/lib", and the file name is "python%s%s.zip" % (sys.version[0],
sys.version[2]). So for Python 2.2 and prefix /usr/local, the path
/usr/local/lib/python2.2/ is already on sys.path, and
/usr/local/lib/python22.zip would be added. On Windows, the file is
the full path to python22.dll, with "dll" replaced by "zip". The zip
archive name is always inserted as the second item in sys.path. The
first is the directory of the main.py (thanks Tim).
What's the shortest way to import a module from a distant, relative directory?
We've been using this code which isn't too bad except your current working directory has to be the same as the directory as this code's or the relative path breaks, which can be error prone and confusing to users.
import sys
sys.path.append('../../../Path/To/Shared/Code')
This code (I think) fixes that problem but is a lot more to type.
import os,sys
sys.path.append(os.path.realpath(os.path.join(os.path.dirname(__file__), '../../../Path/To/Shared/Code')))
Is there a shorter way to append the absolute path? The brevity matters because this is going to have to be typed/appear in a lot of our files. (We can't factor it out because then it would be in the shared code and we couldn't get to it. Chicken & egg, bootstrapping, etc.)
Plus it bothers me that we keep blindly appending to sys.path but that would be even more code. I sure wish something in the standard library could help with this.
This will typically appear in script files which are run from the command line. We're running Python 2.6.2.
Edit:
The reason we're using relative paths is that we typically have multiple, independent copies of the codebase on our computers. It's important that each copy of the codebase use its own copy of the shared code. So any solution which supports only a single code base (e.g., 'Put it in site-packages.') won't work for us.
Any suggestions? Thank you!
Since you don't want to install it in site-packages, you should use buildout or virtualenv to create isolated development environments. That solves the problem, and means you don't have to fiddle with sys.path anymore (in fact, because Buildout does exactly that for you).
You've explained in a comment why you don't want to install "a single site-packages directory", but what about putting in site-packages a single, tiny module, say jebootstrap.py:
import os, sys
def relative_dir(apath):
return os.path.realpath(
os.path.join(os.path.dirname(apath),
'../../../Path/To/Shared/Code'))
def addpack(apath):
relative = relative_dir(apath)
if relative not in sys.path:
sys.path.append(relative)
Now everywhere in your code you can just have
import jebootstrap
jebootsrap.addpack(__file__)
and all the rest of your shared codebase can remain independent per-installation.
Any reason you wouldn't want to make your own shared-code dir under site-packages? Then you could just import import shared.code.module...
You have several ways to handle imports, all documented in the Python language manual.
See http://docs.python.org/library/site.html and http://docs.python.org/reference/simple_stmts.html#the-import-statement
Put it in site-packages and have multiple Python installations. You select the installation using the ordinary PATH environment variable.
Put the directory in your PYTHONPATH environment variable. This is a per-individual-person setting, so you can manage to have multiple versions of the codebase this way.
Put the directory in .pth files in your site-packages. You select the installation using the ordinary PATH environment variable.