I'm trying to use python's bitstring module in a script and am getting an import error. This error does not happen when running from interactive mode.
Here's the code:
import bitstring
b = bitstring.BitArray(bin='001001111')
When run like this:
python test.py
I get this:
AttributeError: 'module' object has no attribute 'BitArray'
However, when I do this:
$ python
Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import bitstring
>>> b = bitstring.BitArray(bin='001001111')
>>> print b
0b001001111
It works just fine! It's the same interpreter being run by the same user. Any pointers?
I predict you have created a bitstring.py in your current directory.
The problem is caused by a bitstring.py file in sys.path of test.py, but not in that of the interactive python shell. Most likely, there's a bitstring.py file in the directory test.py is in, and you started your shell from another working directory.
Since python traverses sys.path from front to end, modules in the current directory - even if accidentally created - overshadow those in system library directories.
Google App Engine actually had a similar issue at one point. The easiest solution there was simply comment the offending line or use try...except. Obviously that won't work here.
In that case, the problem was initialization order. A half second later a similar line of code was called again with success. Their solution? refactor. :-(
The best I've seen is a dynamic lookup of the class: bitstring.__dict__.get("BitArray") or getattr(bitstring, "BitArray");. It isn't ideal (and I believe I've even seen those return null), but hopefully it can get you somewhere.
Related
This question is related to Can't import module with importlib.import_module, which I don't think got to the root of the issue at all.
I'm trying to understand the reason for differences with importlib used in the interactive interpreter vs. a non-interactive script, because I'm getting different results with each.
I have a folder containing myscript.py, and I want it to import a module called module from a subfolder, my_subdir of the initial working directory. I want to do this by moving into that directory and working there. I know there are other solutions to accessing a module in a subdirectory, but this is the use case that I'd like to talk about.
The following is the contents of myscript.py, which simply moves into the subdirectory and tries to import the module from there, and then call some hello-world function from the module:
#!/home/chris/anaconda2/bin/python
import sys
import platform
import os
import importlib
print("Python version "+ platform.python_version())
print(sys.path)
os.chdir('my_subdir')
my_module = importlib.import_module('module')
my_module.my_function()
When I run this from the command line, Python can't find the module. It seems that the import system doesn't know about the change of directory from os.chdir():
(base) chris#linux-om3m:~/workspace> python myscript.py
Python version 2.7.15
['/home/chris/workspace', '/home/chris/anaconda2/lib/python27.zip', '/home/chris/anaconda2/lib/python2.7', '/home/chris/anaconda2/lib/python2.7/plat-linux2', '/home/chris/anaconda2/lib/python2.7/lib-tk', '/home/chris/anaconda2/lib/python2.7/lib-old', '/home/chris/anaconda2/lib/python2.7/lib-dynload', '/home/chris/anaconda2/lib/python2.7/site-packages', '/home/chris/anaconda2/lib/python2.7/site-packages']
Traceback (most recent call last):
File "myscript.py", line 11, in <module>
my_module = importlib.import_module('module')
File "/home/chris/anaconda2/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
ImportError: No module named module
However, when I type the same script into the interactive interpreter, it works fine:
(base) chris#linux-om3m:~/workspace> python
Python 2.7.15 | packaged by conda-forge | (default, Jul 2 2019, 00:39:44)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> import platform
>>> import os
>>> import importlib
>>> print("Python version "+ platform.python_version())
Python version 2.7.15
>>> print(sys.path)
['', '/home/chris/anaconda2/lib/python27.zip', '/home/chris/anaconda2/lib/python2.7', '/home/chris/anaconda2/lib/python2.7/plat-linux2', '/home/chris/anaconda2/lib/python2.7/lib-tk', '/home/chris/anaconda2/lib/python2.7/lib-old', '/home/chris/anaconda2/lib/python2.7/lib-dynload', '/home/chris/anaconda2/lib/python2.7/site-packages', '/home/chris/anaconda2/lib/python2.7/site-packages']
>>> os.chdir('my_subdir')
>>> my_module = importlib.import_module('module')
>>> my_module.my_function()
Here I am!
>>>
I can fix this issue by manually updating sys.path with a sys.path.append(os.getcwd()) after I do the os.chdir - but I do not understand why this is only necessary in non-interactive mode.
The only thing I do notice (and is printed here in the pasted output) is that under the interactive interpreter, sys.path has an empty-string entry, whereas in non-interactive mode it contains the working dir where I invoked python.
I know I could fix this problem using relative imports, and for now I'm fixing the problem using a sys.path.append(os.getcwd()), but I'd like to understand why there is a difference between running interactively vs. non-interactively.
I'm getting the same results with Python 2.7 and Python 3.6.
As you noted in your comment, appending '' to sys.path fixes the issue. This is because '' represents your cwd, so os.chdir() knows where to find your subdirectory my_subdir correctly (see Why is the first element in python's sys.path an empty string?).
Using an absolute path for my_subdir should also fix your issue.
My code needs to be able to search through a list of directories for a specific file. It needs to handle an alias as part of a path name and still find the actual path to a file. For example, it needs to turn something like $HOME/test.dat into /home/greenmatt/test.dat. As such, I'm trying to use os.path.abspath.
Unfortunately, the results come back with the alias name glued onto the end of the actual directory name. The following demonstrates what I'm seeing.
From the command line:
> echo $HOME
/home/greenmatt
Then inside Python:
> python
Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> print(os.path.abspath('$HOME'))
/home/greenmatt/$HOME
os.path.realpath does the same thing:
>>> print(os.path.realpath('$HOME'))
/home/greenmatt/$HOME
Why is the extra '$HOME' getting added to the path? How do I get the absolute path without it?
os.path.expandvars(path) expands environment variables in a path. After expanding, the absolute path can be created. Together:
os.path.abspath(os.path.expandvars(path))
HOME is a shell variable, and abspath doesn't use the shell at all, so the parameter isn't expanded. As a result, it's treated as a literal string naming a subdirectory in your current directory. You have to expand it yourself. Since HOME is a standard environment variable, you can use os.getenv.
> os.path.abspath(os.getenv('HOME')) # $ must be omitted
/home/greenmatt
(Of course, if it wasn't an environment variable, you couldn't access it at all, even if it was defined in the shell from which you started Python.)
Consider this an explanation of why your original attempt didn't work. In practice, use expandvars as suggested by Michael Butscher.
My default Python binary is set to the one with the Anaconda distribution of Python. This is found at /home/karnivaurus/anaconda/bin/python, and I have made this the default by adding to my .bashrc file the following: export PATH=/home/karnivaurus/anaconda/bin:$PATH.
I also have a Python package called caffe, which is located at /home/karnivaurus/caffe/distribute/python, and I have added this to the package search path by adding to my .bashrc file the following: export PYTHONPATH=${PYTHONPATH}:/home/karnivaurus/caffe/distribute/python.
Now, I have a simple Python file, called test.py, with the following contents:
import caffe
print "Done."
If I run this by entering python test.py into the terminal, it runs fine, printing out "Done.". The problem I am having is when I run this in the PyCharm IDE. In PyCharm, I have set the interpreter to be /home/karnivaurus/anaconda/bin/python. But when I open test.py in PyCharm, and run the file in the IDE, I get the following error:
ImportError: No module named caffe
So my question is: Why can PyCharm not find the caffe module when it runs the Python script, but it can be found when I run the script from the terminal?
There are a few things that can cause this. To debug, please modify your test.py like so:
# Is it the same python interpreter?
import sys
print(sys.executable)
# Is it the same working directory?
import os
print(os.getcwd())
# Are there any discrepancies in sys.path?
# this is the list python searches, sequentially, for import locations
# some environment variables can fcuk with this list
print(sys.path)
import caffe
print "Done."
Try again in both situations to find the discrepancy in the runtime environment.
edit: there was a discrepancy in sys.path caused by PYTHONPATH environment variable. This was set in the shell via .bashrc file, but not set in PyCharm's runtime environment configuration.
For an additional option, you can use pycharm by terminal. And export the corresponding environment paths beforehand. This works for me. And I think it's better than make some changes in the code. You gonna need run the code by terminal after your debugging.
For example, in terminal type:
$ export LD_LIBRARY_PATH=~/build_master_release/lib:/usr/local/cudnn/v5/lib64:~/anaconda2/lib:$LD_LIBRARY_PATH
$ export PYTHONPATH=~/build_master_release/python:$PYTHONPATH
Then run pycharm by charm (pycharm can be soft linked by charm bash):
$ charm
Well this may be a redundant answer, however I think it's important to explicitly called out what causes this error.
It happened to me many times and I got it fixed by making sure that IDE ( pycharm or vscode or any other) is set to same working directory where the code resided.
for example : I have two files train.py and config.py in mlproject/src directory. I'm trying to run import config in train.py
**When run in /mlproject/ directory, I get error when try to import config **
(ml) dude#vscode101:~/mlproject$ python
Python 3.7.6 (default, Jan 8 2020, 19:59:22)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import config
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'config'
>>>
When run in /mlproject/src/` directory, I'm able to successfully import config
(ml) dude#vscode101:~/mlproject/src$ python
Python 3.7.6 (default, Jan 8 2020, 19:59:22)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import config
>>>
I just installed lxml for parsing xml file in python. I am using TextMate as an IDE. Problem is that when I try to import lxml (from lxml import entree) then I get
ImportError:'No module named lxml'
But when I use Terminal then everything is fine
Python 2.7.2 (v2.7.2:8527427914a2, Jun 11 2011, 15:22:34)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from lxml import etree
>>> root=etree.element("root")
>>> root=etree.Element("root")
>>> print (root.tag)
root
>>> root.append(etree.Element("child1"))
>>> child2 = etree.SubElement(root, "child2")
>>> child3 = etree.SubElement(root, "child3")
>>> print (etree.tostring(root,pretty_print=True))
<root>
<child1/>
<child2/>
<child3/>
</root>
It's pretty weird. Does it have something to do with TextMate?
Suggestion Please!
This most probably means that you have more than one python installation on your system and that TextMate and the Terminal using different ones by default.
One workaround: In your python file, you can specify an interpreter directive to point to the python installation (and executable) of your choice:
#!/usr/local/bin/python
# Even thought standard python is in /usr/bin/python, here we want another ...
You need to define the shell variables in TextMate's settings, specifically 'TM_PYTHON' needs to point to your Python binary.
To find which Python your using, in a terminal you could type 'which python'
It's likely that TextMate is using a different PYTHONPATH than your terminal. I'm not a TextMate user so I can't help you there, but it should point you in the right direction.
You might be be running a different version of Python from TextMate. I had a similar issue with RedHat having 2 versions of Python. I had installed the module to one, but was trying to execute with another.
Does anyone know where i can find this python module 'contextlib'?
root#overo:~# python
Python 2.6.6 (r266:84292, Mar 9 2011, 10:05:36)
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import contextlib
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named contextlib
I did not compile python myself personally. I'm just in this mess. It's running on an ARM based CPU so maybe some things were left out to save space.
I ran find / | grep contextlib which resulted in nothing.
Can i download this module from somewhere and just plonk it in /usr/lib/python2.6? Will that work?
I got this error in a different way.
I created a pipenv virtual environment using the 32bit version of Python 3.6.5 on Windows 10. I then realized I needed the 64bit version. Uninstalled the 32bit, installed the 64bit, and then tried to go back to my existing virtual env. The previously created env was now broken in odd ways and gave me this error.
I solved this by removing the old pipenv pipenv --rm and creating a new one with the newly installed version of python.
As others have noted, that module should be in the standard library, but if it's an embedded device, it may have been dropped to save space (if true, a foolish choice IMO, since leaving out contextlib.contextmanager robs the with statement of much of its power and convenience)
If you can name the specific device or manufacturer (or ask the vendor directly), you may be able to get a better answer.
As far as fixing it goes, grabbing http://hg.python.org/cpython/file/2.6/Lib/contextlib.py and dropping it in sys.path somewhere should do the trick (running python -m site will dump the list of directories that you can use)
It has been part of the standard library since 2.5 according to the docs. It seems a bit weird that you don't have it, it works with 2.6.6 for me (Ubuntu 10.10):
blair#blair-eeepc:~$ python
Python 2.6.6 (r266:84292, Sep 15 2010, 15:52:39)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import contextlib
>>> contextlib.__file__
'/usr/lib/python2.6/contextlib.pyc'
Somebody may have a better suggestion, but if it comes to it there is a link at the top of the documentation to the source code (which is Python, so you should be able to use it directly without any compilation or anything).
Edit: Unless, as Santiago Lezica suggested, you compiled your copy of Python manually, in which case it should be a simple matter of copying the module into the correct library path.
Edit for updated question: To the best of my knowledge, just dropping the source into a directory on the Python path should work. You could do this in the system library, but, to avoid it being deleted/replaced/otherwise borked in future updates, I'd recommend putting it in a separate directory and adding that directory to the Python path. You could put it under /usr/local, or somewhere in your home directory.
With Angsrom Linux, contextlib is included in the python-misc package. You can grab it by running:
opkg install python-misc
This won't, however, get you all expected python modules, so you may also want to install python-modules:
opkg install python-modules
I found one more occasion, which produces the same error.
I had made a virtual environment with python 3.6. After a updated my python version to 3.7 I tried to activate the old virtual environment and got this error.
The solution was to delete the old environment and recreate it with the new python version.
Check sys.path to make sure your python interpreter is looking in the right directories. It should look something like this (not necessarily identical):
>>> import sys
>>> sys.path
['', '/usr/lib/python2.6', '/usr/lib/python2.6/plat-linux2', '/usr/lib/python2.6/lib-tk', '/usr/lib/python2.6/lib-old', '/usr/lib/python2.6/lib-dynload', '/usr/local/lib/python2.6/dist-packages', '/usr/lib/python2.6/dist-packages', '/usr/lib/python2.6/dist-packages/PIL', '/usr/lib/pymodules/python2.6']
EDIT: With the updated information in the question that this is an install of unknown origin on a constrained device, assuming that unnecessary modules were removed to save space makes sense. However, for the record, I'll mention another, perhaps more common scenario where modules cannot be found: when there are file permissions issues. For example:
$ python -c 'import contextlib; print(contextlib.__file__)'
/usr/lib/python2.6/contextlib.pyc
$ ls -l /usr/lib/python2.6/contextlib.py*
-rw-r--r-- 1 root root 4136 Dec 26 16:42 /usr/lib/python2.6/contextlib.py
-rw-r--r-- 1 root root 4127 Jan 1 21:45 /usr/lib/python2.6/contextlib.pyc
$ sudo chmod go-r /usr/lib/python2.6/contextlib.py*
$ python -c 'import contextlib; print(contextlib.__file__)'
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named contextlib
Especially with custom installations, import problems due to file permission issues and path problems are some of the easiest things to check and, usually, to fix.
Python 2
sudo apt-get install python-contextlib2
Python 3
sudo apt-get install python3-contextlib2
contextlib was introduced in Python 2.5, can you remove and re-install your Python 2.6.6 again? From my copy of Python 2.6.6:
Python 2.6.6 (r266:84292, Sep 15 2010, 15:52:39)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import contextlib
>>>