I recently migrated my solution using pytz to dateutil.tz. I've tested everything with iPython, and it worked nicely. However, when in production with python... ERROR!
Can anyone explain why Python and iPython deal differently with the imports, and if there is a way to make python behave the same way as iPython does?
$ python3
Python 3.8.5 (default, Jul 21 2020, 10:48:26)
Type "help", "copyright", "credits" or "license" for more information.
>>> import dateutil
>>> dateutil.__file__
'/usr/local/lib/python3.8/site-packages/dateutil/__init__.py'
>>> dateutil.tz
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'dateutil' has no attribute 'tz'
>>>
$ ipython
Python 3.8.5 (default, Jul 21 2020, 10:48:26)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.19.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import dateutil
In [2]: dateutil.__file__
Out[2]: '/usr/local/lib/python3.8/site-packages/dateutil/__init__.py'
In [3]: dateutil.tz
Out[3]: <module 'dateutil.tz' from '/usr/local/lib/python3.8/site-packages/dateutil/tz/__init__.py'>
Because IPython wants to be too clever. For any reason it has already imported dateutil.tz and when you manually import dateutil, then dateutil.tz is available in you main namespace.
But as is is a submodule, you should consistently import it if you want to use it. So you should replace import dateutil with import dateutil.tz and your code will work fine on both environment.
$ python3
Python 3.8.5 (default, Jul 21 2020, 10:48:26)
Type "help", "copyright", "credits" or "license" for more information.
>>> import dateutil.tz
>>> dateutil.__file__
'/usr/local/lib/python3.8/site-packages/dateutil/__init__.py'
>>> dateutil.tz
<module 'dateutil.tz' from '/usr/local/lib/python3.8/site-packages/dateutil/tz/__init__.py'>
Related
I have distutils installed and it works in some cases. However when trying to use a submodule it won't import unless I explicitly import it.
$ python
Python 3.8.5 (default, Jul 21 2020, 10:42:08)
[Clang 11.0.0 (clang-1100.0.33.17)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import distutils
>>> distutils.spawn
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'distutils' has no attribute 'spawn'
>>> from distutils import spawn
>>> distutils.spawn
<module 'distutils.spawn' from '/usr/local/Cellar/python#3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/spawn.py'>
I'm on Mac and have tried inside a venv and outside of one. I have a dependency that calls distutils.spawn.find_executable('python3') and isn't working.
This is perfectly normal. This is how Python import works. Unless distutils/__init__.py imports .spawn itself (which it doesn't do) you have to import it yourself to make it available. Importing just distutils is not enough to access submodules.
Counterexample: import os makes os.path automatically available but that's because os.py makes it available for you.
When I try to load matplotlib using python I get an error about loading the backports.functools_lru_cache module. When I try to load matplotlib using ipython it loads just fine.
As near as I can tell ipython uses the same version of python that calling python uses. I've gone through the path that both python and ipython use and uninstalled matplotlib and backports.functools_lru_cache. I've tried then reinstalling using apt and pip, always uninstalling the previous attempt before trying the next.
I've gone through a bunch of stackexchange solutions and tried some github solutions without any luck. I run Ubuntu 18.04.
Error:
Python 2.7.15rc1 (default, Apr 15 2018, 21:51:34)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import matplotlib
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/dist-packages/matplotlib/__init__.py", line 127, in <module>
from matplotlib.rcsetup import defaultParams, validate_backend, cycler
File "/usr/lib/python2.7/dist-packages/matplotlib/rcsetup.py", line 29, in <module>
from matplotlib.fontconfig_pattern import parse_fontconfig_pattern
File "/usr/lib/python2.7/dist-packages/matplotlib/fontconfig_pattern.py", line 32, in <module>
from backports.functools_lru_cache import lru_cache
ImportError: No module named functools_lru_cache
I won't show ipython, because it loads just fine.
Update:
If I install using:
pip install matplotlib==2.0.2
matplotlib works for both python and ipython.
The problem appears to be a path problem I am not sure how to solve. I use apt on Ubuntu 18.04 to install system modules into /usr/lib/python2.7/dist-packages/. This directory is where backports.functools_lru_cache lives. I also have some compiled modules installed in /usr/local/lib/python2.7/dist-packages/ where there is a backports module WITHOUT functools_lru_cache. The problem is that python is not following the system path and only seems to be looking in /usr/local/lib/python2.7/dist-packages/ for backports. However, ipython looks in both which is why ipython was working while python was not.
python:
Python 2.7.15rc1 (default, Apr 15 2018, 21:51:34)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/home/brendan/.local/lib/python2.7/site-packages', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/gtk-2.0']
>>> import backports
>>> backports.__path__
['/usr/local/lib/python2.7/dist-packages/backports']
ipython:
Python 2.7.15rc1 (default, Apr 15 2018, 21:51:34)
Type "copyright", "credits" or "license" for more information.
IPython 5.5.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: sys.path
Out[1]:
['',
'/usr/lib/python2.7/dist-packages',
'/usr/lib/python2.7',
'/usr/lib/python2.7/plat-x86_64-linux-gnu',
'/usr/lib/python2.7/lib-tk',
'/usr/lib/python2.7/lib-old',
'/usr/lib/python2.7/lib-dynload',
'/home/brendan/.local/lib/python2.7/site-packages',
'/usr/local/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages/gtk-2.0']
In [2]: import backports
In [3]: backports.__path__
Out[3]:
['/usr/local/lib/python2.7/dist-packages/backports',
'/usr/lib/python2.7/dist-packages/backports']
Solution:
To solve the problem I manually copied functools_lru_cache.py from /usr/lib/python2.7/dist-packages/backports/ to /usr/local/lib/python2.7/dist-packages/backports/. Now I can run matplotlib 2.2.2 no problem in python and ipython.
I'm going to mark this answered because my question now revolves around python and path.
I'm using python 2.7 on ubuntu 16.04. As described in the code below, I can't use any function from the np.matlib, but after I import, then it can be used. Is there any way to troubleshoot the problem? Thanks in advance!
$ python
Python 2.7.12 (default, Dec 4 2017, 14:50:18)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> a = np.matlib.repmat([1,1],1,2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'matlib'
>>> import numpy.matlib as npm
>>> a = npm.repmat([1,1],1,2)
>>> print a
[[1 1 1 1]]
>>>
I think this is a library clash, and if so, how do I know which clashes against which?
The Python import system does not automatically load submodules of a package when a package is imported. NumPy's __init__.py does automatically load most NumPy submodules on a plain import numpy, but numpy.matlib is not included.
Until some code somewhere in the program explicitly imports numpy.matlib, numpy.matlib will not exist, and its contents will not be accessible.
import numpy.matlib as npm
This does not introduce the name numpy.matlib into the namespace; it only introduces the name npm. Python 2.7 doc reference.
If you want the module to be available through both numpy.matlib and npm, you can just define it that way, i.e. npm = numpy.matlib.
I'm trying to import multiple imports in one module, and then import that module and have everything I need imported. Can I do that?
I'm trying it like this:
>>> os.system('cat python_imports.py') # just to show the contents of a file
import sys
0
>>> import python_imports # now import
>>> sys # now use the imported stuff
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'sys' is not defined
So what to do if this doesn't work?
If someone didn't understand the question: When working with python shell; I'm using some same modules everytime, which need to be imported manualy. I don't want to import the same modules every time I start Python shell, I want to import them faster.
So is there any way of doing this?
Thanks!
It looks like you just want a convenient way to import modules into a shell session without a lot of tedious typing.
Python 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> open('python_imports.py').read()
'import sys\n\n'
>>> from python_imports import *
>>> sys
<module 'sys' (built-in)>
>>>
I installed numpy and other packages in virutalenv vpy1. I can correct import those packages if I invoke python command in console after activated the vpy1.
However, if I use IDLE, it cannot import those packages:
>>> import numpy
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
import numpy
ImportError: No module named numpy
I then found out the IDLE does not have correct virtualenv setting. If I invoke python in console, the setting will be like this:
(vpy1)xxx ~ $ python
Python 2.7.6 (default, Jun 1 2014, 03:20:25)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.real_prefix
'/home/xxx/python-2.7.6'
>>> sys.executable
'/home/xxx/virtualenvs/vpy1/bin/python'
if I use IDLE:
(vpy1)xxx ~ $ which idle
/home/xxx/virtualenvs/vpy1/bin/idle
(vpy1)xxx ~ $ idle
#in IDLE
Python 2.7.6 (default, Jun 1 2014, 03:20:25)
[GCC 4.6.3] on linux2
Type "copyright", "credits" or "license()" for more information.
>>> import sys
>>> sys.executable
'/home/xxx/python-2.7.6/bin/python2.7'
>>> hasattr(sys,'real_prefix')
False
How can I fix this?
Finally solved this problem. In the vpy1/bin the idle is directly link to /home/xxx/python-2.7.6/bin/idle, which using a wrong shebang. Change the shebang to #!/usr/bin/env python solve this problem.