I want to print python's sys.path from the command line, but this one is not working:
python -m sys -c "print (sys.path)"
although print -c "import sys; print (sys.path)" would work. It seems that "-m" in the first one above does not load the module "sys". Any clarification on how to correctly import a module from python flags? Thanks.
There is no such flag. -m does something completely different from what you want. You can go through the Python command line docs to see the lack of such a flag if you want.
Just put the import in the command.
python -c "import sys; print (sys.path)"
I am under Ubuntu 16.04 LTS.
I have two python installations. I am actually using them via pvpython, but this is likely irrelevant for the present question.
The versions are:
Python 2.7.12, installed with apt-get, residing in system dirs.
Python 2.7.11, residing in ~/apps/ParaView-5.4.1-Qt5-OpenGL2-MPI-Linux-64bit, simply expanded from a tar file. To get the python prompt I run ~/apps/ParaView-5.4.1-Qt5-OpenGL2-MPI-Linux-64bit/bin/pvpython.
I mean to use readline from version 1 in version 2 (since it does not have its own, strange as it may be).
To do this:
Find where is readline in version 1:
>>> import readline
>>> readline.__file__
'/usr/lib/python2.7/lib-dynload/readline.x86_64-linux-gnu.so'
Use it in version 2, following this. I placed needed stuff in a directory dir1 which is an element of sys.path (I tried with both /home/santiago/apps/ParaView-5.4.1-Qt5-OpenGL2-MPI-Linux-64bit/lib/python2.7 and /home/santiago/apps/ParaView-5.4.1-Qt5-OpenGL2-MPI-Linux-64bit/lib/python2.7/lib-dynload).
2.1. Get the .so file.
$ cd dir1
$ ln -s /usr/lib/python2.7/lib-dynload/readline.x86_64-linux-gnu.so
2.2. Create readline.py
$ nano readline.py
with contents (as per ref above):
def __bootstrap():
global __bootstrap, __loader__, __file__
import sys, pkg_resources, imp
__file__ = pkg_resources.resource_filename(__name__,'readline.x86_64-linux-gnu.so')
__loader__ = None; del __bootstrap, __loader__
imp.load_dynamic(__name__,__file__)
__bootstrap()
Now when I use version 2 with ~/apps/ParaView-5.4.1-Qt5-OpenGL2-MPI-Linux-64bit/bin/pvpython, I still get the error (that I wanted to get rid of)
ImportError: No module named readline
from an import in my ~/.pythonrc.
How can I import readline from version 1 into version 2?
I managed to solve this.
The key was to link with the name readline.so instead of the original name.
The rest was irrelevant.
In 2.1 of the OP:
$ cd ~/apps/ParaView-5.4.1-Qt5-OpenGL2-MPI-Linux-64bit/lib/python2.7/lib-dynload
$ ln -s /usr/lib/python2.7/lib-dynload/readline.x86_64-linux-gnu.so readline.so
That is it.
It turns out that readline.py with __bootstrap (item 2.2) was not needed.
If I have packages installed from easy_install, the eggs are prepended to sys.path before the items in the PYTHONPATH variable.
For example, if I have an egg package called foo installed as well as a package called foo in the current directory, and then do this:
PYTHONPATH="." python
>>> import foo
This will use the egg version of foo instead of the local directory. Inspecting sys.path shows that eggs are placed before items from PYTHONPATH. This seems broken. Is there any way to override this behavior?
Unfortunately this is done with a hard-coded template deep inside setuptools/command/easy_install.py. You could create a patched setuptools with an edited template, but I've found no clean way to extend easy_install from the outside.
Each time easy_install runs it will regenerate the file easy_install.pth. Here is a quick script which you can run after easy_install, to remove the header and footer from easy_install.pth. You could create a wrapper shell script to run this immediately after easy_install:
#!/usr/bin/env python
import sys
path = sys.argv[1]
lines = open(path, 'rb').readlines()
if lines and 'import sys' in lines[0]:
open(path, 'wb').write(''.join(lines[1:-1]) + '\n')
Example:
% easy_install gdata
% PYTHONPATH=xyz python -c 'import sys; print sys.path[:2]'
['', '/Users/pat/virt/lib/python2.6/site-packages/gdata-2.0.14-py2.6.egg']
% ./fix_path ~/virt/lib/python2.6/site-packages/easy_install.pth
% PYTHONPATH=xyz python -c 'import sys; print sys.path[:2]'
['', '/Users/pat/xyz']
For more clarification, here is the format of easy-install.pth:
import sys; sys.__plen = len(sys.path)
./gdata-2.0.14-py2.6.egg
import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p=getattr(sys,'__egginsert',0); sys.path[p:p]=new; sys.__egginsert = p+len(new)
The two import sys lines are the culprit causing the eggs to appear at the start of the path. My script just removes those sys.path-munging lines.
Consider using the -S command-line option to suppress *.pth processing:
python -c 'import sys; print("\n".join(sys.path))'
python -S -c 'import sys; print("\n".join(sys.path))'
https://docs.python.org/3/library/site.html#site.main
You can also use -S with site.main() to delay *.pth processing until runtime, say to capture the original sys.path for appending:
export PYTHONPATH=$(
PYTHONPATH='' \
python -c 'import sys; \
sys.path.extend(sys.argv[1:]); old=list(sys.path); \
import site; site.main(); \
[ old.append(p) for p in sys.path if p not in old ]; \
sys.path=old; \
print ":".join(sys.path)' \
$EXTRA_PATH $ANOTHER_PATH)
python -S ... # using explicit PYTHONPATH
Start from explicit empty PYTHONPATH
Append to sys.path explicitly with extend
Import site and call site.main()
Append new paths to old path and then install it in sys.path
Print with ":" for PYTHONPATH
python -S is desirable for later runs only using $PYTHONPATH
python -S may or may not be desirable while setting PYTHONPATH (depending on if you need sys.path expanded before extending)
I have done something like the following to prepend to the system path when running a top-level python executable file:
import sys
sys.path = ["<your python path>"] + sys.path
Often, the "<your python path>" for me involves use of the __file__ attribute to do relative look up for a path that includes the top-level module for my project. This is not recommended for use in producing, eggs, though I don't seem to mind the consequences. There may be another alternative to __file__.
How do I learn where the source file for a given Python module is installed? Is the method different on Windows than on Linux?
I'm trying to look for the source of the datetime module in particular, but I'm interested in a more general answer as well.
For a pure python module you can find the source by looking at themodule.__file__.
The datetime module, however, is written in C, and therefore datetime.__file__ points to a .so file (there is no datetime.__file__ on Windows), and therefore, you can't see the source.
If you download a python source tarball and extract it, the modules' code can be found in the Modules subdirectory.
For example, if you want to find the datetime code for python 2.6, you can look at
Python-2.6/Modules/datetimemodule.c
You can also find the latest version of this file on github on the web at
https://github.com/python/cpython/blob/main/Modules/_datetimemodule.c
Running python -v from the command line should tell you what is being imported and from where. This works for me on Windows and Mac OS X.
C:\>python -v
# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
# C:\Python24\lib\site.pyc has bad mtime
import site # from C:\Python24\lib\site.py
# wrote C:\Python24\lib\site.pyc
# C:\Python24\lib\os.pyc has bad mtime
import os # from C:\Python24\lib\os.py
# wrote C:\Python24\lib\os.pyc
import nt # builtin
# C:\Python24\lib\ntpath.pyc has bad mtime
...
I'm not sure what those bad mtime's are on my install!
I realize this answer is 4 years late, but the existing answers are misleading people.
The right way to do this is never __file__, or trying to walk through sys.path and search for yourself, etc. (unless you need to be backward compatible beyond 2.1).
It's the inspect module—in particular, getfile or getsourcefile.
Unless you want to learn and implement the rules (which are documented, but painful, for CPython 2.x, and not documented at all for other implementations, or 3.x) for mapping .pyc to .py files; dealing with .zip archives, eggs, and module packages; trying different ways to get the path to .so/.pyd files that don't support __file__; figuring out what Jython/IronPython/PyPy do; etc. In which case, go for it.
Meanwhile, every Python version's source from 2.0+ is available online at http://hg.python.org/cpython/file/X.Y/ (e.g., 2.7 or 3.3). So, once you discover that inspect.getfile(datetime) is a .so or .pyd file like /usr/local/lib/python2.7/lib-dynload/datetime.so, you can look it up inside the Modules directory. Strictly speaking, there's no way to be sure of which file defines which module, but nearly all of them are either foo.c or foomodule.c, so it shouldn't be hard to guess that datetimemodule.c is what you want.
If you're using pip to install your modules, just pip show $module the location is returned.
The sys.path list contains the list of directories which will be searched for modules at runtime:
python -v
>>> import sys
>>> sys.path
['', '/usr/local/lib/python25.zip', '/usr/local/lib/python2.5', ... ]
from the standard library try imp.find_module
>>> import imp
>>> imp.find_module('fontTools')
(None, 'C:\\Python27\\lib\\site-packages\\FontTools\\fontTools', ('', '', 5))
>>> imp.find_module('datetime')
(None, 'datetime', ('', '', 6))
datetime is a builtin module, so there is no (Python) source file.
For modules coming from .py (or .pyc) files, you can use mymodule.__file__, e.g.
> import random
> random.__file__
'C:\\Python25\\lib\\random.pyc'
Here's a one-liner to get the filename for a module, suitable for shell aliasing:
echo 'import sys; t=__import__(sys.argv[1],fromlist=[\".\"]); print(t.__file__)' | python -
Set up as an alias:
alias getpmpath="echo 'import sys; t=__import__(sys.argv[1],fromlist=[\".\"]); print(t.__file__)' | python - "
To use:
$ getpmpath twisted
/usr/lib64/python2.6/site-packages/twisted/__init__.pyc
$ getpmpath twisted.web
/usr/lib64/python2.6/site-packages/twisted/web/__init__.pyc
In the python interpreter you could import the particular module and then type help(module). This gives details such as Name, File, Module Docs, Description et al.
Ex:
import os
help(os)
Help on module os:
NAME
os - OS routines for Mac, NT, or Posix depending on what system we're on.
FILE
/usr/lib/python2.6/os.py
MODULE DOCS
http://docs.python.org/library/os
DESCRIPTION
This exports:
- all functions from posix, nt, os2, or ce, e.g. unlink, stat, etc.
- os.path is one of the modules posixpath, or ntpath
- os.name is 'posix', 'nt', 'os2', 'ce' or 'riscos'
et al
On windows you can find the location of the python module as shown below:i.e find rest_framework module
New in Python 3.2, you can now use e.g. code_info() from the dis module:
http://docs.python.org/dev/whatsnew/3.2.html#dis
Check out this nifty "cdp" command to cd to the directory containing the source for the indicated Python module:
cdp () {
cd "$(python -c "import os.path as _, ${1}; \
print _.dirname(_.realpath(${1}.__file__[:-1]))"
)"
}
Just updating the answer in case anyone needs it now, I'm at Python 3.9 and using Pip to manage packages. Just use pip show, e.g.:
pip show numpy
It will give you all the details with the location of where pip is storing all your other packages.
On Ubuntu 12.04, for example numpy package for python2, can be found at:
/usr/lib/python2.7/dist-packages/numpy
Of course, this is not generic answer
Another way to check if you have multiple python versions installed, from the terminal.
$ python3 -m pip show pyperclip
Location: /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
$ python -m pip show pyperclip
Location: /Users/umeshvuyyuru/Library/Python/2.7/lib/python/site-packages
Not all python modules are written in python. Datetime happens to be one of them that is not, and (on linux) is datetime.so.
You would have to download the source code to the python standard library to get at it.
For those who prefer a GUI solution: if you're using a gui such as Spyder (part of the Anaconda installation) you can just right-click the module name (such as "csv" in "import csv") and select "go to definition" - this will open the file, but also on the top you can see the exact file location ("C:....csv.py")
If you are not using interpreter then you can run the code below:
import site
print (site.getsitepackages())
Output:
['C:\\Users\\<your username>\\AppData\\Local\\Programs\\Python\\Python37', 'C:\\Users\\<your username>\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages']
The second element in Array will be your package location. In this case:
C:\Users\<your username>\AppData\Local\Programs\Python\Python37\lib\site-packages
In an IDE like Spyder, import the module and then run the module individually.
enter image description here
as written above
in python just use help(module)
ie
import fractions
help(fractions)
if your module, in the example fractions, is installed then it will tell you location and info about it, if its not installed it says module not available
if its not available it doesn't come by default with python in which case you can check where you found it for download info