For me, it's located at C:\Python33\libs.
For reference - this is not the same folder as C:\Python33\Lib - note the capitalization and lack of an 's'.
On one computer I was working on, I simply dropped a .py file into the libs folder and could import and use it like a library / module (sorry, I don't really know terminology very well), regardless of where the project I was working on is.
However, in trying to duplicate this on another machine, this doesn't work. Attempting to import simply gives a "no module named X" error.
So, clearly I'm misunderstanding the purpose of the libs folder, and how it differs from the Lib folder.
So, what exactly is the difference?
If you compare libs/ vs. Lib/ you'll notice that the latter is full of *.py files and the former has *.lib files. Further investigation with a text editor will show that *.py files are human-readable (I hope) and the *.lib files are not.
And that's really the difference. If you want to know more, the .lib files are static-link libraries, used for building .dlls, C extensions, and all that good stuff. Head on down the rabbit hole if that interests you.
On to the meat of your question: are you supposed to be able to drop modules in there and be able to import them? Not really. That is a side effect of that folder being included in your path. From the Modules docs:
When a module named spam is imported, the interpreter first searches
for a built-in module with that name. If not found, it then searches
for a file named spam.py in a list of directories given by the
variable sys.path. sys.path is initialized from these locations:
the directory containing the input script (or the current directory).
PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
the installation-dependent default.
Various installation methods will modify %PATH% or %PYTHONPATH% so I can't tell you exactly where to look; on my windows box, the python installer modified %PATH% for me, so you should probably look there first. Notably, my path does not include Python33/libs/ so I would not expect it to be there by default.
Just looking on mine (Windows 7) /libs appears to be the native code libraries (*.lib) vs the straight python libraries in /Lib. The readme also mentions a configuration flag:
--with-libs='libs': Add 'libs' to the LIBS that the python interpreter
is linked against.
Which may or may not be set on different installs/platforms.
This isn't really a answer; hopefully someone with a firmer knowledge of it will explain further - was just a bit too much info to squeeze into a comment.
Related
I have a case for needing to add a path to a python package to sys.path (instead of its parent directory), but then refer to the package normally by name.
Maybe that's weird, but let me exemplify what I need and maybe you guys know how to achieve that.
I have all kind of experimental folders, modules, etc inside a path like /home/me/python.
Now I don't want to add that folder to my sys.path (PYTHONPATH) since there are experimental modules which names could clash with something useful.
But, inside /home/me/python I want to have a folder like pyutils. So I want to add /home/me/python/pyutils to PYTHONPATH, but, be able to refer to the package by its name pyutils...like I would have added /home/me/python to the path.
One helpful fact is that adding something to the python path is different from importing it into your interpreter. You can structure your modules and submodules such that names will not clash.
Look here regarding how to create modules. I read the documents, but I think that module layout takes a little learning-by-doing, which means creating your modules, and then importing them into scripts, and see if the importing is awkward or requires too much qualification.
Separately consider the python import system. When you import something you can use the "import ... as" feature to name it something different as you import, and thereby prevent naming clashes.
You seem to have already understood how you can change the PYTHONPATH using sys.path(), as documented here.
You have a number of options:
Make a new directory pyutilsdir, place pyutils in pyutilsdir,
and then add pyutilsdir to PYTHONPATH.
Move the experimental code outside of /home/me/python and add python to
your PYTHONPATH.
Rename the experimental modules so their names do not clash with
other modules. Then add python to PYTHONPATH.
Use a version control system like git or hg to make the
experimental modules available or unavailable as desired.
You could have a master branch without the experimental modules,
and a feature branch that includes them. With git, for example, you could switch between
the two with
git checkout [master|feature]
The contents of /home/me/python/pyutils (the git repo directory) would
change depending on which commit is checked out. Thus, using version control, you can keep the experimental modules in pyutils, but only make them present when you checkout the feature branch.
I'll answer to my own question since I got an idea while writing the question, and maybe someone will need that.
I added a link from that folder to my site-packages folder like that:
ln -s /home/me/python/pyutils /path/to/site-packages/pyutils
Then, since the PYTHONPATH contains the /path/to/site-packages folder, and I have a pyutils folder in it, with init.py, I can just import like:
from pyutils import mymodule
And the rest of the /home/me/python is not in the PYTHONPATH
I'm running this python script on a remote machine and asks for pyfits, which is technically installed on the machine, but python doesn't find it.
I've already tried adding the supposed directory it's installed in to my paths (I have access to the folder too) by the sys.path.append('folder') method. But it still doesn't find it.
Here's some thought process to illustrate:
The user who installed the modules has all the source at "/otheruser/code/pyfits" so I've tried adding that folder or any folder with pyfits and an init file (that I have access to) in it, without success.
So my main questions are:
Should I be looking elsewhere for the module?
Should I install the modules again as --myuser? or should I mess with the site-packages? If so does one add the module there?
According to the PyFITS documentation, it looks like the actual modules are installed in lib/python or possibly lib/python<version>/site-packages (depending on the flags they used for the install) under the top-level PyFITS install directory. So, normally you'd want to do something like this:
sys.path.append(r'/otheruser/code/pyfits/lib/python')
# Might be sys.path.append(r'/otheruser/code/pyfits/lib/python2.7/site-packages')
# or something similar
import numpy
import pyfits
If you are able to read the contents of the directory you appended and its subdirectories, you should be good to go.
However, as I mentioned in my comment on your original question, you might want to edit your question to include the actual code you're using to do this, since any problem might be easier to guess that way.
Sources of difficulty might be:
Adding a directory to your sys.path that doesn't point to the actual modules.
Absence of an __init__.py file in the module directory.
Not having permissions to view folders' contents or read the enclosed files.
Problems with Python's string handling screwing your path up somehow (like not using raw mode and not adequately escaping backslashes when using windows paths containing backslashes.)
Something overwriting your sys.path before your import occurs.
A typo.
I recently upgraded my Python 2.7.1 installation to 2.7.2 using the .msi installer and the process seemed to go OK. However afterwards I happened to be looking at what the default values were for sys.path and noticed the entry for 'C:\Windows\system32\python27.zip'. Wondering exactly what was in it, I decided to try opening it with a Zip utility to check out its contents — however I soon discovered that the file wasn't there (although there is a python27.dll).
Anyhow I'm now wondering if something is wrong. Several existing Python programs I frequently use all seem to work without problems, so I'm not sure whether the installation is messed up or not (or how to fix it, if it is).
Update
I'm aware of and have read PEP273, so know about .zip file modules. That's not what I'm asking about. What I want to know is the fact that there is no python27.zip installed on my system even though it's referred to in my sys.path a problem? I've never encountered problems importing standard Python libraries, which is what I would expect not having one would affect.
From PEP 273 -- Import Modules from Zip Archives:
Just as sys.path currently has default directory names, a default zip
archive name is added too. Otherwise there is no way to import all
Python library files from an archive.
...
The problem is what the name should be. The name should be linked
with the Python version, so the Python executable can correctly find
its corresponding libraries even when there are multiple Python
versions on the same machine.
We add one name to sys.path. On Unix, the directory is sys.prefix +
"/lib", and the file name is "python%s%s.zip" % (sys.version[0],
sys.version[2]). So for Python 2.2 and prefix /usr/local, the path
/usr/local/lib/python2.2/ is already on sys.path, and
/usr/local/lib/python22.zip would be added. On Windows, the file is
the full path to python22.dll, with "dll" replaced by "zip". The zip
archive name is always inserted as the second item in sys.path. The
first is the directory of the main.py (thanks Tim).
I have installed python oauth on my python2.4 platform, however making the python twitter package work requires some tweaks in oauth.. I am quite new to python but I assume I cannot alter the egg.. how do I install a non-egg version and how do I remove the egg safely ?
Python eggs (like java jar files) use the zip format. So to answer your question on how to make your tweaks:
Find the file location
Navigate to location, make a backup copy
If the file is stored as oauth.egg, unzip it
Start modifying!
Find the egg location
Open up a python interpreter and run the following:
>>> import oauth
>>> oauth.__file__
'/usr/lib/python2.6/dist-packages/oauth/__init__.pyc'
Your path will differ, but that will tell you where to look. Often the source code will be unpacked and available in the same directory as a .py file, in this case oauth.py.
(By the way the __file__ attribute is available on all modules unless they represent linked C libraries, but that should not be your case with oauth.)
I'll skip the file navigation, backup, and unzip details, as those will depend on your system.
Removing a Python Egg Safely
I'm afraid my knowledge is lacking here. Removing the egg file is easy, but I'm not at all sure how to check for dependencies from other packages, other than running $ ack python.module.to.remove across your python library. But some basic facts that may help
Directories that include __init__.py in them are treated as part of the python path. See Modules and Packages
Python eggs will add a .pth file containing additional places to add to the path.
>>> import sys; sys.path will show every directory that Python searches for modules/packages.
The PYTHONPATH environment variable can be configured to add paths you choose to the python search path
PS If you are new to Python, I highly recommend finding out more about IPython. It makes the Python intepreter much nicer to deal with.
Good luck and welcome to Python!
What's the shortest way to import a module from a distant, relative directory?
We've been using this code which isn't too bad except your current working directory has to be the same as the directory as this code's or the relative path breaks, which can be error prone and confusing to users.
import sys
sys.path.append('../../../Path/To/Shared/Code')
This code (I think) fixes that problem but is a lot more to type.
import os,sys
sys.path.append(os.path.realpath(os.path.join(os.path.dirname(__file__), '../../../Path/To/Shared/Code')))
Is there a shorter way to append the absolute path? The brevity matters because this is going to have to be typed/appear in a lot of our files. (We can't factor it out because then it would be in the shared code and we couldn't get to it. Chicken & egg, bootstrapping, etc.)
Plus it bothers me that we keep blindly appending to sys.path but that would be even more code. I sure wish something in the standard library could help with this.
This will typically appear in script files which are run from the command line. We're running Python 2.6.2.
Edit:
The reason we're using relative paths is that we typically have multiple, independent copies of the codebase on our computers. It's important that each copy of the codebase use its own copy of the shared code. So any solution which supports only a single code base (e.g., 'Put it in site-packages.') won't work for us.
Any suggestions? Thank you!
Since you don't want to install it in site-packages, you should use buildout or virtualenv to create isolated development environments. That solves the problem, and means you don't have to fiddle with sys.path anymore (in fact, because Buildout does exactly that for you).
You've explained in a comment why you don't want to install "a single site-packages directory", but what about putting in site-packages a single, tiny module, say jebootstrap.py:
import os, sys
def relative_dir(apath):
return os.path.realpath(
os.path.join(os.path.dirname(apath),
'../../../Path/To/Shared/Code'))
def addpack(apath):
relative = relative_dir(apath)
if relative not in sys.path:
sys.path.append(relative)
Now everywhere in your code you can just have
import jebootstrap
jebootsrap.addpack(__file__)
and all the rest of your shared codebase can remain independent per-installation.
Any reason you wouldn't want to make your own shared-code dir under site-packages? Then you could just import import shared.code.module...
You have several ways to handle imports, all documented in the Python language manual.
See http://docs.python.org/library/site.html and http://docs.python.org/reference/simple_stmts.html#the-import-statement
Put it in site-packages and have multiple Python installations. You select the installation using the ordinary PATH environment variable.
Put the directory in your PYTHONPATH environment variable. This is a per-individual-person setting, so you can manage to have multiple versions of the codebase this way.
Put the directory in .pth files in your site-packages. You select the installation using the ordinary PATH environment variable.