Is there a way to change the directory where .pyc file are created by the Python interpreter? I saw two PEPs about that subject (0304 and 3147), but none seems to be implemented in the default interpreter (I'm working with Python 3).
Did I miss something ?
Yes, starting from Python 3.8 you can control this behavior. The original discussion starts from pep 304 in 2003.
While this original PEP was withdrawn, a variant of this feature was
eventually implemented for Python 3.8 in https://bugs.python.org/issue33499
In the result, you can control using PYTHONPYCACHEPREFIX=path, -X pycache_prefix=path and sys.pycache_prefix.
This might be useful for some:
Miscellaneous options, especially -B option:
If given, Python won’t try to write .pyc files on the import of source
modules. See also PYTHONDONTWRITEBYTECODE.
There's no way to change where the .pyc files go. Python 3.2 implements the __pycache__ scheme whereby all the .pyc files go into a directory named __pycache__. Python 3.2 alpha 1 is available now if you really need to keep your directories clean.
Until 3.2 is released, configure as many tools as you can to ignore the .pyc files.
Python files are NOT always stored in the same directory and will get directed to pycache if you use pycache. Any answers on being able to have custom locations and names, because importing the module is rather temperamental and doesn't always work. You will be able see what I mean at github.com/LolexInc/Lolex-Tools/tree/Beta and view JTToolsInstaller.py. There is a lot of mess in it.
Related
For me, it's located at C:\Python33\libs.
For reference - this is not the same folder as C:\Python33\Lib - note the capitalization and lack of an 's'.
On one computer I was working on, I simply dropped a .py file into the libs folder and could import and use it like a library / module (sorry, I don't really know terminology very well), regardless of where the project I was working on is.
However, in trying to duplicate this on another machine, this doesn't work. Attempting to import simply gives a "no module named X" error.
So, clearly I'm misunderstanding the purpose of the libs folder, and how it differs from the Lib folder.
So, what exactly is the difference?
If you compare libs/ vs. Lib/ you'll notice that the latter is full of *.py files and the former has *.lib files. Further investigation with a text editor will show that *.py files are human-readable (I hope) and the *.lib files are not.
And that's really the difference. If you want to know more, the .lib files are static-link libraries, used for building .dlls, C extensions, and all that good stuff. Head on down the rabbit hole if that interests you.
On to the meat of your question: are you supposed to be able to drop modules in there and be able to import them? Not really. That is a side effect of that folder being included in your path. From the Modules docs:
When a module named spam is imported, the interpreter first searches
for a built-in module with that name. If not found, it then searches
for a file named spam.py in a list of directories given by the
variable sys.path. sys.path is initialized from these locations:
the directory containing the input script (or the current directory).
PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
the installation-dependent default.
Various installation methods will modify %PATH% or %PYTHONPATH% so I can't tell you exactly where to look; on my windows box, the python installer modified %PATH% for me, so you should probably look there first. Notably, my path does not include Python33/libs/ so I would not expect it to be there by default.
Just looking on mine (Windows 7) /libs appears to be the native code libraries (*.lib) vs the straight python libraries in /Lib. The readme also mentions a configuration flag:
--with-libs='libs': Add 'libs' to the LIBS that the python interpreter
is linked against.
Which may or may not be set on different installs/platforms.
This isn't really a answer; hopefully someone with a firmer knowledge of it will explain further - was just a bit too much info to squeeze into a comment.
I recently upgraded my Python 2.7.1 installation to 2.7.2 using the .msi installer and the process seemed to go OK. However afterwards I happened to be looking at what the default values were for sys.path and noticed the entry for 'C:\Windows\system32\python27.zip'. Wondering exactly what was in it, I decided to try opening it with a Zip utility to check out its contents — however I soon discovered that the file wasn't there (although there is a python27.dll).
Anyhow I'm now wondering if something is wrong. Several existing Python programs I frequently use all seem to work without problems, so I'm not sure whether the installation is messed up or not (or how to fix it, if it is).
Update
I'm aware of and have read PEP273, so know about .zip file modules. That's not what I'm asking about. What I want to know is the fact that there is no python27.zip installed on my system even though it's referred to in my sys.path a problem? I've never encountered problems importing standard Python libraries, which is what I would expect not having one would affect.
From PEP 273 -- Import Modules from Zip Archives:
Just as sys.path currently has default directory names, a default zip
archive name is added too. Otherwise there is no way to import all
Python library files from an archive.
...
The problem is what the name should be. The name should be linked
with the Python version, so the Python executable can correctly find
its corresponding libraries even when there are multiple Python
versions on the same machine.
We add one name to sys.path. On Unix, the directory is sys.prefix +
"/lib", and the file name is "python%s%s.zip" % (sys.version[0],
sys.version[2]). So for Python 2.2 and prefix /usr/local, the path
/usr/local/lib/python2.2/ is already on sys.path, and
/usr/local/lib/python22.zip would be added. On Windows, the file is
the full path to python22.dll, with "dll" replaced by "zip". The zip
archive name is always inserted as the second item in sys.path. The
first is the directory of the main.py (thanks Tim).
I have installed python oauth on my python2.4 platform, however making the python twitter package work requires some tweaks in oauth.. I am quite new to python but I assume I cannot alter the egg.. how do I install a non-egg version and how do I remove the egg safely ?
Python eggs (like java jar files) use the zip format. So to answer your question on how to make your tweaks:
Find the file location
Navigate to location, make a backup copy
If the file is stored as oauth.egg, unzip it
Start modifying!
Find the egg location
Open up a python interpreter and run the following:
>>> import oauth
>>> oauth.__file__
'/usr/lib/python2.6/dist-packages/oauth/__init__.pyc'
Your path will differ, but that will tell you where to look. Often the source code will be unpacked and available in the same directory as a .py file, in this case oauth.py.
(By the way the __file__ attribute is available on all modules unless they represent linked C libraries, but that should not be your case with oauth.)
I'll skip the file navigation, backup, and unzip details, as those will depend on your system.
Removing a Python Egg Safely
I'm afraid my knowledge is lacking here. Removing the egg file is easy, but I'm not at all sure how to check for dependencies from other packages, other than running $ ack python.module.to.remove across your python library. But some basic facts that may help
Directories that include __init__.py in them are treated as part of the python path. See Modules and Packages
Python eggs will add a .pth file containing additional places to add to the path.
>>> import sys; sys.path will show every directory that Python searches for modules/packages.
The PYTHONPATH environment variable can be configured to add paths you choose to the python search path
PS If you are new to Python, I highly recommend finding out more about IPython. It makes the Python intepreter much nicer to deal with.
Good luck and welcome to Python!
Currently, I'm deploying a full python distribution (the original python 2.7 msi) with my app. Which is an embedded web server made with delphi.
Reading this, I wonder if is possible to embed the necessary python files with my app, to decrease load files and avoid conflict with several python versions.
I have previous experience with python for delphi so I only need to know if only shipping the python dll + zip with the distro + own scripts will work (and if exist any caveats I must know or a sample where I can look)
zipimport should work just fine for you -- I'm not familiar with Python for Delphi, but I doubt it disables that functionality (an embedding application can do that, but it's an unusual choice). Just remember that what you can zip up and import directly are the Python-coded modules (or just their corresponding .pyc or .pyo byte codes) -- DLLs (even if renamed as .pyds;-) need to be on disk to be loaded (so if you have a zipfile with them it will need to be unzipped at the start of the app, e.g. into a temporary directory).
Moreover, you don't even need to zip up all modules, just those you actually need (by transitive closure) -- and you can easily find out exactly which modules those are, with the modulefinder module of the standard Python library. The example on the documentation page I just pointed to should clarify things. Happy zipping!
Yes it is possible.
I'm actually writing automatisation script in Python with the Zipimport library. I actually included every .py files in my zip as well as configuration or xml files needed by those script.
Then, I call a .command file targeting a __main__.py class that redirect towards the desired script according to my sys.argv parameters which is really useful!
Is there a way to put together Python files, akin to JAR in Java? I need a way of packaging set of Python classes and functions, but unlike a standard module, I'd like it to be in one file.
After looking for a solution to the same problem, I ended up writing a simple tool which combines multiple .py files into one: PyBreeder
It will only work with pure-Python modules and may require some trial-and-error to get the order of modules right, but it is quite handy whenever you want to deploy a script with some dependencies as a single .py. Comments/patches/critique are very welcome!
Take a look at Python Eggs: http://peak.telecommunity.com/DevCenter/PythonEggs
Or, you can use regular zips: http://docs.python.org/library/zipimport.html
The simplest approach is to just use zip. A jar file in Java is a zipfile containing some metadata such as a manifest; but you don't necessarily need the metatada -- Python can import from inside a zipfile as long as you place that zipfile on sys.path, just as you would do for any directory. In the zipfile you can have the sources (.py files), but then Python will have to compile them on the fly each time a process first imports them; or you can have the bytecode files (.pyc or .pyo) but then you're limited to a specific release of Python and to either absence (for .pyc) or presence (for .pyo) of flag -O (or -OO).
As other answers indicated, there are formats such as .egg that enrich the zipfile with metatada in Python as well, like Java .jar, but whether in a particular use case that gives you extra value wrt a plain zipfile is a decision for you to make
You can create zip files containing Python code and import from zip files using zipimport. A system such as PyInstaller (cross-platform) or py2exe (Windows) will do all this for you.
Read this PEP for informations.
Also, Import modules from Zip.
stickytape (mentioned before) worked for me.
stickytape importingSomeModules.py > resultWithEmbeddedImports.py
Sometimes you just want to redistribute just a single .py file, so neither PyInstaller, zipimport nor Python Eggs cut the deal then.
PyBreeder is (currently) throwing exceptions.