I am writing a little python app. I want to be able to easily deploy the app. I know that python 2.6 will allow one to execute an egg directly if there is a main module at the egg's root. I actually have that working.
The one kink is that when I try to use the argparse library, I cannot include the library in the egg without installing it into my source directory (or symlinking in the argparse.py into my source dir) since the argparse module is in the top-level package.
If I install it into a subdirectory called "argparse", I have to import it like "from argparse import argparse" instead of the normal "import argparse".
I would like to be able to specify a site-packages type directory in the egg where I could just install the third party modules/packages. Is there any way to do this with setuptools (or some other egg builder)?
Thanks!
I believe you can create a subdirectory called toplevel and in your entry point do
import sys
sys.path.insert(0, './toplevel')
Untested, though.
Related
I see there are some built in packages that I can import from any script like:
from datetime import date
today = date.today()
print today
How can I create a simple package and add it to the system library so that I can import it like datetime in the above example?
You're trying to make a module.
Start by installing the setuptools package; on either Windows or Linux you should be able to type pip install setuptools at a terminal to get that installed. You should now be able to write import setuptools at a python prompt without getting an error.
Once that's working, set up a directory structure containing a setup.py and a folder for your project's code to go in. The directory must contain a file called __init__.py, which allows you to import the directory as though it's a file.
some_folder/
| setup.py
| my_project/__init__.py
In setup.py, drop the following content:
# setup.py
from setuptools import setup
setup(name="My Awesome Project",
version="0.0",
packages=["my_project"])
In my_project/__init__.py, drop some stuff that you'd like to be able to import. Let's say...
# my_project/__init__.py
greeting = "Hello world!"
Now, in order to install the project at a system-wide level, run python setup.py install. Note that you'll need to run this as root if you're on Linux, since you're making changes to the system-wide python libraries.
After this, you should be able to run python from any directory you like and type:
>>> from my_project import greeting
>>> print greeting
Hello world!
>>>
Note that this is enough to tell you how to make a module, but there's one hell of a lot of stuff that setuptools can do for you. Take a look at https://pythonhosted.org/setuptools/setuptools.html for more info on building stuff, and https://docs.python.org/2/tutorial/modules.html for more info on how modules actually work. If you'd like to look at a package that (I hope) is reasonably simple, then I made my LazyLog module a couple of weeks ago on a train, and you're welcome to use it for reference.
The quick way, if you're just making something for your own use and not worrying about packaging, is to put the module (which could be as simple as a single file) in your system's site-packages directory. (On Debian-based systems, you probably want to use dist-packages instead).
To find out where your site-packages/dist-packages directory is, start Python and:
>>> from sys import path
>>> path
['', '/usr/lib/python3.4/site-packages/pip-7.1.2-py3.4.egg', '/usr/lib/python34.zip', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-cygwin', '/usr/lib/python3.4/lib-dynload', '/usr/lib/python3.4/site-packages']
Note the last item in that example: /usr/lib/python3.4/site-packages. That's the sort of thing you're looking for. So in this example, if I save the following to /usr/lib/python3.4/site-packages/foo.py:
def bar():
print('Hello world!')
Then from anywhere on my system:
>>> from foo import bar
>>> bar()
Hello world!
If you really need a package, you can do it with boost, which allows interacts with C++. You may implement algorithms using C++ and compile it as a Python lib. However it's poorly documented. And as doc described, C API should be a basic option. Boost is built on C API any way.
Sample: I made it several years ago on a class. You can do: import tfidf.
If you simply want the module for personal use, just drop it in a folder and add that folder to the PYTHONPATH environment variable.
For example, create a folder in your home directory called ~/python-packages, then add the following line to your .bashrc (assuming you are using bash):
export PYTHONPATH=$HOME/python-packages`
Then, simply drop any modules/packages you want to make available in ~/python-packages.
Add you python script or package's path to sys.path or just move them to one of the location in sys.path.
BUT i do not suggest to do this...
I have created a simple python module with SWIG (from C++). It is in the form of a .pyd file and a .py file.
I want to be able to give this module to my colleagues so they can install and use it.
I am aware that python modules are often installed using:
python setup.py install
Perhaps this is the functionality I want, so that my colleagues can run this module from anywhere without worrying about PATH etc.
I also would like to be able to send them an updated module from time to time, which will overwrite the older version when they install it.
How do I go about this?
As far as I can understand this question, you want a setup.py file to install your module
go through the steps given below :
create a directory on Desktop (name it "source" for example)
put your .py and .pyc files in it.
go to Desktop and create setup.py file and enter the following code
as
from distutils.core import setup
setup(name='Some_name',
version='1.0',
author='Author_name',
author_email='abc#xyz.com',
packages=['source']
)
4. To use it in any program.
import module_name from source
hope this helps !
I have a Flask project which I've put the flask module (version 0.9) directly beside my app.py file. I've done this so that I can bundle everything into a version control repository that won't require anyone else using it to install additional Python modules.
I want to use flask-login so I've tried to manually install it by downloading the latest version and putting the flask_login.py file in my "local" flask/ext/ directory. However, while I can import flask and import flask.ext, I am unable to import flask.ext.login with Python throwing ImportError: No module named flask.ext.login. import flask.ext.flask_login throws an import error as well.
Is there something I have to do differently if Flask and it's extensions are local to the app.py?
The solution is to put the flask_login.py file in the same directory as my app.py file. No modification of the flask/ext/__init__.py file is necessary.
The flask.ext module is intended only as a an extension importer, not a repository for installed extensions. Based on the import path, I thought the flask/ext folder was where extensions were to be copied. This was incorrect. Extensions simply need to be somewhere on the python path.
Python doesn't throw the exception in your case, this Flask module is responsible for modifying the standard import hook in the ext folder:
https://github.com/mitsuhiko/flask/blob/master/flask/exthook.py
So, I don't think putting flask_login.py into flask/ext is the right way to use extensions in Flask. The documentation recommends to use pip install flask-login or python setup.py install. After that you can do:
import flask_login
If you still want to do it manually, remove the setup() call from ext/__ init__.py.
It probably has a good reason why the Flask guys did it this way though :-)
Right, but the point was to manually install everything so it could be self-contained in a version control repository that won't require additional installs. I've looked at the setup.py file but can't determine how it achieves the installation. – Soviut Oct 25 at 10:06
Why not use virtualenv to achieve this? You can push your env to source control as well, although that's not normal usage.
See: http://www.virtualenv.org/en/latest/
I have the following folder structure
lib
my_module
I have moved all the libraries I need into the lib/ folder.
In my module/__init__.py, I think I will do:
import sys
sys.path.append('../lib/')
import my_dependency
Then when I need to use this dependency, I will refer to it as
my_module.my_dependency
Is this a bad usage of Python import?
NOTE: the dependencies consists of some third-party libraries not available via pip/easy_install and some C++ stuff that I wrote.
sys.path.append('../lib/') assumes that the current working directory is the directory of your script, which may or may not be the case.
A version that doesn't depend on the working directory is:
import sys, os
sys.path.append(os.path.join(os.path.split(os.path.split(os.path.abspath(sys.argv[0]))[0])[0], "lib"))
import my_dependency
The above in plain language takes the full path to the script, chops off the last two components (script directory and script filename) and appends lib.
If the libraries you use are third-party modules—especially popular ones—namespacing them like that is going to get inconsistent pretty fast. You will end up with code that sometimes is referenced as bar and sometimes as foo.bar. Maintaining such a codebase will not be worth whatever gains you expect to get from prefixing them like that.
If you keep third-party code in your own repository, consider replacing it with a requirements.txt file that can be fed to utilities like easy_install and pip. Both these tools support a --user switch that installs to your home directory (not touching system stuff).
pip install -r requirements.txt --user
I just wrote a python script to fix some database issues and it uses the psycopg2 module. For some reason the person that needs to run it (on the server) is claiming that they can't install psycopg2 on their server machine... is there a way that I can package that module within my script such that they don't have to have psycopg2 installed? Something similar to adding a lib folder to the classpath in Java?
Thx in advance,
Andre
Make a directory, put your script into it.
cd into that directory, then run:
$ easy_install -Z --prefix . psycopg2
That should result in a copy of the python module in the same directory.
Zip it all up and send the whole thing off.
As to be unable to install psycopg2, I sure it relates to security policies about who can have root access on database servers. If you can't find someone with the necessary permissions, there are two methods to skin this particular cat:
1) You could build an exe using py2exe, that way you know that all of the dependencies for your scripts are there and available. Also this will allow to you escape dealing with any python incompatibilities. You made no mention of your python version (2.5, 2.6, 2.7, 3.1?) nor the version on the server.
2) Or you could place the psycopg2 module into its own directory (c:\mypymods) and add the path to sys.path before importing.
import sys
sys.path = sys.path.append("c:/mypymods")
import psycopg2
....