I am writing a project which uses data from a database. Now I wanted to package it and have made a setup.py for it using help from here. Now when I run setup.py from terminal it installs the package. The database is created using a create_db.py script and it is created in the created in the same directory as the script. So obviously it doesn't work after I install the package since its not accessible from there.
I am looking for a way to create the database from the script at the time the package is being installed. Is it possible?
Add this to your main script?
import time
time.sleep(5)
execfile("./create_db.py")
Related
I have a python project which takes in a bunch of pdf files from a directory, scrapes data from them, and then does some matching of that scraped data with some data in a CSV file.
The whole work has 2-3 python scripts, used as modules, and also uses dependencies of pdftotext, pandas, NumPy, etc.
Now I can pip freeze my Conda env and it can give me a requirements.txt file with all packages to install.
However, I want this main python script (which calls other modules and runs the whole project) to be run by a less technical person who doesn't work on pandas and other such python stuff.
So is there a way I can make this whole project as an executable file that encapsulates all dependencies, packages, scripts, and just running that executable in the terminal should run the whole project without having the other person install all dependencies themselves using requirements.txt file.
I can't use docker unfortunately as that is not permitted right now for my work.
I was thinking buck build if that works?
https://buck.build/
or if there is an easy way?
Thanks!
One approach is to package the Python application directory as a .zip file and execute that. Zip files that have a __main__.py entry point can be run this way.
This can be done easily in version 2.6 and up. Additional “zipapp” support was added in 3.6.
The main challenge has to do with compatibility for non-pure-Python libraries. What you zip up needs to be compatible with the machine where it will be run.
pip install cx_freeze
cxfreeze main.py --target-name your_exe_name
Replace your_exe_name. It will generate a build folder with your .exe in it.
Is there a simply way to download/save a Python script hosted on Github and then import a function within that script? I could manually save it and then import the function I need (of course), but I'd like to find a more self-contained solution. Ideally, I'd like to be able to send it to someone and have them run my script without having to manually download anything.
Any help?
You could use setuptools and write custom setup script for your package. Wrap your script into python package.
So users will be able install it via pip with just a single line:
pip install git+https://github.com/yournickname/reponame.git
I am currently writing a command line application in Python, which needs to be made available to end users in such a way that it is very easy to download and run. For those on Windows, who may not have Python (2.7) installed, I intend to use PyInstaller to generate a self-contained Windows executable. Users will then be able to simply download "myapp.exe" and run myapp.exe [ARGUMENTS].
I would also like to provide a (smaller) download for users (on various platforms) who already have Python installed. One option is to put all of my code into a single .py file, "myapp.py" (beginning with #! /usr/bin/env python), and make this available. This could be downloaded, then run using myapp.py [ARGUMENTS] or python myapp.py [ARGUMENTS]. However, restricting my application to a single .py file has several downsides, including limiting my ability to organize the code and making it difficult to use third-party dependencies.
Instead I would like to distribute the contents of several files of my own code, plus some (pure Python) dependencies. Are there any tools which can package all of this into a single file, which can easily be downloaded and run using an existing Python installation?
Edit: Note that I need these applications to be easy for end users to run. They are not likely to have pip installed, nor anything else which is outside the Python core. Using PyInstaller, I can generate a file which these users can download from the web and run with one command (or, if there are no arguments, simply by double-clicking). Is there a way to achieve this ease-of-use without using PyInstaller (i.e. without redundantly bundling the Python runtime)?
I don't like the single file idea because it becomes a maintenance burden. I would explore an approach like the one below.
I've become a big fan of Python's virtual environments because it allows you to silo your application dependencies from the OS's installation. Imagine a scenario where the application you are currently looking to distribute uses a Python package requests v1.0. Some time later you create another application you want to distribute that uses requests v2.3. You may end up with version conflicts on a system where you want to install both applications side-by-side. Virtual environments solve this problem as each application would have its own package location.
Creating a virtual environment is easy. Once you have virtualenv installed, it's simply a matter of running, for example, virtualenv /opt/application/env. Now you have an isolated python environment for your application. Additionally, virtual environments are very easy to clean up, simply remove the env directory and you're done.
You'll need a setup.py file to install your application into the environment. Say your application uses requests v2.3.0, your custom code is in a package called acme, and your script is called phone_home. Your directory structure looks like this:
acme/
__init__.py
models.py
actions.py
scripts/
phone_home
setup.py
The setup.py would look something like this:
from distutils.core import setup
install_requires = [
'requests==2.3.0',
]
setup(name='phone_home',
version='0.0.1',
description='Sample application to phone home',
author='John Doe',
author_email='john#doe.com',
packages=['acme'],
scripts=['scripts/phone_home'],
url='http://acme.com/phone_home',
install_requires=install_requires,
)
You can now make a tarball out of your project and host it however you wish (your own web server, S3, etc.):
tar cvzf phone_home-0.0.1.tar.gz .
Finally, you can use pip to install your package into the virtual environment you created:
/opt/application/env/bin/pip install http://acme.com/phone_home-0.0.1.tar.gz
You can then run phone_home with:
/opt/application/env/bin/phone_home
Or create a symlink in /usr/local/bin to simply call the script using phone_home:
ln -s /opt/application/env/bin/phone_home /usr/local/bin/phone_home
All of the steps above can be put in a shell script, which would make the process a single-command install.
And with slight modification this approach works really well for development environments; i.e. using pip to install / reference your development directory: pip install -e . where . refers to the current directory and you should be in your project directory alongside setup.py.
Hope this helps!
You could use pip as suggested in the comments. You need to create a MANIFEST.in and setup.py in your project to make it installable. You can also add modules as prerequisites. More info can be found in this question (not specific to Django):
How do I package a python application to make it pip-installable?
This will make your module available in Python. You can then have users run a file that runs your module, by either python path/run.py, ./path/run.py (with +x permission) or python -c "some code here" (e.g. for an alias).
You can even have users install from a git public reporitory, like this
pip install git+https://bitbucket.org/yourname/projectname.git
...in which case they also need git.
I have created my program using virtual env. It is working in my project folder fine. Now i need to take this program and release it to the production environment that is supposed to be accessible by everybody.So this program should be runnable as is or it might be incorporated into other programs as a step. How am i supposed to deploy it? Zip the whole project folder? Is it possible to do without requiring clients to copy it and then unzip and run? Or the only way is to create a commonly accessible script that automates unzipping of the thing and configuring virtual env and then running it or there is a smarter way?
More complicated scenario is when it supposed to be used as library. How to deploy it so others could specify it as their dependency and pick it up? Seems like the only way is to create your own PyPi-like local repository - is that correct?
Thanks!
So here is what i have found:
If we have a project A as API:
create a folder where you will store the wheels (~/wheelhouse)
using pip config specify this folder as one to find links in http://www.pip-installer.org/en/latest/configuration.html
i have:
[global]
[install]
no-index = yes
find-links = /home/users/me/wheelhouse
Make sure the wheel package is installed.
In your project create setup.py file that will allow for the wheel creation and execute
python setup.py bdist_wheel
copy the generated wheel to the wheelhouse so it has:
~/wheelhouse/projectA-0.1-py33-none-any.whl
Now we want to create a project that uses that projectA API - project B
we are creating a separate folder for this project and then create a virtual environment for it.
mkdir projectB; cd projectB
virtualenv projectB_env
source projectB_env/bin/activate
pip install projectA
Now if you run python console in this folder you will be able to import the classes from the projectA! One problem solved!
Now you have finished the development of projectB and you need to run it.
For that purpose I'd recommend to use Pex (twitter.common.python) library. Pex now supports (v0.5.1) wheels lookup as dependencies. I'm feeding it the content of requirements.txt file to resolve dependencies. So as the result you will get the executable lightweight archived virtualenv that will have everything necessary for the project to run.
This should get you started:
http://docs.python.org/2/distutils/
http://guide.python-distribute.org/
http://pythonhosted.org/setuptools/
I am pretty new to python and pyramid. I am working on pyramid application that I run with the following command:
pserve development.ini
When I make some changes and restart the server (kill it and run it again) it keeps the old versions of the files in cache.
I have noticed that I can clean the cache by re-installing the application with
python setup.py install
but I am sure that there is a nicer way for this?
I have noticed that the cache files are kept in the build folder:
build/lib.linux-x86_64-2.7/*
Instead of using python setup.py install, use python setup.py develop. This will link your application's directory into the site-packages without creating a separate "installed" source tree.