I have made a python interactive script project containing a few directories with project files and a main python script.
The script does the work of batch processing scientific images for biological systematics.
I wrote os agnostic code, but I was thinking about trying to package/freeze the script as a cli utility that a (more or less)lay person could download and use.
I have been reading about packaging and freezing techniques in python and the more I read the more I feel I'm confused. (I'm linux user)
Am I conceiving of this script as a utility correctly? Is is worth it/possible to pass command line args to an .exe, and should I package/freeze the files for this kind of interactive cli script?
I don't have a lot of experience with windows. I'm looking for advice/pointer where to look next/search.
You are on the correct path, but you need to understand a little more about python packaging to know how to proceed.
You are right that you need to package your code with setup tools. This will make it cake to install the package for others and will let python decide where to properly put things. -including script utilities/non-code files-
Setuptools has special support built in JUST FOR SCRIPTS! that will let you pass arguments, ect..
However, in packaging the code, you must understand that what you are really doing is making it a library that can be installed with pip. Therefore, that python script utility that you wrote will be treated as a library by setuptools. Setuptools will have you make a separate script file to import your new library and call the script main() function.
I know that you were already using this manual, and that's fantastic. Everything you need to first set up and understand a python package is in there. RTFM up a storm.
As I understand it, you can then look more into freezing the package (and register it with Pypi) for other systems once you have a handle on packaging.
note: use scripts keyword in setup tools to specify your script
Related
I'm new to using python modules.
I'm currently working on a python 2.7 script that will be deployed to many remote computers (which have python 2.7 on them). The problem is that the script needs to use a module, which I am not allowed to install on those computers.
I'm wondering if it is possible to include the module files in the same package as my script (possibly have them compiled first), and then have the script import the library from that local folder, thus achieving a "portable" script.
If that is possible, how would I go about doing that?
Specifics: I'm running 2.7.11 on Windows needing to use Paramiko.
I'm asking this question because the similar questions that I can find either do not answer mine, or expect me to be familiar with core python structures with which I am not. I also DON'T want to include the entirety of python and then install the module onto that, something I see is often called Portable Python. I just want to send my script and the module and nothing more.
Many thanks!
To install modules in a specific directory, you can try pip install module --target=.
By default python search for those modules in same directory as the script first, then, if not available, it will search for python install lib files.
I created a command-line python tool for Unix systems.
I want my script to be executed from anywhere just like any Unix command. One solution is to make my script executable and move it to /usr/bin.
But the script works with external files, and I guess moving all these files with my script in /usr/bin is a bad habit, as it will be hard to delete them one by one in the future.
Where should I put the directory of my application ? How to add the main script to the PATH, to execute it from anywhere ?
I would like then to create a package that could move my files in the right place, and delete them in the case the user wants to uninstall my application.
I don't know how to do this.
Thank you.
There are many way of distributing a Linux application.
It will depend on the distribution you are using, since they do not all use the same package manager. For example, you would create a .deb package for Debian, Ubuntu and there derivatives, an Arch package for Archlinux, etc...
You could then share the package with anyone to let them install your tool.
However, since your tool is in written in Python, you can also make a python package. You could then upload it to the Python Package Index to let anyone install it using python's pip package manager.
To create a python package, you will need to create a file called setup.py, and, from it, to call the setup method from the setuptool packages.
You will probably want to read the python documentation about writing such a script: https://setuptools.readthedocs.io/en/latest/setuptools.html
You may especially be interested by this sections:
Including Data Files
Automatic Script Creation
If you do things correctly, setuptools will take care of installing your script and its files somewhere in the PATH so it can be executed from the command line.
You should use setuptools to distribute your application, creating a setup.py file to configure its setup and installation.
Binaries can be delivered using the console_scripts option, while data files can be delivered using either package_data.
This is probably a question that has a very easy and straightforward answer, however, despite having a few years programming experience, for some reason I still don't quite get the exact concepts of what it means to "build" and then to "install". I know how to use them and have used them a lot, but have no idea about the exact processes which happen in the background...
I have looked across the web, wikipedia, etc... but there is no one simple answer to it, neither can I find one here.
A good example, which I tried to understand, is adding new modules to python:
http://docs.python.org/2/install/index.html#how-installation-works
It says that "the build command is responsible for putting the files to install into a build directory"
And then for the install command: "After the build command runs (whether you run it explicitly, or the install command does it for you), the work of the install command is relatively simple: all it has to do is copy everything under build/lib (or build/lib.plat) to your chosen installation directory."
So essentially what this is saying is:
1. Copy everything to the build directory and then...
2. Copy everything to the installation directory
There must be a process missing somewhere in the explanation...complilation?
Would appreciate some straightforward not too techy answer but in as much detail as possible :)
Hopefully I am not the only one who doesn't know the detailed answer to this...
Thanks!
Aivoric
Building means compiling the source code to binary in a sandbox location where it won't affect your system if something goes wrong, like a build subdirectory inside the source code directory.
Install means copying the built binaries from the build subdirectory to a place in your system path, where they become easily accessible. This is rarely done by a straight copy command, and it's often done by some package manager that can track the files created and easily uninstall them later.
Usually, a build command does all the compiling and linking needed, but Python is an interpreted language, so if there are only pure Python files in the library, there's no compiling step in the build. Indeed, everything is copied to a build directory, and then copied again to a final location. Only if the library depends on code written in other languages that needs to be compiled you'll have a compiling step.
You want a new chair for your living-room and you want to make it yourself. You browse through a catalog and order a pile of parts. When they arrives at your door, you can't immediately use them. You have to build the chair at your workshop. After a bit of elbow-grease, you can sit down in it. Afterwards, you install the chair in your living-room, in a convenient place to sit down.
The chair is a program you want to use. It arrives at your house as source code. You build it by compiling it into a runnable program. You install it by making it easier to use.
The build and install commands you are refering to come from setup.py file right?
Setup.py (http://docs.python.org/2/distutils/setupscript.html)
This file is created by 3rd party applications / extensions of Python. They are not part of:
Python source code (bunch of c files, etc)
Python libraries that come bundled with Python
When a developer makes a library for python that he wants to share to the world he creates a setup.py file so the library can be installed on any computer that has python. Maybe this is the MISSING STEP
Setup.py sdist
This creates a python module (the tar.gz files). What this does is copy all the files used by the python library into a folder. Creates a setup.py file for the module and archives everything so the library can be built somewhere else.
Setup.py build
This builds the python module back into a library (SPECIFICALLY FOR THIS OS).
As you may know, the computer that the python library originally came from will be different from the library that you are installing on.
It might have a different version of python
It might have a different operating system
It might have a different processor / motherboard / etc
For all the reasons listed above the code will not work on another computer. So setup.py sdist creates a module with only the source files needed to rebuild the library on another computer.
What setup.py does exactly is similar to what a makefile would do. It compiles sources / creates libraries all that stuff.
Now we have a copy of all the files we need in the library and they will work on our computer / operating system.
Setup.py install
Great we have all the files needed. But they won't work. Why? Well they have to be added to Python that's why. This is where install comes in. Now that we have a local copy of the library we need to install it into python so you can use it like so:
import mycustomlibrary
In order to do this we need to do several things including:
Copy files to their library folders in our version of python.
Make sure library can be imported using import command
Run any special install instructions for this library. (seting up paths, etc)
This is the most complicated part of the task. What if our library uses BeautifulSoup? This is not a part of Python Library. We'd have to install it in a way such that our library and any others can use BeautifulSoup without interfering with each other.
Also what if python was installed someplace else? What if it was installed on a server with many users?
Install handles all these problems transparently. What is does is make the library that we just built able to run. All you have to do is use the import command, install handles the rest.
So I wrote a Python script which does some simple stuff. It was originally going to run on a Unix server but due to crappy network security settings which TPTB refuse to change, we need to run it on a Windows server instead. However, the administrators of said Windows server refuse to do anything helpful like install Python.
What are my options for running a Python script on Windows without Python?
Consideration 1:
Something like Py2Exe - I found this after a quick Google search and it seems promising. From what I can tell, it'll generate a bunch of files but we can just xcopy that directory to our Windows machine and it will be completely isolated and not have any external dependencies. Does anyone have any insight on how well this works? Obviously, it depends on my Python script but fortunately this script is quite simple and only uses built in Python libraries such as urllib2 and urlparse.
Consideration 2:
We can assume the Windows server has at least some version of the .NET Framework installed too, which brings IronPython to mind. I've never used this before, but I've always wanted to. From what I can tell, it will compile Python code into CLS compliant IL code which can be run natively under the .NET runtime. However, does this require additional .NET libraries to be installed on the server? Can I just bundle those DLLs with my program? Or, does it require I rewrite my Python script to call into .NET Framework specific classes instead of using things like urllib2 or urlparse?
Thanks!
PS - The ironic part: I actually barely know Python and I'm a .NET expert, but I wrote the script in Python because I was told it would run on a Unix server. Had I known we'd end up running this on a Windows server, I'd have written the thing in C# to begin with in about 1/10th of the time. Fail.
Will they let you copy executables onto the server at all? If so then you should be able to do a non-admin installation of Python or use Portable Python which can just be copied into a folder without any installation at all.
Nothing wrong with Py2exe, but it does mean you then have to build the script into a fresh executable each time you update it. Also Py2exe has a slightly longer startup time than a Python interpreter because it has to extract the Python dlls into a temporary folder each time it runs; that only matters of course if you run your script a lot.
I wrote a program which uses a number of built in modules. The program is meant to be used by different persons on their systems. They dont have enough knowledge in python to install it when their system doesnt have the module needed to run the program. Is there any way of handling that.
Also I want to package the program as an executable in linux. It contains 3 py files and one text file only.
I think what you need is to create a debian package that handles the dependencies and the installation process.
I'm an Ubuntu user but this Complete Ubuntu Packaging Guide should help you get started. Good luck!
You can create an executable that contains your python modules and the python interpreter. You can use PyInstaller for creating such an executable.
I think the easiest way to achieve this on a debian distribution is to package your python application in a debian package. You can use this module to make life easier.