I have a windows 7 computer and was wondering whether to use the windows version of python or the one in cygwin. Especially with regard to modules that do not come pre-installed, which one is easier to install new modules?
ActivePython works just fine on Win7. Cygwin would add an unnecessary layer of complexity.
It really depends on what you want to use it for, far more than which one is easier to install new modules.
If you plan on running scripts from within the cygwin bash shell, or to access files within the cygwin subtree of your directory, to WSGI to a cygwin web server, etc., you probably want the Cygwin version.
If you plan on running scripts from within the cmd.exe DOS prompt, or to access files in special locations under your home directory, or to WSGI to a native web server, you probably want the native version.
If you really do want to know which one is easier to install new modules for… the answer is that it depends.
If you want pre-built binary packages, Cygwin only has a handful of them, while Cristoph Gohkle has a ton of packages for native Python.
If you want to build packages yourself (manually, or just using pip automatically), and you need any packages that use C extension modules, you'll need to set up a compiler, which is a bit easier in Cygwin, but not all that hard in either.
There are a few modules out there that aren't developed or tested for cygwin, and will guess your platform as linux or Windows or something else wrong, or just make inappropriate assumptions.
There are also a few modules that only work on POSIX, that do happen to work with cygwin but not with native Windows.
Both work, it all comes down to which you're more comfortable using; windows or cygwin.
Related
up until recently I have only worked with one version of Python and used virtual environments every now and then. Now, I am working with some libraries that require older version of Python. So, I am very confused. Could anyone please clear up some of my confusion?
How do I install multiple Python versions?
I initially had Python version 3.8.x but upgraded to 3.10.x last month. There is currently only that one version on my PC now.
I wanted to install one of the Python 3.8.x version and went to https://www.python.org/downloads/. It lists a lot of versions and subversions like 3.6, 3.7, 3.8 etc. etc. with 3.8.1, 3.8.2 till 3.8.13. Which one should I pick?
I actually went ahead with 3.8.12 and downloaded the Tarball on the page: https://www.python.org/downloads/release/python-3812/
I extracted the tarball (23.6MB) and it created a folder with a setup.py file.
Is Python 3.8.12 now installed? Clicking on the setup.py file simply flashes the terminal for a second.
I have a few more questions. Hopefully, they won't get me downvoted. I am just confused and couldn't find proper answers for them.
Why does Python have such heavy dependency on the exact versions of libraries and packages etc?
For example, this question
How can I run Mozilla TTS/Coqui TTS training with CUDA on a Windows system?. This seems very beginner unfriendly. Slightly mismatched package
version can prevent any program from running.
Do virtual environments copy all the files from the main Python installation to create a virtual environment and then install specific packages inside it? Isn't that a lot of wasted resources in duplication because almost all projects require there own virtual environment.
Your questions depend a bit on "all the other software". For example, as #leiyang indicated, the answer will be different if you use conda vs just pip on vanilla CPython (the standard Windows Python).
I'm also going to assume you're actually on Windows, because on Linux I would recommend looking at pyenv. There is a pyenv-win, which may be worth looking into, but I don't use it myself because it doesn't play as nice if you also want (mini)conda environments.
1. (a) How do I install multiple Python versions?
Simply download the various installers and install them in sensible locations. E.g. "C:\Program Files\Python39" for Python 3.9, or some other location where you're allowed to install software.
Don't have Python add itself to the PATH though, since that'll only find the last version to do so and can really confuse things.
Also, you probably want to use virtual environments consistently, as this ties a specific project very clearly to a specific Python version, avoiding future confusion or problems.
1. (b) "3.8.1, 3.8.2 till 3.8.13" which should I pick?
Always pick the latest 3.x.y, so if there's a 3.8.13 for Windows, but no 3.8.14, pick that. Check if the version is actually available for your operating system, sometimes there are later versions for one OS, but not for another.
The reason is that between a verion like 3.6 and 3.7, there may be major changes that change how Python works. Generally, there will be backwards compatibility, but some changes may break how some of your packages work. However, when going up a minor version, there won't be any such breaking changes, just fixes and additions that don't get in the way of what was already there. A change from 2.x to 3.x only happens if the language itself goes through a major change, and rarely happens (and perhaps never will again, depending on who you ask).
An exception to the "no minor version change problems" is of course if you run some script that very specifically relies on something that was broken in 3.8.6, but no fixed in 3.8.7+ (as an example). However, that's very bad coding, to rely on what's broken and not fixing it later, so only go along with that if you have no other recourse. Otherwise, just the latest minor version of any version you're after.
Also: make sure you pick the correct architecture. If there's no specific requirement, just pick 64-bit, but if your script needs to interact with other installed software at the binary level, it may require you to install 32-bit Python (and 32-bit packages as well). If you have no such requirement, 64-bit allows more memory access and has some other benefits on modern computers.
2. Why does Python have such heavy dependency on the exact versions of libraries and packages etc?
It's not just Python, this is true for many languages. It's just more visible to the end user for Python, because you run it as an interpreted language. It's only compiled at the very last moment, on the computer it's running on.
This has the advantage that the code can run on a variety of computers and operating systems, but the downside that you need the right environment where you're running it. For people who code in languages like C++, they have to deal with this problem when they're coding, but target a much smaller number of environments (although there's still runtimes to contend with, and DirectX versions, etc.). Other languages just roll everything up into the program that's being distributed, while a Python script by itself can be tiny. It's a design choice.
There are a lot of tools to help you automate the process though and well-written packages will make the process quite painless. If you feel Python is very shakey when it comes to this, that's probable to blame on the packages or scripts you're using, not really the language. The only fault of the language is that it makes it very easy for developers to make such a mess for you and make your life hard with getting specific requirements.
Look for alternatives, but if you can't avoid using a specific script or package, once you figure out how to install or use it, document it or better yet, automate it so you don't have to think about it again.
3. Do virtual environments copy all the files from the main Python installation to create a virtual environment and then install specific packages inside it? Isn't that a lot of wasted resources in duplication because almost all projects require there own virtual environment.
Not all of them, but quite a few of them. However, you still need the original installation to be present on the system. Also, you can't pick up a virtual environment and put it somewhere else, not even on the same PC without some careful changes (often better to just recreate it).
You're right that this is a bit wasteful - but this is a difficult choice.
Either Python would be even more complicated, having to manage many different version of packages in a single environment (Java developers will be able to tell you war stories about this, with their dependency management - or wax lyrically about it, once they get it themselves).
Or you get what we have: a bit wasteful, but in the end diskspace is a lot cheaper than your time. And unlike your time, diskspace is almost infinitely expandable.
You can share virtual environments between very similar projects though, but especially if you get your code from someone else, it's best to not have to worry and just give up a few dozen MB for the project. On the upside: you can just delete a virtual environment directory and that pretty much gets rid of the whole things. Some applications like PyCharm may remember that it was once there, but other than that, that's the virtual environment gone.
Just install them. You can have any number of Python installations side by side. Unless you need to have 2 different minor versions, for example 3.10.1 and 3.10.2, there is no need to do anything special. (And if you do need that then you don't need any advice.) Just set up separate shortcuts for each one.
Remember you have to install any 3rd-party libraries you need in each version. To do this, navigate to the Scripts folder in the version you want to do the install in, and run pip from that folder.
Python's 3rd-party libraries are open-source and come from projects that have release schedules that don't necessarily coincide with Python's. So they will not always have a version available that coincides with the latest version of Python.
Often you can get around this by downloading unofficial binaries from Christoph Gohlke's site. Google Python Gohlke.
Install Python using the windows executable installers from python.org. If the version is 3.x.y, use the highest y that has a windows executable installer. Unless your machine is very old, use the 64-bit versions. Do not have them add python to your PATH environment variable, but in only one of the installs have it install the python launcher py. That will help you in using multiple versions. See e.g. here.
Python itself does not. But some modules/libraries do. Especially those that are not purely written in Python but contain extensions written in C(++). The reason for this is that compiling programs on ms-windows can be a real PITA. Unlike UNIX-like operating systems with Linux, ms-windows doesn't come with development tools as standard. Nor does it have decent package management. Since the official Python installers are built with microsoft tools, you need to use those with C(++) extensions as well. Before 2015, you even had to use exactly the same version of the compiler that Python was built with. That is still a good idea, but no longer strictly necessary. So it is a signigicant amount of work for developers to release binary packages for each supported Python version. It is much easier for them to say "requires Python 3.x".
I am currently starting a business where I will be providing support to clients directly on their business offices. I need to be able to go to different computers and be able to run custom python scripts, my question is if there's a way to make my python environment portable?
Assuming that your users are running Windows, I see two options here.
If you have already defined which scripts you will be running, compile them into exe files using py2exe, that way you can just plug a USB and run them as needed. (the caveat is that some antivirus will automatically block the unsigned executables)
The other option is to use WinPython, that is a full python environment with a lot of packages already preinstalled that ives in it's own directory. In case you need to install a new package, just use the Powershell or CMD that comes with it and use the preinstalled "pip".
I found something interesting here Portable Python. I use that method to create portable Python 3.9 and 3.10 and everything works so have a look.
I need to automate a cross-platform application build. Entire build runs on Windows machine. Part of it is written in Python and compiles for OS X. Currently this part of build is done manually on OS X.
I tried pyinstaller but it looks like it only building for the platform that it is running on. I also tried py2app but it did not install on Windows.
Are there any tools to compile Python script to OS X app on Windows machine?
Short answer:
Apparently, no simple way to do this with the standard set of tools you have mentioned. I outline a completely unprobable solution in the end that's probably too complex to consider.
End result: Keep doing it manually, it's probably the best option so far.
Drawing from credible and/or official sources:
There's already a long and curated list of options specified in micheal0x2a's excelent answer in Create a single executable from a Python project which outlines most tools available for creating standalone distributions of a Python program. One line stands out:
Also, unless otherwise noted, all programs listed below will produce an exe specifically for the operating system it's running in.
I might have read things wrong but there's no explicit note of cross platform support.
Similar lists can be found in in Distribution Utilities and Freezing Your Code — The Hitchhiker's Guide to Python. From these we can take a look at the Windows supported projects:
bbfreeze: Not maintained, no documentation on limitations generally superceded by the rest in this list.
PyInstaller: According to their documentation:
PyInstaller is tested against Windows, Mac OS X, and Linux. However, it is not a cross-compiler: to make a Windows app you run PyInstaller in Windows; to make a Linux app you run it in Linux, etc. PyInstaller has been used successfully with AIX, Solaris, and FreeBSD, but is not tested against them.
PyInstaller once tried to support cross-compilation (From Linux -> Windows) but later dropped the idea.
cx_freeze: Looking at their Freezing for other platforms question in their Frequently Asked Questions list:
cx_Freeze works on Windows, Mac and Linux, but on each platform it only makes an executable that runs on that platform. So if you want to freeze your program for Windows, freeze it on Windows; if you want to run it on Macs, freeze it on a Mac.
py2app: This is not an option since py2app is supported only OSX machines based on their note:
NOTE: py2app must be used on OSX to build applications, it cannot create Mac applications on other platforms.
So installing it on windows is a no-go.
This wraps out the tools available on for creating standalone applications on Windows. Even if not on Windows though, solutions don't exist for creating an OS agnostic tool.
The general consensus for achieving these sort of things is via Virtual Machines; you create a VM image of the target OS, run the dedicated tool for that OS inside the vm and then tranfer the bundled executable to compatible machines. Since OSX is generally not easy to virtualize, from what I know, you kinda run out of luck here.
One complex probable way:
Now, the reason why I said there is no simple way to do this is because there might be one but, from what I can see, it is so tedious you shouldn't even consider it. The general issue we're facing here is that executables for windows are simply not compatible with OSX, heck, Linux executables aren't either. So what we need is the ability to cross-compile things.
One compiler that I've heard supports cross-compilation is clang. So, in order remedy the incompatibility of executables, you could theoretically use clang as a cross compiler. Problem is, this is not a simple task; it is probably way harder than what you're willing to do since it is riddled with complex issues (From OSX to Windows example).
If you do actually find a way to that you now need a .cpp/.c file from your python scripts. Thankfully this is something that can be done by using tools like Nuitka or Cython.
The gist of these is the same, enter .py exit .cpp/.c. The second tricky part might be setting up the clang compiler to be used with these. I honestly have no idea if anyone has done this.
I haven't looked into them much, but for Nuitka, I know that it can also be used to create a standalone executable when passed the --standalone flag (See: Moving to other machines). So, theoretically you could use Nuitka with clang configured to cross-compile for OSX. Then you can rest; you've earned it.
You can use a docker image https://github.com/sickcodes/Docker-OSX like this to simulate a mac computer.
Then from this simulated mac you could install the pyinstaller and run your command from there.
This would then produce the desired file.
Some people do the same way to create windows executables using pyinstaller on linux.
I dont see why this could not work from windows to mac.
Install fabric (Python module, easily installed with pip) on your Windows machine so that you can run the build on your Mac as part of the same automated build process.
Fabric allows you to utilize SSH from Python. So long as you know how to access the Mac over SSH (just need the IP, username, and password), this is easy. Set it up like this:
from fabric.api import env
env.host_string = '<IP of the Mac Here>'
env.user = '<Username on the Mac>'
env.password = '<Password of the user>'
Then copy over the necessary source files from the Windows machine to the Mac like this (once fabric has been set up as above):
from fabric.operations import put
put(r'\path\to\local\source\files', '/path/to/where/you/want/them')
Now you want to run your build tool, whatever it is. You'll need to use run or sudo for that, depending on if the tool requires admin privileges or not.
from fabric.operations import run, sudo
# sudo works the same as run if you need that instead
run('/path/to/build/tool arguments for build tool')
Finally you have the build output which you can retrieve using get.
from fabric.operations import get
get('/path/to/dist/on/mac', '\local\path\to\store\on\windows')
There we go. Pretty simple. No VM needed - just automate the process that you were already manually doing before. The only real requirement is that your Mac has to be available to be connected to during the build process. If you don't have a readily available Mac server, consider just picking up a used Mac Mini - you can get them for as little as $50 on ebay, if budget is a concern.
We have an app (a bunch of Twisted classes actually) which runs on a specific Python version and depends on quite a bit of modules. This app needs to be deployed onto a Windows Server machine which has no access to Internet.
Currently we are choosing between:
having to install Python prior to everything else, and a Python script which unpacks all modules and runs setup.py,
making an NSIS installer which installs Python, then all modules with .exe installers, then unpacks smaller modules into some other dir, then adds the dir to %PYTHONPATH%.
What is the good accepted way of dealing with such situation? Obviously we cannot use pip, easy_install.exe and other blessed tools, and our approaches are silly and inelegant.
As a third option you can consider deploing the application as an executable using PyInstaller (http://www.pyinstaller.org). You dont need to install anything on the client machine (not even python)
PyInstaller is a program that converts (packages) Python programs into stand-alone executables, under Windows, Linux, Mac OS X, Solaris and AIX. Its main advantages over similar tools are that PyInstaller works with any version of Python since 2.4, it builds smaller executables thanks to transparent compression, it is fully multi-platform, and use the OS support to load the dynamic libraries, thus ensuring full compatibility.
I have used it in a project to deploy standalone application in both Linux and Windows. Worked like a charm. My project also used Twisted.
Between your current two choices the setup.py approach is more pythonic. But beware that if any of your modules has some c implementation for faster performance that need to be compiled, you can't do that on you client's machine.
Try to use wheels. It designed to cover your case, i.e. you build wheel once, downloading all required packages. Then you just copy wheel archive to the target machine and install your application without downloading anything.
Pythons installed under WinXP have dirs like DLLs, DOC, include, etc. but python (2.5) installed with cygwin is a bare python.exe. My motivation for asking is that 'things' under XP don't seem to be finding 'other things' under cygwin and vice versa, I want to start developing with Qt, I like shells, and I do not like MS; I thought if I got all the components under one roof, I could finally start to have scripts find executables which could find files and such. 1. Can I simply copy the contents of an XP installation into the cygwin tree? 2. Is the XP flavor of Python different from the cygwin flavor? (Same CPU, he pointed out, naively.) 3. Someone must work with a full-fledged (if snakes had feathers...) Python from within cygwin; how is it done?
Disclaimer 1: I have never compiled anything under XP or cygwin; had hoped not to have to go there, hence, python in the first place. Disclaimer 2: sorry if this is a ServerFault question, but they seemed to be system people over there and this is (in my case) a lowly desktop.
I use Python from within cygwin, but I don't use the version that cygwin gives you the option of installing as I don't have the control over version number used that I need (we use an older version at work). I have my python version installed via the windows installer (the xp version as you put it) and add the /cygdrive/c/Python2x directory to my PATH environment variable.
Well, in my windows environment I use active python and it, so far, works for me.
Just a little off the question, but...
Have you considered running Sun's VirtualBox with Fedora or Ubuntu inside of it? I'm assuming you have to / need to use windows because you still are, but don't like it. Then you would have python running inside a native linux desktop without any of the troubles you mentioned.
And if you want something that is really easy and portable, then just use Python on Windows, not mixed in with cygwin.
$0.02
This probably has little value, but... I found myself in this exact situation -- we use ActivePython2.5 in production (pure windows environment) and I was trying to do my development within cygwin and cygwin's Python...
After ripping out half of my hair, I have now switched over to Console2, gvim, iPython and ActivePython2.5.
I'm less than thrilled dealing with Windows tools (and their concomitant warts), but at least I'm not getting in my own way when it comes to development. For a while I found I was spending more time trying to get my tools to play nice than actually getting any work done.
Good luck on this one.
I accidentally stumbled on this - If I launch Cygwin from the Cygwin.bat file (which is present directly under the main folder), I get access to the Python version installed under Cygwin (i.e 2.6.8)
If I instead launch the Cygwin from bash.exe under bin directory (C:\Cygwin\bin\bash.exe for me), running "Python -V" shows that I have access to 2.7.3 version of Python (that was installed for Windows).
So, I guess you can do the same.