I am working on a POC of running tensorflow container via Singularity 3.6. I have a requirement that the user may need to pick an older version of CUDA which are available in a customized location rather than the system installed one.
I read the 3.6 document for GPGPU support, the -nv flag does all magic automatically, I could not see any step where I can specific which CUDA to use. I have multiple CUDA versions on the system and they are summoned by module load command normally.
Any idea is welcomed.
Cheers
Related
My python script is finished and working and I want to compile and have other users enjoy/benefit from it. The users don't need to install Pycharm or Visual Studio Code, something like an executable file or run in a command prompt then execute on their local machine or is there a way to convert it on a Tampermonkey Script?
How do I achieve this? Thank you very much in advance!
Googled and Youtubed but it's not what I'm looking for.
This question is probably answered multiple times, but the PyInstaller module is a great way to generate an executable that will run on Windows, and an app that will run on macOS.
Check out PyInstaller on PyPI.org: https://pypi.org/project/pyinstaller/
Project description
PyPI PyPI - Python Version Read the Docs (version) PyPI - Downloads
PyInstaller bundles a Python application and all its dependencies into a single package. The user can run the packaged app without installing a Python interpreter or any modules.
Documentation:
https://pyinstaller.org/
Code:
https://github.com/pyinstaller/pyinstaller
PyInstaller reads a Python script written by you. It analyzes your code to discover every other module and library your script needs in order to execute. Then it collects copies of all those files – including the active Python interpreter! – and puts them with your script in a single folder, or optionally in a single executable file.
PyInstaller is tested against Windows, macOS, and GNU/Linux. However, it is not a cross-compiler: to make a Windows app you run PyInstaller in Windows; to make a GNU/Linux app you run it in GNU/Linux, etc. PyInstaller has been used successfully with AIX, Solaris, FreeBSD and OpenBSD, but is not tested against them as part of the continuous integration tests.
Main Advantages
Works out-of-the-box with any Python version 3.7-3.11.
Fully multi-platform, and uses the OS support to load the dynamic libraries, thus ensuring full compatibility.
Correctly bundles the major Python packages such as numpy, PyQt5, PySide2, PyQt6, PySide6, wxPython, matplotlib and others out-of-the-box.
Compatible with many 3rd-party packages out-of-the-box. (All the required tricks to make external packages work are already integrated.)
Works with code signing on macOS.
Bundles MS Visual C++ DLLs on Windows.
Installation
PyInstaller is available on PyPI. You can install it through pip:
pip install pyinstaller
Requirements and Tested Platforms
Python:
3.7-3.11. Note that Python 3.10.0 contains a bug making it unsupportable by PyInstaller. PyInstaller will also not work with beta releases of Python 3.12.
tinyaes 1.0+ (only if using bytecode encryption). Instead of installing tinyaes, pip install pyinstaller[encryption] instead.
Windows (32bit/64bit):
PyInstaller should work on Windows 7 or newer, but we only officially support Windows 8+.
Support for Python installed from the Windows store without using virtual environments requires PyInstaller 4.4 or later.
Note that Windows on arm64 is not yet supported. If you have such a device and want to help us add arm64 support then please let us know on our issue tracker.
Linux:
GNU libc based distributions on architectures x86_64, aarch64, i686, ppc64le, s390x.
musl libc based distributions on architectures x86_64, aarch64.
ldd: Console application to print the shared libraries required by each program or shared library. This typically can be found in the distribution-package glibc or libc-bin.
objdump: Console application to display information from object files. This typically can be found in the distribution-package binutils.
objcopy: Console application to copy and translate object files. This typically can be found in the distribution-package binutils, too.
Raspberry Pi users on armv5-armv7 should add piwheels as an extra index url then pip install pyinstaller as usual.
macOS (x86_64 or arm64):
macOS 10.15 (Catalina) or newer.
Supports building universal2 applications provided that your installation of Python and all your dependencies are also compiled universal2.
Usage
Basic usage is very simple, just run it against your main script:
pyinstaller /path/to/yourscript.py
For more details, see the manual.
Untested Platforms
The following platforms have been contributed and any feedback or enhancements on these are welcome.
FreeBSD
ldd
Solaris
ldd
objdump
AIX
AIX 6.1 or newer. PyInstaller will not work with statically linked Python libraries.
ldd
Linux on any other libc implementation/architecture combination not listed above.
Before using any contributed platform, you need to build the PyInstaller bootloader. This will happen automatically when you pip install pyinstaller provided that you have an appropriate C compiler (typically either gcc or clang) and zlib’s development headers already installed.
Support
Official debugging guide: https://pyinstaller.org/en/v5.6.2/when-things-go-wrong.html
Assorted user contributed help topics: https://github.com/pyinstaller/pyinstaller/wiki
Web based Q&A forums: https://github.com/pyinstaller/pyinstaller/discussions
Email based Q&A forums: https://groups.google.com/g/pyinstaller
Changes in this Release
You can find a detailed list of changes in this release in the Changelog section of the manual.
I have created a python library using sklearn and some other dependencies. I want other developers to be able to use it in their programs, in a non-public environment(e.g.within a organization) They will use this library for to write their own applications.
Some questions that I have are -
What is a best way to make it available to other developers?
Let's say , the developers have their own python installation, and they use a version 1.x of a package(e.g. sklearn etc) but my
package uses 2.x, will there be a problem? If yes, how can i ensure they
can use my library.
I want to make my library available for both Python 2.7 and 3.x users. Do I need two different deployments? Currently my library
works(no version specific calls for 2.7/3.x) in both 2.7 and 3.x, if
the correct dependencies are pre-installed by the user.
The best way is to publish at PyPI. That way your user have ti just run pip install $LIB and got it with all dependencies (if you properly configured dependencies). See Python Packaging User Guide.
Just recommend your users to use virtualenv. Virtual environments are the way to separate and install different versions of Python libraries and programs to coexist at one system.
Very much depends on the nature of your library. There are libraries that can be installed to both Python 2 and 3 from one source and there are libraries that require different package for every python version.
I am not a regular Linux user so this might be completely trivial question. I am running 6.2 PUIAS version i386_64 on one of my GPU based "super" computers due to the unavailability of NVidia drivers for NetBSD. The installed version of Python is 2.6.6. I need 2.7.2 Python and newer version of scipy, numpy, matlibplot and friends. I have PUIAS and EPEL repositories enabled. However they do not have newer versions of Python. What is the "recommended" way to install newer version of Python without braking the system which depends on it. I am not interested in Python 3.2 due to the lack of libraries for scientific computing.
When the install-Python-from-source routine tells you to use make install, type make altinstall instead. This will leave the normal python executable untouched and instead create python2.7 for you to use. Install the other packages from source using this new executable. Don't forget to change the shebang line in your scripts accordingly.
I am going to answer my own question. For people who are using Python for scientific computing on RedHat clones (PUIAS for example) the easiest way to get all they need is to use rpm package manager and Enthought Python Distribution (EPD for short). EPD installs everything in a sandbox so system tools which are based on an obsolete version of Python are not massed up. However, paths have to be adjusted for system or even easier on the user base so that the using shell invokes non-system tools. One should never compile Python from source unless you are interesting in Python itself or in porting it to your favorite operating system rather than in your own research!
Short Description
Is it possible to install Python 2.7.1 64/32bit from python.org on top of an existing install (from python.org) of Python 2.7.1 32bit?
Background
I installed the 32bit version for wxPython(2.8) support which until now has given me zero issues. There are a few modules that I am now having difficultly installing (psycopg2 and mysql-python). The warning messages in homebrew constantly warn me about not having a 64bit version of Python on the path. These warnings only add to my list of wanting to use a 64 bit version.
At the time I selected the 32bit install, the GUI for a particular project was the main focus. Now the GUI has been come very simple and the database back-end support (using Django) is much more important. This being said, using the development version of wxPython 2.9 (which supports OS X and 64bit 2.7) has become acceptable.
For the non-mac users, to help explain why I had to use the 32bit version please see this Brief guide to using virtualenv in a wxpython project
System Information
Development System
OS: Mac OS X Snow Leopard (10.6.7)
Python: 2.7.1 with virutalenv / virutalenv-wrapper
Project Dependencies:
Note that MySQL could be PostgresSQL's psycopg2 if I can get the postgresql to install with homebrew
Django==1.2.5
MySQL-python==1.2.3
PIL==1.1.7
PyVISA==1.3
pyserial==2.5
virtualenv==1.5.1
virtualenvwrapper==2.6.3
wsgiref==0.1.2
wxPython==2.8.11.0
wxPython-common==2.8.11.0
Deployment System
OS: Windows XP / Windows 7
Python: Hopefully none (goal to use py2exe, or similar tool)
Current Thoughts
I fear that my goal cannot be accomplished based on the file paths alone. In Windows 7 the identifier (x86) is placed in the path showing that it is a 32bit program, but on OS X the path would be the same for 32bit or 64/32bit installs (/Library/Frameworks/Python.framework/Versions/2.7/).
Any thoughts or comments would be helpful!
Update 5-18-2011: 8:40 AM
I have confirmed that using the pre-compiled (.dmg) framework builds, installing the a 64 bit version does blow away the 32 bit install. This did negatively effect my virtual environments in that all of the things I have installed in the environments was based on the 32 bit install nearly every module threw an error of some sort.
I still have not achieved installing 32 bit and 64 bit on the same machine, however looking into homebrew with greater detail does look this could be possible. The trick would be to define your own formulas for each of the python builds (from source) and rename the install directory to something like 'Python27_32' and 'Python27_64'. I'll keep exploring this front as I have time.
EDIT 7-12-2011: 10:51 AM CST
Has anyone out there been able control how homebrew compile options? Specifically how to select to use a 64bit compiler or a 32 bit compiler? I will create a new SO question if this doesn't bring anything up.
Check out http://www.macports.org/ Which provides ports of various flavors of linux/unix tools that don't appear in the default MacInstallation and duplicates those that it needs otherwise. It installs everthing in /opt/local instead of stepping on the installed base. With some manipulation of PATH and LD_LIBRARY_PATH this might give you the options you need.
I would use something like virtualenv. This way you can have as many different installs of python as you wish.
EDIT: To use different versions of python use:
virtualenv -p python2.6
Reference
Another possible solution that was provided in an answer to one of my questions aimed at using a virtual environment in the Windows world was using Buildout. This approach looks like it would allow me create an environment with the 32bit installed and another one with the 64bit.
I have not tested this out yet, but wanted to pass it along to anyone else attempting this feat.
Cheers!
I have a linux VPS that uses an older version of python (2.4.3). This version doesn't include the UUID module, but I need it for a project. My options are to upgrade to python2.6 or find a way to make uuid work with the older version. I am a complete linux newbie. I don't know how to upgrade python safely or how I could get the UUID modules working with the already installed version. What is a better option and how would I go about doing it?
The safest way to upgrading Python is to install it to a different location (away from the default system path).
To do this, download the source of python and do a
./configure --prefix=/opt
(Assuming you want to install it to /opt which is where most install non system dependant stuff to)
The reason why I say this is because some other system libraries may depend on the current version of python.
Another reason is that as you are doing your own custom development, it is much better to have control over what version of the libraries (or interpreters) you are using rather than have a operating system patch break something that was working before. A controlled upgrade is better than having the application break on you all of a sudden.
The UUID module exists as a separate package for Python 2.3 and up:
http://pypi.python.org/pypi/uuid/1.30
So you can either install that in your Python2.4, or install Python2.6. If your distro doesn't have it, then Python is quite simple to compile from source. Look through the requirements to make sure all the libraries you need/want are installed before compiling Python. That's it.
The best solution will be installing python2.6 in the choosen directory - It will you give you access to many great features and better memory handling (infamous python=2.4 memory leak problem).
I have got several pythons installed onto my two computers, I found that the best solution for are two directories:
$HOME/usr-32
$HOME/usr-64
respectively to using operating system (I share $HOME between 32 and 64 bit versions of Linux).
In each I have one directory for every application/program, for example:
ls ~/usr-64/python-2.6.2/
bin include lib share
It leads completetely to avoiding conflicts between version and gives great portability (you can use usb pendrives etc).
Python 2.6.2 in previously example has been installed with option:
./configure --prefix=$HOME/usr-64/python-2.6.2