I am implementing an interface which allows Python to be loaded and used from a Windows C++ application. It should be able to use any Python registered in the Windows registry. This seems to work fine with e.g. Python 3.8 downloaded from www.python.org. However, with current Anaconda/Python 3.7 I am having problems. It looks like it expects a lot of directories to be present at PATH, and this knowledge seems to be hidden in a cryptic conda.bat file. Especially the folder C:\ProgramData\Anaconda\Library\bin seems critical, without this e.g. numpy won't load.
In short, assuming that I start just with the following info piece from registry:
[HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\3.7\InstallPath]
#="C:\\ProgramData\\Anaconda3"
how should I prepare PATH and potentially other things so that the loaded Python DLL, from Anaconda or not, would be fully functional? For this particular Anaconda version I can add them by hand, but this seems not so sustainable.
I know that one solution is to select a check box during Anaconda install which will put the needed directories to the system PATH. However, this check box is not selected by default and is marked as "not recommended", so there is not much hope that our end users will actually check it.
Related
I created a Python package that is actually a command line interface. Everything works fine, but I would like to be able to store simple user settings (a few values) in some file (json, yaml, or whatever). Handling such files itself is not a problem, but I don't know where I could store them. My program can be installed using pip install https://github.com/repo and works fully off-line.
It seems that Python does not allow me to store settings files in the same folder, where the compiled program itself is located (Programs\Python\Python38\Scripts) which makes sense, but I don't know how to do it any other way.
Inspired by the comments under the question, I searched a little bit more and found the answer to my problem.
I store my settings file in C:\users\me\appdata\local\programs\python\python38\lib\site-packages\my_package\config.json which can be accessed with
os.path.join(os.path.dirname(os.path.abspath(__file__)), 'config.json')
I haven't tested this on Linux or macOS yet, but I think it will work.
EDIT: Unfortunately it does not work on Linux without sudo. For this reason, I used the following:
os.path.join(os.path.expanduser("~"), '.my_package_name', 'config.json')
I have to do a new course for University using python. A year or so ago I have installed Anaconda, but never really worked with it. Before starting I wanted to update everything, so I uninstalled my python and Anaconda version and reinstalled the newest version (I know I could have just updated everything).
I would like to work with VS2017, since this is the IDE I'm used to work with (coming from a c# background), however within the python environment window, my old versions are still visible, although with a strike-through font:
VS2017 has no option to remove the broken/uninstalled environments, but refers you to this website. In the bottom section there is a description to my solve my problem. Normally I don't really like to edit the registry, since I'm not know my way around this stuff, however this being directly from the learn.microsoft.com pages, I thought it was ok.
Problem only is, the changes didn't have any effect on my issue whatsoever.
(already did the obvious stuff like restarting VS2017 and Windows).
Additional Info
My problem is that I wanted to run the python script skeleton we got from the course to check if all the modules and python itself is working properly. However I always get a dll load failed error on some of the modules (matplotlib for example). Running the scripts on other IDEs (like Anaconda's integrated Spyder IDE) however works just fine, so I know the modules are good to go on my machine. I wanted to rule the above mentioned issue out as source of error before looking further.
Checking with Process Monitor (starting up VS with monitoring active, up to bringing up the Python environments list in it; then stopping monitoring and setting filters: Process name is devenv.exe, Path contains python, conda or ContinuumAnalytics (three separate filters)) shows that VS searches these locations for Python installation data:
Registry keys, under HKCU (the document fails to mention this) and HKLM:
\Software\Python and \Software\Wow6432Node\Python (which is seen as the former by 32-bit processes)
Files:
<user profile>\.conda directory
It also looks for conda.exe in a few locations
I don't have it, but if I did, it would be possible to see with procmon which command lines VS is invoking it with. Then you could e.g. do the same yourself and see what information VS gets from it.
If VS finds the entries that you list, something referring to what you see in the list must be under these locations somewhere.
To remove the entries, as I already mentioned,
First check if you have the corresponding product installed and uninstall it if you do. Entries under HKCU refer to products installed per-user so you'll have to run appwiz.cpl as yourself (or rather, as the same user that you run VS as) to see them.
If you really don't have it installed, do the usual manual cleansing procedure. Delete anything from the registry and disk that looks relevant (by name, location), including the above entries. At your own risk, of course. For VS to stop finding them, deleting the entries should be enough. You can also try to reinstall and uninstall the exact same version of software (which can be tricky to find) and hope it uninstalls correctly this time.
To correct python environment in Visual Studio that doesn't have a repair option, or to remove an invalid environment, use the following steps to modify the registry directly. Visual Studio automatically updates the Python Environments window when you make changes to the registry.
Run regedit.exe.
Navigate to HKEY_LOCAL_MACHINE\SOFTWARE\Python. For IronPython, look
for IronPython instead.
Expand the node that matches the distribution, such as Python Core
for CPython or ContinuumAnalytics for Anaconda. For IronPython,
expand the version number node.
Inspect the values under the InstallPath node:
If the environment still exists on your computer, change the value of ExecutablePath to the correct location. Also correct the (Default) and WindowedExecutablePath values as necessary.
If the environment no longer exists on your computer and you want to remove it from the Python Environments window, delete the parent node of InstallPath, such as 3.6 in the image above.
Ref: Microsoft Doc - https://learn.microsoft.com/en-us/visualstudio/python/managing-python-environments-in-visual-studio?view=vs-2019#fix-or-delete-invalid-environments
I don't even know the right way to put my question but i will try my best.
I downloaded (Python-3.4.2.tar) the source code of a python interpreter from www.python.org
I extracted the files(using 7-zip).
Now lets say i latter want to use the unziped/extracted fies to create an installer i.e put it in a form that i can double click and Python-3.4.2 will be installed in my computer.
i guess it is called creating a build distriution.
I know i can just download Python-3.4.2.exe from the site and install right away but i want to know how it goes from being source code to becoming something one can install.
Here's a starting point for compiling:
https://github.com/python/cpython/blob/2.7/PCbuild/readme.txt
I was able to compile 2.7.13 with:
cd PCBuild
cmd /c get_externals.bat
cmd /c build.bat -e --no-tkinter "/p:PlatformToolset=v100"
3.4 may require a later compiler.
Here's how to build an installer:
https://github.com/python/cpython/blob/2.7/Tools/msi/README.txt
For building the CPython python interpreter from source, you'll want to have a look at the instructions at https://docs.python.org/devguide/
These instructions probably also (somewhere) contain the steps to produce a Windows installer, which seems to be done with some tool called PCbuild.
Basically, I think what you are asking is how you can distribute the Python runtime along with your program. The process is pretty simple. First, you may want to take a look at Python's distutils. Secondly, you will need to distribute the Python runtime. Python is currently released within installation binaries for pretty much every major operating system. You have the option of compiling target system(s) yourself in order to make Python silently install.
You will then need to decide where you want the files to go. Most people don't like this sort of behavior, so I recommend putting the files in your programs directory. Shortcuts, system variables, et cetera will need to be looked into.
As another option you can consider porting your script(s) to Jython, as most people tend to have at least the Java runtime. In the process of porting your code there is a way of constructing Java .class files with Jython when coded a certain way. They can then be placed into an executable Jar file. Easy peasy, but I haven't tried this myself, to be honest.
If you want to stick with pure Python, the last hurtle is putting everything in one file. There are plenty of tools for that. I would try Github for starters. For bonus points you can always code your own self-extracting binary.
Post Script: With the title changed, I think you are looking for Creating Build Distributions. It is the second link within the Google results for the query "how to compile python installer". Further down the same page I found this gem.
Is there a way to distribute a python library with an application, so that it can be run with out any installation? The app is primarily going to be used in a computer lab, where users do not have permission to install global libraries. Ideally, users would simply be able to unzip a folder and run the app. The following can be assumed:
The python interpreter is present
Linux operating system
The specific library I need is matplotlib, but I would like to find a generic solution. I've looked at programs like PyInstaller, but they create very large programs that are slow to start. They also include a python interpreter, which is unnecessary.
Firstly, p2exe is Windows only.
In principle you can put all of you libraries into the ZIP file so they get expanded in place with thie application. At most you may need to adjust the PYTHONPATH variable to point at the lib's location.
There is no technical difference between modules installed on the path and in the system's python installation.
Have you looked at cxfreeze?
I'm developing python C++ extensions for use in both OSX and linux. Currently, I can run my code with a wrapper script wrapper.sh:
#!/bin/bash
trunk=`dirname $0`
trunk=`cd $trunk; pwd`
export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:$trunk/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$trunk/lib/:$trunk/src/hdf5/lib/:$trunk/src/python/lib
$trunk/src/python/bin/python "$#"
which is able to set up my run like this: wrapper.sh app.py
What I would like to do is to eliminate the need for wrapper.sh, so I need alternatives for DYLD_LIBRARY_PATH and LD_LIBRARY_PATH. I can not put my libraries in some standard location like /usr/local/lib because on my machine, I maintain several independent instances of my libraries. That is, my libraries need to be kept somewhere relative to my installation path. I can't put these environment variables in my login script for the same reason. Currently, I need to call one of my wrapper.sh scripts to use the associated libraries. My goal is to be able to run merely app.py, which if it lives in my installation path, should be able to find its associated python and libraries. The purpose is to simplify execution for users, and to simplify usage of external tools like nosetests.
One alternative seems to be using rpath when I build my version of python:
./configure --enable-shared --prefix=$(CURDIR)/$(PYTHON_DIR) LDFLAGS="-Wl,-rpath,$(CURDIR)/lib/ -Wl,-rpath,$(CURDIR)/src/hdf5/lib -Wl,-rpath,$(CURDIR)/src/python/lib"
This trick seems to work fine on linux, even though one of my libraries ended up needing to be copied directly into trunk/src/python/lib/python2.6/lib-dynload for some reason unclear to me. However, this trick is not working on OSX; it looks like I need to run install_name_tool on all my dylibs libraries.
The other alternative I came up with was to do something like this:
ln -s wrapper.sh python
so that my scripts could all use #! ../python, but I'm getting Unmatched ". errors. Same thing if I use #! ../wrapper.sh. I'm not really an expert in bash...
However, these all seem so unnecessarily complicated, and surely this is something that other people have solved?? Thanks for any advice!
For python extensions, consider using PYTHONPATH: the Python interpreter will search the PYTHONPATH for .py/.pyc/.pyo/.so modules, as well as packages. See docs for Python 2.x as well as docs for Python 3.x; specifically the section named "The Module Search Path" on both pages. This also references information that seems to indicate that it is possible to update the module search path at runtime, which, if true, means that you could add all that logic to your program and it can hunt for its libraries on its own (say if it installs a copy in /usr/libexec/pkgname/... somewhere or something).
For all but the most complex of cases, though, setting PYTHONPATH and using a shell-script or native-compiled binary wrapper to start the core program is an okay approach, and one that is also used in other language environments including Mono and Java.
Not sure if this would be an acceptable (partial) solution in your circumstances, but another way to get libraries noticed by ld on linux is to add the path to the libraries to /etc/ld.so.conf and then runldconfig
For the Mac I don't remember the details, but I think Apple provide some resources for distributing apps packaged as a .app which includes some default locations (relative to the root of the .app) for libraries, or "frameworks" as they call them. Would require some googling from there - sorry can't help further on that but hope you get some progress :-)