If a python script using package X is running and package X is being upgraded, will it lead to permission problem that will cause the upgrade to fail?
I am using Windows 10, Anaconda v5.2 running on python v3.6 and conda to do update of packages.
Although pure Python files are compiled in memory when imported, and the source is (almost) no longer relevant after that, that's not the end of the story.
Packages may have extra assets that are lazily loaded, or your program or its dependencies may load dependent modules on demand, so, if running during an upgrade, it may load unexpected versions of packages/resources, or even halfway-upgraded packages.
Also, native (=non Python-only) modules - .pyd files on Windows - are dlls that are loaded in the importing process. As dlls are mapped in memory with no sharing, replacing them while they are loaded is not allowed, so this may block the upgrade of the relevant packages.
I tried a simple test just now. I ran a python script which loops forever. It uses numpy. Then, I tried to install a python package(pytorch) that requires downgrading the numpy version. When the script is running, the installation failed with some "no permission" error message. After I stopped the script, the installation succeeded.
Based on the results of this experiment, the conclusion is that it is recommended to stop all python scripts when performing python package upgrade.
When you import, you're creating a local instance of the package in your RAM for running. So upgrading your packages should not affect your scripts running.
You can look at from importlib import reload to reload your packages while your scripts are still running
Related
I created an executable out of a simple Python script, using pyinstaller on Ubuntu 18.04, and tested it in a different computer (also with Ubutnu 18) and worked perfectly.
However when trying the same with a more complex script (more library imports) the executable fails in the other computer with the error
ImportError: /lob/x86_64-linux-gnu/libc.so.6: version 'GLIBC_2.25' not found
This can't be a Python incompatibility (see https://github.com/pyinstaller/pyinstaller/issues/4758), as the other script did work fine. So it most probably is based on some of the libraries the second script imports.
How can I include the libraries imported in the executable made by Pyinstaller (if that is even the origin of this error)?
Solution A
I have not confirmed this solution, but it sometimes helps. Delete directories ./build and ./dist, then try creating the executable again with pyinstaller.
Solution B
The solution, for me at least, is to build your executable on an older version of your OS.
I was seeing the same error.
ImportError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /tmp/_MEIjdcWu4/./libX11.so.6)
[32614] Failed to execute script 'test_executable' due to unhandled exception!
I built my exectuable with Pyinstaller on Ubuntu 22.04. Then I copied and ran the executable on the older Ubuntu 20.04 and the error was encountered.
Per the comment below, this might be a compatibility issue where the executable built on a newer OS is not compatible with older OSs.
"For what is worth, the issue could be that the libraries bundled with
the built program conflict with the system libraries, preventing the
DRI driver from properly loading.
The culprit could be either standard c/c++ libraries (libgcc_s.so.1,
libstdc++.so.6) or maybe the X11 libraries (libX11.so.6, libXau.so.6,
libXdmcp.so.6, libXext.so.6, `libXrender.so.1˙). Perhaps more likely
former than the latter.
For example, if libstdc++.so.6 on the build system is older than the
one used by the target system, then the non-bundled libraries will
fail to load due to missing symbols (which are present in the newer,
system version of the library, but not in the bundled one). This is
actually quite a common issue with binary-only software on linux,
especially on more bleeding edge distributions. In those cases,
removing the bundled version of the offending library may help.
(You have a similar issue with system libgvfsdbus.so, which is missing
a symbol that is not available in the bundled libglib-2.0.so.0, which
is probably older than the glib library available on the system)."
Source:
https://github.com/cryptoadvance/specter-desktop/issues/373#issuecomment-694476451
Sorry in advance if this question has been asked before,
So after some time, I wanted to start a new python project. My previous computer (on which my python files were) died. I had saved my projects in my Dropbox. Now I installed python (3.8, there is also an anaconda installation, but it should not interfere with the python installation) on my new PC, and I cannot import any library to those files.
The python shell can find the imported packages (imported using pip), but even when I move the files to C:\Users\Username\AppData\Local\Programs\Python\Python38-32\Scripts (single user installation). It doesn't work.
I have tried uninstalling and re-installing pygame (in this example. Any library is unusable) using pip, pip3 and even pip3.8, I have added the .whl file by hand, it all didn't work. I have tried a virtual environment, but I can't get that to work either.
I run Windows 10 on a 64-bit computer.
first be sure you know which python installation you use with which import files etc.
you can copy your files not in scripts, but in lib somewhere in site-packages dir.
add your scripts to the python path! sys.path.add(.....) Otherwise python is blind and can't see them
I use conda update --all to update my packages. Recently, I encountered an error with Anaconda build, posted at Error while trying to update and use scipy module in Anaconda. It seems now the issue has been fixed. Is there any way, I can test all modules one by one by importing them and deleting them ? I am requesting this because I have noticed that if import doesn't work, I spend a lot of time figuring out the dependency and then the package that is causing this. For instance, a few minutes ago I found that PyCharm 2018.2.4 breaks with the latest version of matplotlib (3.0.0). Hence, it might be helpful to run some type of test script after running conda update --all to ensure that all packages are indeed working--i.e. importable.
I did some research on this topic and found three sources.
First, Anaconda offers run_test.py (Source: https://conda.io/docs/user-guide/tasks/build-packages/recipe.html). However, being new to the world of Python, I am unsure how to go about running a script in Anaconda terminal.
Second, I found: https://conda.io/docs/user-guide/install/test-installation.html. However, this just tells me the version of the package. I am not interested in the version. I need to know whether all packages import properly.
Finally, I found out that there is a method to run test script for all packages at https://anaconda-installer.readthedocs.io/en/latest/testing.html. However, I am unsure how I can run make in Anaconda terminal. I used to use make long time ago when I worked on gcc on Unix environment. Being new to Python, I am unsure how to go about handling this.
I'd appreciate any thoughts or any test script that could help us verify two things:
a) whether all packages have been installed
b) packages are indeed importable; If the package import fails, the script should terminate with handsome error message highlighting the source (package) where import failed.
Our project is a mostly J2EE based development with the automatic functional and integration tests written in Python. The test environment is Linux nonetheless developers use Windows 7 (64-bit). We would like to be able to execute the functional tests on the developer machines as well (before comitting). Unfortunately the pexpect-windows-portability issue would leave us no choice but:
To do some serious refactoring on our test libraries to be able to use both winpexpect (or wexpect) and pexpect depending on the os settings.
Or to use cygwin. Guess what, with this second option we seem to have an issue :-) Using Python 2.7.5 on Cygwin64 installing the requests package results in error:
pip says it can not find a file after downloading and extracting the library
easy install doesn't throw an explicit error, but leaves everything in the temporary dir
after copying the files under the site-packages directory a simple import requests in python causes the interpreter to exit
Has anybody encountered this problem? With Cygwin-32 requests install smoothly. (however we have some other issues - see my next post ;-))
Thank you in advance: Joe, the public
Also ran into the same issues when trying to install requests, all the options on http://docs.python-requests.org/en/latest/user/install/#install did not work. I went to https://github.com/kennethreitz/requests and then clicked on "Download Zip" and I got requests-master.zip.
Update: This should be fixed in Cygwin.
This was a bug in CPython that has been fixed in their master branch. I've pushed a candidate package to the Python maintainer for Cygwin, but you might try this hotfix.
I downloaded from "https://github.com/requests/requests" and then i just ran the setup.py from the requests-master folder ( this was placed in cygwin folder). After that I went to cygwin terminal and then I ran python --> import requests. Voila it worked.
I've been learning Python for several months but now finding some problems with my 2.7 installation as I've looked into modules such as nltk.
However, when I want to list modules using help ("modules) I have the main error which I think explains the problem is:
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/distribute-0.6.28-py2.7.egg/setuptools/command/install_scripts.py:3: UserWarning: Module numpy was already imported from /Library/Python/2.7/site-packages/numpy-override/numpy/__init__.pyc, but /Library/Python/2.7/site-packages/numpy-1.8.0.dev_5c944b9_20120828-py2.7-macosx-10.8-x86_64.egg is being added to sys.path
from pkg_resources import Distribution, PathMetadata, ensure_directory
I also receive the following error to do with deprecated modules:
/Library/Python/2.7/site-packages/statsmodels-0.5.0-py2.7-macosx-10.8-intel.egg/scikits/statsmodels/__init__.py:2: UserWarning: scikits.statsmodels namespace is deprecated and will be removed in 0.5, please use statsmodels instead
I'm still trying to get to grips with paths. How can I avoid this issue in future?
You have installed packages under your operating system Python library. This is big no no. What you should have done is to create an isolated, disposable, Python environment with virtualenv tool:
http://opensourcehacker.com/2012/09/16/recommended-way-for-sudo-free-installation-of-python-software-with-virtualenv/
This way, when you upgrade your packages or need to get rid of them you can always reset the state of all of your Python packages by simply deleting the environment and creating new one.
Python packages installed via pip or easy_install commands are easy to install, but impossible to uninstall...
But when the damage has already happened you nede to manually try to clean up /Library/Python/2.7/site-packages/ by deleting files by and trying not to destroy your system Python in the process.