How to fix Python ValueError:bad marshal data? - python

Running flexget Python script in Ubuntu, I get an error:
$ flexget series forget "Orange is the new black" s03e01
Traceback (most recent call last):
File "/usr/local/bin/flexget", line 7, in <module>
from flexget import main
File "/usr/local/lib/python2.7/dist-packages/flexget/__init__.py", line 11, in <module>
from flexget.manager import Manager
File "/usr/local/lib/python2.7/dist-packages/flexget/manager.py", line 21, in <module>
from sqlalchemy.ext.declarative import declarative_base
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/ext/declarative/__init__.py", line 8, in <module>
from .api import declarative_base, synonym_for, comparable_using, \
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/ext/declarative/api.py", line 11, in <module>
from ...orm import synonym as _orm_synonym, \
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/__init__.py", line 17, in <module>
from .mapper import (
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/mapper.py", line 27, in <module>
from . import properties
ValueError: bad marshal data (unknown type code)

If you get that error, the compiled version of the Python module (the .pyc file) is corrupt probably. Gentoo Linux provides python-updater, but in Debian the easier way to fix: just delete the .pyc file. If you don't know the pyc, just delete all of them (as root):
find /usr -name '*.pyc' -delete

There also appears to have been some sort of regression in setuptools with use with python 3.7. See for an example - https://github.com/pypa/setuptools/issues/1257
Forcing reinstallation of setuptools fixed this issue for me.
sudo pip3 install --upgrade --force-reinstall setuptools

Just delete
/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/properties.pyc
it is corrupt as the text indicates. You'll probably have to do so as root.
After that start (again as root) run python (2.7):
/usr/bin/python -c "import sqlalchemy.orm.properties"
to recreate this .pyc file.
If you don't recreate the .pyc file, your program starts slower than necessary as the .py file takes longer to load than the .pyc (and a normal user cannot write the .pyc file).

This can happen if you have Python 2.7 .pyc files and you try to load them using Python 3.5. In my case this was a third-party tarball that erroneously included pre-compiled Python 2.7 .pyc files along with the source code.

I get this error in Ubuntu 18.04 Raspberry Pi 3 when I trying update my system typing sudo apt-get update and solve this error just typing:
sudo find /usr -name '*.pyc' -delete
This is remove all .pyc file in my system. Now I typing again sudo apt-get update && sudo apt-get upgrade and I get my update without thie error marshal-data

I resolved a similar error by un-installing and re-installing the Python application I was using, and all dependencies, using the system package manager.
In my case I was using awscli on Debian 9 and the error was "ValueError: bad marshal data (set size out of range)".
I ran as root:
apt-get purge awscli
apt-get autoremove
apt-get install awscli
And then the error was fixed.
I could imagine cases where the broken package might not get removed (for example because it was marked as manually installed, or was a dependency of another application still installed), in those cases this action may not resolve the error. However I thought I should try this way before manually deleting .pyc files the system installed, and I got lucky.

I also got this problem in Windows environment(win 10).
I fixed it by going to the Settings and repairing Python 3.7 with its installer.
Everything works fine since then.
As far as I could recall, I had kept a dash server running when my computer went to hibernation.
Maybe the damage was done in the hibernating process somehow.

I solved this problem by the following procedure :
In the error code message, you can see
from sqlalchemy.ext.declarative import declarative_base cause this error.
So just pip uninstall sqlalchemy and pip install sqlalchemy, problem solved.

It may be because of the damage of the library. Try re-install the package.

I had the same error in a conda environment which traced back to importing the matplotlib package.
simply pip uninstall matplotlib and then pip install matplotlib solved the problem.

Related

Biopython - very basic questions about how to install

I'm trying to install Biopython but I've had some trouble. I've looked for an answer on the README, PDF manual and in other people's questions but they focus on more advanced problems while I'm still stuck at level 0.
I am a complete n00b so please, bear with me.
I have downloaded python3 and installed it on my Mac.
python3 --version
Python 3.7.3
pip3 --version
pip 19.2.3 from /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pip (python 3.8)
I have then gone and downloaded biopython-1.77.tar.gz and uncompressed on my desktop.
From the biopython-1.77 folder, I did
python3 setup.py build
which I think worked OK (things moved on the screen). It was followed by
sudo python3 setup.py install
Likewise, I think this too worked.
Now, to test if the installation worked and if I'm ready to use Biopython I tried
python3
[>>>import Bio
but I get the error:
/Users/user/Desktop/biopython-1.77/Bio/__init__.py:126: BiopythonWarning: You may be importing Biopython from inside the source tree. This is bad practice and might lead to downstream issues. In particular, you might encounter ImportErrors due to missing compiled C extensions. We recommend that you try running your code from outside the source tree. If you are outside the source tree then you have a setup.py file in an unexpected directory: /Users/user/Desktop/biopython-1.77
BiopythonWarning,
I then tried to do this from elsewhere and I changed directory to the Python3.8 folder and tried
python3
[>>> import Bio
but I get the error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'Bio'
I've also tried to specify the path to Bio within biopython-1.77 but I get a SyntaxError: invalid syntax on the first "/" within the path.
Any help to install Biopython will be much appreciated.

Getting module 'google.protobuf.descriptor_pool' has no attribute 'Default' in my python script

I am new to python and was using a python script written by someone else. I was running it fine in a different PC. Just had to install coupe of packages including pip3, google-cloud, google-cloud-bigquery and pandas.
Now when I installed the same packages on a different PC, I am unable to run the script. It is showed the following error first:
module = 'google.protobuf.descriptor_pb2' TypeError: expected bytes, Descriptor found
However, when In purged/re-installed/updated packages and also added protobuf3 and protobuf-py3 package, The error has been updated to the following message:
from google.cloud import bigquery
File "/home/mobeen/.local/lib/python3.6/site-packages/google/cloud/bigquery/__init__.py", line 35, in <module>
from google.cloud.bigquery.client import Client
File "/home/mobeen/.local/lib/python3.6/site-packages/google/cloud/bigquery/client.py", line 50, in <module>
import google.cloud._helpers
File "/home/mobeen/.local/lib/python3.6/site-packages/google/cloud/_helpers.py", line 33, in <module>
from google.protobuf import duration_pb2
File "/home/mobeen/.local/lib/python3.6/site-packages/google/protobuf/duration_pb2.py", line 8, in <module>
from google.protobuf import symbol_database as _symbol_database
File "/home/mobeen/.local/lib/python3.6/site-packages/google/protobuf/symbol_database.py", line 193, in <module>
_DEFAULT = SymbolDatabase(pool=descriptor_pool.Default())
AttributeError: module 'google.protobuf.descriptor_pool' has no attribute 'Default'
.Any help or leads in this will be appreciated
I solved the problem by uninstalling protobuf:
pip3 uninstall protobuf
pip3 uninstall python3-protobuf
NB: You should repeat this command until you get a message that there is no package named protobuf.
After that execute:
pip3 install protobuf
Install just the protobuf , don't install python3-protobuf
Hope this solution can help you.
I solved it by first executing twice
pip3 uninstall protobuf
The second time the terminal returned
Found existing installation: protobuf 3.6.1
Not uninstalling protobuf at /usr/lib/python3/dist-packages, outside environment /usr
Can't uninstall 'protobuf'. No files were found to uninstall.
Then I removed protobuf manually
sudo rm -r /usr/lib/python3/dist-packages/google/protobuf*
sudo rm -r /usr/lib/python3/dist-packages/protobuf*
And finally I executed
pip3 install --upgrade protobuf
And the problem was solved
Did you try this too?
"I solved the problem of showed Attribute Error: 'module' object has no attribute 'Default' when import tensorflow after installed by delete redundant protobuf file.
The reason is some google/protobuf/descriptor_pool.py do not have a 'Default' defined. This usually happened at old version of protobuf so I upgraded successfully, but problem not solved. And by checking PATH and search about 'google/protobuf', I found it existed in both "/usr/local/lib/python2.7/dist-packages/google/protobuf/" and "/usr/lib/python2.7/dist-packages/google/protobuf/". Previous one have attribute 'Default' but the second one not.
I tried import google.protobuf and google.protobuf.file, It shows '/usr/lib/python2.7/dist-packages/google/protobuf/init.pyc'. I deleted /usr/lib/python2.7/dist-packages/google/protobuf and tried to import tensorflow, worked."
Actually I've ran into a similar case, we had 2 packages installed protobuf and python3-protobuf. I actually dont know the root cause for this but apparently when you do that:
pip install protobuf
pip install python3-protobuf
that error you described is happening, looks like it gives you some different version, like those two packages have overlapping files, and they override each other or something.
the solution for me was simply to reverse the installation order (make sure to uninstall them both first):
pip install python3-protobuf
pip install protobuf
or just
pip install python3-protobuf protobuf
hope this helps anyone here.
The reason can be that the interpreter that you are using to run the python programm uses the previous version of google.protobuf
You can assure it to run in the interpreter
>>> import google.protobuf
>>> print google.protobuf.__version__
Then compare it running in the terminal
$pip show protobuf
If the versions are different that's the reason
So I suggest deleting this package right from the python interpreter console
>>> pip uninstall protobuf -y
After you can even install the package right from the python console
>>> pip install protobuf
After that you good to go ✌️

Python Error : No module named pkg_resources

I would like to use Python3.7 on MacOS
I already Python 2.7 version.
I created an alias on .bash_profile, alias python="/usr/local/bin/python3.7" then source ~/.bash_profile.
So I deleted Python2.7 to /usr/local/lib/
Now, when I try to execute pip install PySide2, I have an error :
Traceback (most recent call last):
File "/usr/local/bin/pip", line 6, in <module>
from pkg_resources import load_entry_point
ImportError: No module named pkg_resources
I think this error has happened since I deleted Python2.7
Someone can help me to resolve my error ?
Thank you !
pyinstaller 3.6 is incompatible with setuptools 45.1.0 on python 3.7.*, should be downgraded to 45.0.0
pip install setuptools==45.0.0
can also be fixed with passing/adding hidden_imports 'pkg_resources.py2_warn' to pyinstaller spec
Issue and solutions are tracked here: https://github.com/pypa/setuptools/issues/1963
I found solution from here.
In my case, I open hook-pkg_resources.py file from the following directory:
~/.local/lib/python3.6/site-packages/PyInstaller/hooks/
After that I added this line of code:
hiddenimports.append('pkg_resources.py2_warn')
between these two lines of code:
hiddenimports = collect_submodules('pkg_resources._vendor')
and
excludedimports = ['__main__']
After that, I ran PyInstaller again and the resulted executable worked like charm.
This is caused because of a broken setuptools package, you just need to reinstall it.
For most operating systems: pip install setuptools
Linux: apt-get install python-setuptools or yum install python-setuptools
Stumbled uppon this answer first on google when searching for this error code, so for future reference ill just leave a link to this issue that solved my problem:
https://stackoverflow.com/a/59979390/10565375
tldr:
pyinstaller --hidden-import=pkg_resources.py2_warn example.py

Python finds glib, but not _glib

I try to import glib in Python (2.7) and get the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/share/pyshared/glib/__init__.py", line 22, in <module>
from glib._glib import *
ImportError: No module named _glib
Seems as if glib is found, while _glib is not found.
Has anybody a hint for solving this problem? I'm running Ubuntu and installed python2.7, python-gobject, python-gda, dbus from Ubuntu's package manager.
I had this issue.
You can try moving the glib and related folders out of the python2.7 install locations e.g. on ubuntu /usr/shared/pyshared in case these have been installed wrongly.
Then remove and reinstalling the related packages with the platform package manager ...
sudo apt-get remove
followed by sudo apt-get install
for ...
python-gobject python-gtk libglib2.0-dev python-cario python-gi
since if they are present in pyshared I found that they were not probably reinstalled
I run ubuntu 10.04, with its native python 2.6.6 and GCC 4.4.5, and an "import glib" in python appears to work.
I installed "python-numpy", "python-matplotlib" and "python-pyfits" with the synaptic package manager and nothing else, in particular nothing installed from a more recent version found on the package's site because previous experience learned me this was usually a very bad idea.
But I have NO /usr/share/pyshared/glib/ folder and do not know how you get it.
Is this folder on your path or PYTHONPATH ?
If yes, try to remove it from the path and retry the "import glib", it is possible that you get some second installation of glib in that place (installed - and maybe needed - by another package).
glib is an interface to the c compiler, so it is used in several places. What is your GCC version ? GTK also makes intensive use of glib. Did you installed "GTK" and "pygtk" from the synaptic or did you did it on the hard way, with all separated little packes to be compiled ? If yes, try to install it from the synaptic.
I mostly think this comes from some installation mix of some python extension somewhere. The radical solution, of course, is to reformat the computer with a fresh O.S., but it's probably something you do not want, so I should try to uninstall all packages python uses, and re-install them from the synaptic.
Hope this helps...

Why is Python easy_install not working on my Mac?

I have a Mac running Python 2.6. When I try to use easy_install I get this message:
/usr/bin/easy_install-2.6:7: UserWarning: Module pkg_resources was already imported from /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/pkg_resources.pyc, but /Library/Python/2.6/site-packages is being added to sys.path
from pkg_resources import load_entry_point
/usr/bin/easy_install-2.6:7: UserWarning: Module site was already imported from /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site.pyc, but /Library/Python/2.6/site-packages is being added to sys.path
from pkg_resources import load_entry_point
Traceback (most recent call last):
File "/usr/bin/easy_install-2.6", line 10, in <module>
load_entry_point('setuptools==0.6c9', 'console_scripts', 'easy_install')()
File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/pkg_resources.py", line 271, in load_entry_point
return False
File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/pkg_resources.py", line 2173, in load_entry_point
deps = []
ImportError: Entry point ('console_scripts', 'easy_install') not found
I am very confused by this and I want to know how I can fix this. Previously I didn't get this message, then after installing pip and uninstalling it, easy_install no longer works. I was wondering how I could fix this, or restore the default Python or easy_install setting on Mac.
Check your /usr/bin and /usr/local/bin for easy_install installations and remove any old script:
sudo rm -f /usr/bin/easy_install*
sudo rm -f /usr/local/bin/easy_install*
Download and run distribute:
curl -O https://svn.apache.org/repos/asf/oodt/tools/oodtsite.publisher/trunk/distribute_setup.py
sudo python distribute_setup.py
sudo rm distribute_setup.py
Try again, and enjoy. E.g.:
sudo easy_install pip
I suspect the easiest way you can get easy_install working again is to install distribute, which is an improved version of distutils that bundles it's own version of easy_install. Installation is simple:
curl -O http://python-distribute.org/distribute_setup.py
/usr/bin/python2.6 distribute_setup.py
I had the same problem just after installing the new Operating System (Lion OSX).
After install python and execute it
sudo easy_install ipython
ipython
I got the following error:
Traceback (most recent call last):
File "/usr/local/bin/ipython", line 8, in <module>
load_entry_point('ipython==0.10.2', 'console_scripts', 'ipython')()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 318, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 2220, in load_entry_point
raise ImportError("Entry point %r not found" % ((group,name),))
ImportError: Entry point ('console_scripts', 'ipython') not found
Then I realize it was a permission problem. If I execute the ipython script as sudo it worked.
sudo ipython
So I changed the ownership from root to my user () of the folder /Library/Python/2.7/site-packages/
sudo chown -R <your_user>:wheel /Library/Python/2.7/site-packages/
Use change ownership only if you don't have several accounts that uses python.
I hope it work for you.
Best
Your setuptools installation is broken. The easy_install-2.6 script in your /usr/bin directory, is only a wrapper that loads the actual easy_install module in your Python installation, and the latter is missing. Presumably uninstalling pip either broke the pointer to it (usually in the form of a .pth file in your site-packages directory) or removed the whole package but didn't remove the /usr/bin/easy_install-2.6 wrapper script.
Your best bet is to re-install setuptools from scratch again, by following the instructions on the setuptools PyPI page.
I had the same problem, I suspect caused by my installation of Homebrew which apparently installs its own version of easy_install blindly and screws up whatever you're currently running (enough times for homebrew to recognize it as a problem). I found my solution here Upgraded Python on Snowleopard Using Homebrew now pip and easy_install don't work
While I'm actually on Lion, it still fixed my problem. You just download ez_setup.py from http://pypi.python.org/pypi/ez_setup - run ez_setup.py through terminal and you're off to the races.
I had a similar error message after following similar steps. I suspect it's because I tried to install the distribute module, and it tried to install a new version of setuptools which
didn't quite work because it failed to correctly override the Mac OS X supplied setuptools. But that's only a suspicion.
I fixed my problem by doing the following. I found that the directory /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python
had 4 files with .OLD in the name. When I did a compare with similarly named files without the .OLD from when things worked (via Time Machine) I found that they were the same except for the name. So I did the following renames:
setuptools.OLD.1305936263.57 ->
setuptools
setuptools-0.6c9-py2.6.egg-info.OLD.1305936263.57 -> setuptools-0.6c9-py2.6.egg-info pkg_resources.py.OLD.1305936263.57
-> pkg_resources site.py.OLD.1305936263.59 -> site.py
I had to use the command line and sudo mv because the files are owned by root. For example: sudo mv setuptools.OLD.1305936263.57 setuptools. You'll have to enter your password.
The other thing I did was clean the /Library/Python/2.6/site-packages directory. I used Time Machine to set it back to its pre-messed up state. If you don't have Time Machine, then you should be able to recover if you delete any files or folders beginning with distribute, easy-install, pip, pkg_resources, setuptools and site.
If that doesn't do the trick, uninstall any modules you can from that directory. For example use pip uninstall if you installed modules with pip. The reason to do this is that installation can copy files into usr/local/bin and other places that pip will remove for you.
Next it's time to remove any remaining modules. Rename the folder (in case there's something you want later) and make a new empty one.
Reinstall the modules you want and with a little luck you're back in business...
Martijn Pieter's answer is correct.
Go to http://pypi.python.org/pypi/setuptools#downloads
Download setuptools-0.6c11-py2.7.egg
From your downloads directory run: sudo ./setuptools-0.6c11-py2.7.egg
Done :-)

Categories

Resources