Python keeps getting corrupted and requiring reinstallation - python

I'm on windows 10 and using Miniconda 64bit version 4.9.2. For some reason my python installation keeps getting corrupted after running scripts, sometimes it will happen randomly, other times it seems like specific scripts cause it to become corrupt. This doesn't happen to my other teammates though who are running similar environments.
What will happen is after trying to run a specific script or command (jupyter lab for example) that worked previously that day, I will randomly get an error with something like the following:
ValueError: bad marshal data (unknown type code)
Usually it seems to involve a library I'm using in the stack trace, but I don't think it's a specific library because it will be different ones in the stack trace from time to time, I'm guessing whatever got corrupted at that moment. I've tried deleting the pycache files for the specific libraries that error out, but that shows me what I think is the root of the problem, files are getting corrupted and overwritten with random characters. See an example error stacktrace below after deleting .pyc files and attempting to run again:
Traceback (most recent call last):
File "c:\users\user\miniconda3\envs\environment\lib\runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\users\user\miniconda3\envs\environment\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "c:\users\user\miniconda3\envs\environment\Scripts\jupyter.exe\__main__.py", line 4, in <module>
File "c:\users\user\miniconda3\envs\environment\lib\site-packages\jupyter_core\command.py", line 18, in <module>
from . import paths
File "c:\users\user\miniconda3\envs\environment\lib\site-packages\jupyter_core\paths.py", line 15, in <module>
import tempfile
File "c:\users\user\miniconda3\envs\environment\lib\tempfile.py", line 832
self._rmtree(ȕ&�n�gH�6�
^
SyntaxError: invalid character in identifier
When I look in the file after getting this error, it literally has been cut off and random characters like those seen above have been inserted. This never happens in the middle of scripts though, it's only after something has run and I try to run it again or I try to run something else afterwards. Not sure what's going on I'm not super versed in python. It's getting to the point where I have to reinstall multiple times a day.
Is it just my system? I'm using pycharm as my IDE, would that have anything to do with it? I'm gonna try different versions of miniconda but not sure if that will do much. I haven't found much support for this issue through my research.

I have the same problem using Miniforge (python 3.7.10, conda 4.10.3, windows 10).
Last line of the .py files of some modules gets randomly corrupted.
I noticed that in addition to lib content, also all the .py files in pkgs subfolders are already corrupted, but the compressed files in tar packages are fine.
I found this temporary workaround:
delete the cached files in lib\brokenmodule\__pycache__
delete the source folder in pkgs\brokenmodule-version-build\
run conda update brokenmodule --force-reinstall

Related

Coursera-dl not downloading classes command prompt Windows

I have been trying to install some lectures from coursera using the coursera-dl command, installed via :
pip install coursera-dl
I also did the coursera-dl.conf file, which includes my username, password, download preferences and cauth from Coursera.org.
I made sure to be logged in on coursera (in ALL of my browsers!!!)
I carefully write the classe's name, i.e. what follows learn/ in the url.
But I encountered errors. I decided to downgrade Python from 3.9.5 to 3.8. I also changed path, in environment variables of advanced settings (control panel) to where the coursera-dl.exe file is.
In addition, I have created the folder for the downloads, have the .conf file in there, and am doing the commands there (it managed to create a coursera-dl folder too, after I git cloned I think).
Anyway, I now get the error below after doing this command:
coursera-dl classical-composition
Can anyone help me please? Thank you
Traceback (most recent call last):
File "c:\users\leo\anaconda3\envs\spyder-env\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\users\leo\anaconda3\envs\spyder-env\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Users\Leo\Anaconda3\envs\spyder-env\Scripts\coursera-dl.exe\__main__.py", line 4, in <module>
File "c:\users\leo\anaconda3\envs\spyder-env\lib\site-packages\coursera\coursera_dl.py", line 61, in <module>
from .cookies import (
File "c:\users\leo\anaconda3\envs\spyder-env\lib\site-packages\coursera\cookies.py", line 9, in <module>
import ssl
File "c:\users\leo\anaconda3\envs\spyder-env\lib\ssl.py", line 98, in <module>
import _ssl # if we can't import it, let the error propagate
ImportError: DLL load failed while importing _ssl: The specified module could not be found.
this works: Make sure to install Coursera Authentication helper for Firefox and add this to your CAUTH (I noticed this sequence changes everyday and has to be updated in the coursera-dl.conf), login Coursera and you should be ready to download all the classes you wish! Make sure to go to the classe's link when part of a specialisation.
If you need more help, you can read my journey below:
EDIT
This has worked for many classes so far and I am not getting any errors. Using Anaconda 3 command promt. Here is what I did:
In environment variables, I added new paths, stated from this post https://stackoverflow.com/a/54897379/16367595
D:\Anaconda3
D:\Anaconda3\Scripts
D:\Anaconda3\Library\bin
Make sure to check what yours looks like, mine was in the C: drive, for instance.
Then I reinstalled:
python -m pip install coursera-dl
AND INTERESTINGLY this has actually only worked for 1 course, i.e. learn to write for screenplay etc initially. I then tried it again when I was getting the error below for another class, and it worked.
AttributeError: 'HTMLParser' object has no attribute 'unescape'
The following day, I was getting this error for other classes:
403 client error, Forbidden for URL
So finally, I made sure I was logged in in all of my browsers (not sure if this contributed) however, I think the solution to the above error comes from updating the CAUTH section of the coursera-dl.conf file. The successful cases above worked with the Google Chrome CAUTH cookies from coursera.org. The following day worked when I installed the Coursera Authentication extension for Firefox and replaced the older CAUTH with the one provided with the extension.
This is working well while using anaconda3 prompt, getting all desired classes downloaded.

VMTK custom script producing "No module named: vmtk.custom" error

I'm working with a module called VMTK to build scripts for analyzing vascular models. They have a tutorial for writing these scripts so they're easily identifiable by the system allowing you to pipe scripts together known as PypeS. I've followed the tutorial exactly as shown and end up with the same "No module named: vmtk.name_of_script" error.
What I've tried:
Putting the script in the same directory as all the preloaded scripts provided by the module so __init__.py could catch it
Copy-pasting the code from the website into my editor so I'm sure there are no errors in what I've written
Submitting the question directly onto the forum meant for questions about VMTK, with no responses for the past week and a half
Changing my Python interpreter to the one within the VMTK directory
The tutorial says you can put the custom script anywhere in your filesystem, but the issue arises no matter what. I'm currently using Python 3.6.10 through Anaconda and a binary install of VMTK (as opposed to building from source) on MacOS High Sierra.
I really don't want to have to build from source, as I only need the preloaded scripts and ability to write new scripts, as opposed to using the C++ files that the scripts are built from. I've been stuck for so long and am not at all sure what the issue is. The closest thing I found was an SO question addressing this problem, but the OP simply said the problem had been resolved and provided no other information.
Here's the aforementioned code:
#!/usr/bin/env python
import sys
from vmtk import pypes
from vmtk import vmtkscripts
customscript = 'customScript'
class customScript(pypes.pypeScript):
def __init__(self):
pypes.pypeScript.__init__(self)
def Execute(self):
pass
if __name__=='__main__':
main = pypes.pypeMain()
main.Arguments = sys.argv
main.Execute()
As per the tutorial, I've also run chmod u+x customscript.py to change the permissions of my file so I can execute it. I get stuck at exactly the point where the script should do the most basic of operations within VMTK. Any ideas?
Edit: Here's the exact error:
$ ./customscript.py
No module named 'vmtk.customscript'
Traceback (most recent call last):
File "/vmtk/lib/python3.6/site-packages/vmtk/pype.py", line 290, in Execute
module = importlib.import_module('vmtk.'+scriptName)
File "/vmtk/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 978, in _gcd_import
File "<frozen importlib._bootstrap>", line 961, in _find_and_load
File "<frozen importlib._bootstrap>", line 948, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'vmtk.customscript'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./customscript.py", line 21, in <module>
main.Execute()
File "/vmtk/lib/python3.6/site-packages/vmtk/pypescript.py", line 688, in Execute
pipe.Execute()
File "/vmtk/lib/python3.6/site-packages/vmtk/pype.py", line 298, in Execute
self.PrintError(str(e))
File "/vmtk/lib/python3.6/site-packages/vmtk/pype.py", line 102, in PrintError
raise RuntimeError(errorMessage)
RuntimeError: No module named 'vmtk.customscript'
I want to backtrace the issue but I don't think editing any of the preloaded files should be necessary.
The solution I found after a week of trying and searching:
git clone the vmtk repo onto your computer and follow the VMTK download instructions on how to build it from source. It will absolutely feel like a headache and be a long process; just wait it out.
If you don't have cmake installed on your computer, they offer a GUI version on their website. I'm not sure how it would work for Windows, but it definitely works for Mac.
Once you've built it, make sure that each terminal instance of VMTK you set source vmtk_env.sh from the path where that file is. This will allow you to use the VMTK environment.
After writing the custom script, making sure to follow all the directions in the tutorial (assuming this is the first time), make sure to save it in the "site-packages" folder in the build folder. The path for me is "vmtk-build/lib/python-2.7/site-packages/vmtk/". I'm currently looking to update the version of Python it's using, but that's not the point.
Once that's done, navigate to that folder from within terminal and run chmod u+x name_of_script.py (if you're in a Unix-based operating system). This will allow you to execute the file with ./name_of_script.py.
I believe it was due to the version of Python VMTK is using, but I had two "No module named" errors come up, each one a separate time: one for tkinter and one for joblib.
The tkinter issue is one that can be resolved by going into the file where the import is, I believe vmtkscripts.py, and changing all instances of tkinter to Tkinter (capital T). This is because they changed the name of the module from Python 2 to 3, so in newer versions of Python it would be tkinter. That fixed the first error.
When I attempted to fix the joblib issue, I saw that the other version of Python available on my system already had it, thus I couldn't install it through pip. Instead, you can do python-2.7 -m pip install joblib to install it for a specific version of Python. Although I did uninstall joblib as a whole from my computer before doing that.
That should fix everything. The next time I attempted to run my script, it ran without any errors.
Extra: I wasn't using any Anaconda environment for this. Might go back and try to set that up, but I'm just glad I got it to work. Additionally, I deleted the binary install of VMTK from my computer. No need to keep it around if you're just going to build it from source. You can run VMTK through the terminal after setting source vmtk_env.sh by running the vmtk command.

Pycharm Startup will not connect to console

I am trying to start up pycharm and am getting the following error while connecting to the console:
C:\Users\i_am_yohan\Anaconda3\pythonw.exe "C:\Program Files\JetBrains\PyCharm Community Edition 2019.3\plugins\python-ce\helpers\pydev\pydevconsole.py" --mode=client --port=52061
Failed to import the site module
Traceback (most recent call last):
File "C:\Users\i_am_yohan\Anaconda3\envs\Base\Lib\site.py", line 579, in <module>
main()
File "C:\Users\i_am_yohan\Anaconda3\envs\Base\Lib\site.py", line 566, in main
known_paths = addsitepackages(known_paths)
File "C:\Users\i_am_yohan\Anaconda3\envs\Base\Lib\site.py", line 349, in addsitepackages
addsitedir(sitedir, known_paths)
File "C:\Users\i_am_yohan\Anaconda3\envs\Base\Lib\site.py", line 207, in addsitedir
addpackage(sitedir, name, known_paths)
File "C:\Users\i_am_yohan\Anaconda3\envs\Base\Lib\site.py", line 159, in addpackage
f = open(fullname, "r")
File "C:\Users\i_am_yohan\Anaconda3\envs\Base\Lib\_bootlocale.py", line 12, in getpreferredencoding
if sys.flags.utf8_mode:
AttributeError: 'sys.flags' object has no attribute 'utf8_mode'
Process finished with exit code 1
I am making a transition from spyder to pycharm and am currently experiencing a lot of problems most notable this one. I have set up the correct project interpreter.
I get the same error, after I uninstalled Python 3.6, which I did after installing Python 3.7. I was already using Python 3.7 through Pycharm successfully for a while, but realized after some time that I had Python 3.6 still installed, without using it. Doing the following fixed this for me.
Before trying anything else, you might want to check that you are using the expected Python interpreter for what you are trying to execute.
I don't know what exactly has changed that fixed this, and I wouldn't necessarily recommend this solution, since my understanding of what really happened is very limited. But it doesn't seem to have broken my project, and so maybe someone is helped by this.
I cut and pasted my .idea file from my project root folder. Note that I cut and paste it, I didn't delete it. Removing this folder makes you lose a many, if not all, of your Pycharm project's settings. So don't delete this folder unless you want to lose your project settings, like your run/debug configurations.
Then I restarted Pycharm, and it automically started to create a new .idea folder (which can be very time consuming). I closed Pycharm again, since I realized I didn't want to go through with this, I had too many settings that I didn't want to lose, and so I removed the newly created .idea folder, and put back the original .idea folder.
After starting up Pycharm again, all the time in the same project folder, Pycharm did have to do some indexing, but then it worked. The project I am working on is a Django project, and I have a run configuration for Django. The error was thrown when I wanted to run the Django configuration (Maybe I should have checked the interpreter is was set to before doing all this). The only thing I was required to do to after the swapping back and forth of the .idea folder was change the Python interpreter for the Django configuration, it seemed to be reset.
I have figured this out. I should have connected to:
C:\Users\i_am_yohan\Anaconda3\python.exe
rather than:
C:\Users\i_am_yohan\Anaconda3\pythonw.exe
I got confused because I was moving from Spyder to Pycharm and Spyder is connected to pythonw.exe rather than python.exe
This happends when python console wants to use Python 3.7 interpreter to connect to a project build based on previous version of python. Here is a quick solution if you have installed Python 3.7 correctlly.
Go to your Pycharm, find 'Switch project interpreter' and change the interpreter into the Python 3.7 version.

How to use Vpython (visual) from an IDE

I'm very new to python and I don't entirely understand versions and libraries, so there may be a simple solution to my problem.
I am working on a project in which it is vital that we visually display our simulation. I installed Vpython on Mac OSX 10.8.4 and so far visualizations work when running the program from the command line. However I'd like to use an IDE (Eclipse, Sublime Text, Spyder. Ideally Spyder) but whenever from visual import * shows up, I get the following error:
>>> runfile(r'/Users/robinnewhouse/code/Spyder/astro/maintest.py', wdir=r'/Users/robinnewhouse/code/Spyder/astro')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Applications/Spyder.app/Contents/Resources/lib/python2.7/spyderlib/widgets/externalshell/sitecustomize.py", line 523, in runfile
execfile(filename, namespace)
File "/Users/robinnewhouse/code/Spyder/astro/maintest.py", line 7, in <module>
from visual import *
ImportError: No module named visual
So I tried using Spyder's PYTHONPATH manager to add the path to the package. I got the same error.
Then in a last ditch effort, I tried to manually add the "visual" package to Spyder's directory of included packages (/Applications/Spyder.app/Contents/Resources/lib/python2.7/) and I got a different (albeit worse) error: Fatal Python error: PyThreadState_Get: no current thread
I have no idea how to deal with this issue, but the most frustrating part is that if I run the same file in command line, it works fine. (speaking of which, if it helps, the version of python that comes up when I use which python is /Library/Frameworks/Python.framework/Versions/2.7/bin/python
Thanks for any advice you can offer.
(Spyder dev here) The right solution is this: In Spyder you need to go
Tools > Preferences > Console > Advanced Settings > Python Executable
and then change the file that appears there to be
/usr/bin/python
However, before doing this you need to reinstall Spyder again, because you added the visual module to it and now it's broken.

Weird issue with platypus (makes .app packages from scripts)

I wrote myself a small Python script that I want to use to automatically do things with certain types of files; as such, I want to create an .app out of it so that I can set certain files to be opened with it automatically.
So I looked around and found Platypus which seems to do what I need.
However, weirdly it doesn't work. Specifically, it does not seem to be finding the right python interpreter. I set it up as follows:
I.e., the script type is env so it should just read the top line of the file like the shell does.
In magic.py, the top line is #!/usr/bin/env python2.7.
Now, when I run the shell script on the command line (that is, ~/devel/magic.py whatever), everything works fine. But when I run the app, it errors with:
Traceback (most recent call last):
File "/Users/jan/Dropbox/devel/Magic.app/Contents/Resources/script", line 8, in <module>
from bencode import *
ImportError: No module named bencode
The same import works just fine when running it from the command line, so I'm thinking it's using the wrong interpreter somehow. How can I fix or debug this?
You are trying to import from bencode module but you didn't add it in the application's bundled resources. Either drag it to the list of included files and export again or just copy it to the resources folder in the package's contents.

Categories

Resources