I want to view a pandas dataframe in Spyder's Variable Explorer.
When I click on the dataframe 'Name' in the Variable Explorer window, a series of files are executed (spyder_kernel.py, datapub.py, datapub.py, and serialize.py), which are part of the anaconda package. However, while executing serialize.py, the following message shows up:
serialize_object
buffers.insert(0, pickle.dumps(cobj, PICKLE_PROTOCOL))
OverflowError: cannot serialize a bytes object larger than 4 GiB
The dataframe is also not displayed. Any solution to this? Thanks.
(Spyder maintainer here) The problem you mentioned was fixed in Python 3.4 (according to this answer), which added a new Pickle protocol.
However, this is a bit tricky to solve for us because Spyder and the kernel that powers its IPython console run in different processes and both use Python. We'll address this in Spyder 4 (to be released this year), by detecting if both are running in Python 3.4+ to be able to use the right Pickle protocol.
Related
I'm running a python script and it used to work (it even does on my other laptop right now) but not on my current computer - I just get the error code:
Process finished with exit code -1073741515 (0xC0000135)
I don't get any other results - not even from "print" commands at the beginning of the file.
I haven't found anything specific to that.
I re-installed python (2.7.9), pygame (1.9.1) and even pycharm (tried 4.5 first, now with 5.0 - same result)
Does anyone know what that error code means? I couldn't find anything about it.
reinstall python – you don't have python33.dll in c:\WINDOWS\system32\
Maybe you have different python versions – look at folders in root of c:
If yes, then point to your version of python.exe in pyCharm > Settings > Project Interpreter
This may be due to another program locking that memory location.
Before you try more drastic measures, know that restarting the computer fixed the problem for me.
The error seems to be related to issues in native code (e.g. C/C++) and might be related to issues in loading some dll file. To know which file, the Windows "Event Viewer" is your friend.
The "Event Viewer" -> "Windows Logs" -> "Application" feed usually shows one or more rows for each crashed application and one of them should list the problematic filename.
Personally I observed similar issue with matplotlib on Python 3.9 (installed using Anaconda on Windows 10). I could see this kind of error when the code was executed using PyCharm. The same code just crashed with no visible errors when starting from Anaconda console.
The Event viewer then showed issue with freetype.dll. In this case, the problem was fixed by downgrading the packege from 2.11.0 to 2.10.4:
conda install freetype=2.10.4
Depending on the exact issue, I could imagine that removing pycache folder might help (like mentioned in other answers). Also, sometimes it might make sense to look at the PATH environment variable to actually understand why certain dll file is loaded from such a location.
I got this error in PyCharm after upgrading pyarrow to the most recent version -- 0.16 --- using pip, running the same code I had ran before this update which used pandas read_parquet specifying the engine="pyarrow" triggering use of this library. After uninstalling and installing the previous version with
pip install pyarrow=="0.15"
the problem resolved.
I encountered the same error when running .py in PyCharm on Windows. Inspired by https://thenewboston.com/forum/topic.php?id=10088, I uninstalled and reinstalled Python. When reinstalling, I checked the ADD PYTHON TO THE PATH Option. After recreating the virtual environment in PyCharm, the error was gone.
Update: On another Windows PC (64-bit), I encountered the same error where reinstalling Python was not enough. I tried two things.
Uninstalling 32-bit Python and installing 64-bit Python. Namely downloading python-3.6.4-amd64.exe instead of python-3.6.4.exe
Recreating the Pycharm project from github.
Between the two actions, the error was gone.
taskkill /F /IM python.exe
Try this if it just happened out of the bloom and it worked normally before that error occurred.
I solved it by disabling "PyQt compatible" checkbox in the Settings under Build, Execution, Deployment --> Python Debugger.
After that, it should debug properly.
Here is caused it and solution:
The computer had two physical processors. The code uses Numba for parallel computations. Upon disabling and re-enabling hyperthreading this issue started. To solve it clear pycache folder and run the program.
I encountered this error in my code as well, in my case the problem was sharing pickle which produced in Unix machine one a Windows one.
**I faced the same situation and the malfunction stems from trying to run both terminal processor and graphical user interfaces. When I clicked the button program stopped running **
Solution: I disabled terminal inputs and work with graphics.
For example:
if you do this
entry1=Entry(root,width=10).pack()
variable= str(input("Whats your name?"))
You will see the error.
if you delete either entry1 or variable, you will not see the problem.
If you're trying create a .svm use dlib probably the problem is in .xml, some image that you use is cause the problem. Try to create a new .xml putting out a image(any image) or remove that your .xml and test, try this even you find out the image problem.
For exemple, the image "treinamento3.jpg" was the problem. I removed this command line:
< image file='delirium\treinamento3.jpg'>
< box top='213' left='86' width='46' height='49'/ >
< box top='531' left='47' width='125' height='123'/ >
< /image >
from my .xml for solve my problem.
I'm working with python 3.7.6 on 64-bit Windows 10 (v 1909) and just updated anaconda navigator to v 1.1.10 and Spyder to 4.1.5. Trying to open a .npy file with "data = np.load('filename.npy', allow_pickle=True)" at IPython console (7.19.0). The data loads into the variable viewer, however i cannot view the contents. it simply shows "ndarray object of numpy module" as the Value, and attempting to open it further yields the error "Object arrays without shape are not supported"
I've read in previous documentation that support for this was supposed to come in Spyder v4. Also, view with the Object Explorer doesnt offer any additional information -- just shows some various built-in methods i can use on "data". The "Callable" column for the loaded "data" variable in Object Explorer is "False".
Hoping someone here can help me identify the problem.
I am running a python notebook in VS Code (see image). It runs fine but when I try to inspect a dataframe with the Data Viewer I get:
"Python package 'pandas' is required for viewing data."
The package is installed, otherwise, the code would not work and the data frames would not be present in the variable panel. When I click on "Install" I get: "Error: All data science packages require an interpreter be passed in"
I only have two environments, Anaconda and one created by VS code. I have tried selecting either one and nothing changes, code runs on both and I get the same errors on both.
Any ideas on how to fix this problem?
EDIT: The previous question Viewing data in the VSCode variable explorer requires pandas does not solve my issue. As mentioned above I have selected different environments without fixing it.
EDIT 2: Updating pandas did not fix it either. It was only solved by updating Anaconda, as suggested by Mrinal Roy.
This answer on a similar issue Viewing data in the VSCode variable explorer requires pandas mentioned it was fixed around a year ago.
It seems that the version.release string of pandas I was using includes characters that are not correctly parsed by the extension. They mentioned that it was going to be addressed yesterday in the Insider version. Looking forward to validating.
So, you probably have an older version of Anaconda or VS Code. Otherwise, what version of pandas do you have in both conda environments? VS Code Data Viewer requires pandas package 0.20 or later. Try upgrading it and related packages to the latest and check.
I updated Spyder to 4.0.1 which has a new object explorer, allowing to display arbitrary objects in a tree-like view as well as displaying metadata
Previous versions of Spyder enabled to view Pandas Dataframes in a simple table view when double clicking on the variable in the explorer.
Is there a simple way of accessing this nice old view in Spyder 4.0.1?
(Spyder maintainer here) You need to have Pandas installed in the same Python installation or environment where Spyder is installed. Else Spyder will use the Object explorer to open Dataframes, just as it does with any other kind of Python object.
As per documentation https://www.spyder-ide.org/blog/spyder-variable-explorer/ you can view the dataframe by right clicking on it and selecting the "View with the Object Explorer" option.
Edit: I don't have Spyder 4.0 installed, you can check if this works for you.
Just open up the terminal >> activate the environment that you're working in >> install pandas again. (If you're using conda, install it using conda)
if your like me and just messed up by trying different hotkeys or something just try saving the file and relaunching spyder. (please note this is incase you could see the dataframes in tabular form earlier and it suddenly started displaying it differently.)
I'm running a python script and it used to work (it even does on my other laptop right now) but not on my current computer - I just get the error code:
Process finished with exit code -1073741515 (0xC0000135)
I don't get any other results - not even from "print" commands at the beginning of the file.
I haven't found anything specific to that.
I re-installed python (2.7.9), pygame (1.9.1) and even pycharm (tried 4.5 first, now with 5.0 - same result)
Does anyone know what that error code means? I couldn't find anything about it.
reinstall python – you don't have python33.dll in c:\WINDOWS\system32\
Maybe you have different python versions – look at folders in root of c:
If yes, then point to your version of python.exe in pyCharm > Settings > Project Interpreter
This may be due to another program locking that memory location.
Before you try more drastic measures, know that restarting the computer fixed the problem for me.
The error seems to be related to issues in native code (e.g. C/C++) and might be related to issues in loading some dll file. To know which file, the Windows "Event Viewer" is your friend.
The "Event Viewer" -> "Windows Logs" -> "Application" feed usually shows one or more rows for each crashed application and one of them should list the problematic filename.
Personally I observed similar issue with matplotlib on Python 3.9 (installed using Anaconda on Windows 10). I could see this kind of error when the code was executed using PyCharm. The same code just crashed with no visible errors when starting from Anaconda console.
The Event viewer then showed issue with freetype.dll. In this case, the problem was fixed by downgrading the packege from 2.11.0 to 2.10.4:
conda install freetype=2.10.4
Depending on the exact issue, I could imagine that removing pycache folder might help (like mentioned in other answers). Also, sometimes it might make sense to look at the PATH environment variable to actually understand why certain dll file is loaded from such a location.
I got this error in PyCharm after upgrading pyarrow to the most recent version -- 0.16 --- using pip, running the same code I had ran before this update which used pandas read_parquet specifying the engine="pyarrow" triggering use of this library. After uninstalling and installing the previous version with
pip install pyarrow=="0.15"
the problem resolved.
I encountered the same error when running .py in PyCharm on Windows. Inspired by https://thenewboston.com/forum/topic.php?id=10088, I uninstalled and reinstalled Python. When reinstalling, I checked the ADD PYTHON TO THE PATH Option. After recreating the virtual environment in PyCharm, the error was gone.
Update: On another Windows PC (64-bit), I encountered the same error where reinstalling Python was not enough. I tried two things.
Uninstalling 32-bit Python and installing 64-bit Python. Namely downloading python-3.6.4-amd64.exe instead of python-3.6.4.exe
Recreating the Pycharm project from github.
Between the two actions, the error was gone.
taskkill /F /IM python.exe
Try this if it just happened out of the bloom and it worked normally before that error occurred.
I solved it by disabling "PyQt compatible" checkbox in the Settings under Build, Execution, Deployment --> Python Debugger.
After that, it should debug properly.
Here is caused it and solution:
The computer had two physical processors. The code uses Numba for parallel computations. Upon disabling and re-enabling hyperthreading this issue started. To solve it clear pycache folder and run the program.
I encountered this error in my code as well, in my case the problem was sharing pickle which produced in Unix machine one a Windows one.
**I faced the same situation and the malfunction stems from trying to run both terminal processor and graphical user interfaces. When I clicked the button program stopped running **
Solution: I disabled terminal inputs and work with graphics.
For example:
if you do this
entry1=Entry(root,width=10).pack()
variable= str(input("Whats your name?"))
You will see the error.
if you delete either entry1 or variable, you will not see the problem.
If you're trying create a .svm use dlib probably the problem is in .xml, some image that you use is cause the problem. Try to create a new .xml putting out a image(any image) or remove that your .xml and test, try this even you find out the image problem.
For exemple, the image "treinamento3.jpg" was the problem. I removed this command line:
< image file='delirium\treinamento3.jpg'>
< box top='213' left='86' width='46' height='49'/ >
< box top='531' left='47' width='125' height='123'/ >
< /image >
from my .xml for solve my problem.