In python2.7, following the pympler example:
from anotherfile import somefunction, somecustomclass
from os import path, listdir
import pandas as pd
import gc
from pympler import tracker, muppy, summary
all_objects = muppy.get_objects()
print 'all objects: ', len(all_objects)
sum1 = summary.summarize(all_objects)
summary.print_(sum1)
This is the first code after the imports. It results in
/usr/bin/python2.7 /myprog.py
all objects: 98755
Traceback (most recent call last):
File "/myprog.py", line 12, in <module>
sum1 = summary.summarize(all_objects)
File "/usr/local/lib/python2.7/dist-packages/pympler/summary.py", line 131, in summarize
total_size[otype] = _getsizeof(o)
File "/usr/local/lib/python2.7/dist-packages/pandas/core/base.py", line 130, in __sizeof__
return super(self, PandasObject).__sizeof__()
TypeError: super() argument 1 must be type, not FrozenList
Process finished with exit code 1
I get the same error when I try to initialize a SummaryTracker object.
It looks like a bug in Pympler, but the fact that I can't find any mentions of it contradicts this. According to the official documentation, "Pympler is written entirely in Python, with no dependencies to external libraries. It has been tested with Python 2.5, 2.6, 2.7, 3.1, 3.2, 3.3, 3.4 on Linux, Windows and MacOS X." In fact, running only the following code with python 2.7 in a new python file does not produce any errors and works as expected:
from pympler import muppy, tracker
tr = tracker.SummaryTracker()
tr.print_diff()
So what am I missing?
it seems to be a problem in pandas library. I solved it by editing the library code. The track of the error indicates you which line is wrong:
File "/usr/local/lib/python2.7/dist-packages/pandas/core/base.py", line 130, in __sizeof__
return super(self, PandasObject).__sizeof__()
You just have to change the order of the parameters like this:
return super(PandasObject, self).__sizeof__()
I did it and I was able to run normally my program.
This is Pandas issue #12924. PandasObject.__sizeof__ had the arguments in the wrong order for the super call. The fix has been pulled, and it should be available in the next release. In the meantime, you could edit pandas/core/base.py to switch the argument order, or you could test for the bug's presence and monkey-patch the method with a corrected version.
Probably the code is supposed to work with Python3, only. In this question, there's the same error for python2 and if you look at the accepted response it says:
super() (without arguments) was introduced in python3:
super() -> same as super(class, )
so that would be the python2 equivalent for new-style classes:
super(CurrentClass, self)
You'll probably have to use python3 if you don't want to alter the library code.
Related
I was following some of the answers in Python ctypes: Python file object <-> C FILE * and I have encountered a problem. The following code:
from ctypes import *
libc = CDLL('libc.so.6')
cstdout = c_void_p.in_dll(libc, 'stdout')
Works without errors on my virtual machine running Ubuntu. However when I try to do the same thing on my MacbookPro running HighSierra, i.e:
from ctypes import *
libc = CDLL('libc.dylib')
cstdout = c_void_p.in_dll(libc, 'stdout')
I get the following error:
Traceback (most recent call last):
File "<stdin>", line 3, in <module>
ValueError: dlsym(0x7fff6bfc4728, stdout): symbol not found
I tried this using python2.7 and python3.5. With both of them I get no errors on Ubuntu and the previously mentioned error on OS X HighSierra.
The reason why I need this is because I am coding a Python wrapper for a C api, and in that C api there is a function that expects FILE* as an input argument, so I want to pass stdout to it.
Any help would be appreciated!
p.s. I would like to apologise for writing a new post, instead of commenting in the existing one, but I am a new user with only 35 reputation at the moment and I am not allowed to comment until I get at least 50 reputation.
From the stdio.h header on macOS:
#define stdout __stdoutp
Therefore your python code should do this instead:
cstdout = c_void_p.in_dll(libc, '__stdoutp')
I hope that the answer was helpful.
I was trying to debug an issue with abc.ABCMeta - in particular a subclass check that didn't work as expected and I wanted to start by simply adding a print to the __subclasscheck__ method (I know there are better ways to debug code, but pretend for the sake of this question that there's no alternative). However when starting Python afterwards Python crashes (like a segmentation fault) and I get this exception:
Fatal Python error: Py_Initialize: can't initialize sys standard streams
Traceback (most recent call last):
File "C:\...\lib\io.py", line 84, in <module>
File "C:\...\lib\abc.py", line 158, in register
File "C:\...\lib\abc.py", line 196, in __subclasscheck__
RuntimeError: lost sys.stdout
So it probebly wasn't a good idea to put the print in there. But where exactly does the exception come from? I only changed Python code, that shouldn't crash, right?
Does someone know where this exception is coming from and if/how I can avoid it but still put a print in the abc.ABCMeta.__subclasscheck__ method?
I'm using Windows 10, Python-3.5 (just in case it might be important).
This exception stems from the fact that CPython imports io, and, indirectly, abc.py during the initialization of the standard streams:
if (!(iomod = PyImport_ImportModule("io"))) {
goto error;
}
io imports the abc module and registers FileIO as a virtual subclass of RawIOBase, a couple of other classes for BufferedIOBase and others for TextIOBase. ABCMeta.register invokes __subclasscheck__ in the process.
As you understand, using print in __subclasscheck__ when sys.stdout isn't set-up is a big no-no; the initialization fails and you get back your error:
if (initstdio() < 0)
Py_FatalError(
"Py_NewInterpreter: can't initialize sys standard streams");
You can get around it by guarding it with a hasattr(sys, 'stdout'), sys has been initialized by this point while stdout hasn't (and, as such, won't exist in sys in the early initialization phase):
if hasattr(sys, 'stdout'):
print("Debug")
you should get good amount of output when firing Python up now.
I'm trying to write some VERY trivial thing in pycharm.
Problem:
sourceText = ""
with open("lang.txt", "rt") as sourceFile:
sourceText = sourceFile.readall()
print sourceText
when I enter "." after "sourceFile", I get popup that offers me "readall()" method. However, when I attempt to run the script, I get"
Traceback (most recent call last):
....languages/languages.py", line 4, in <module>
sourceText = sourceFile.readall()
AttributeError: 'file' object has no attribute 'readall'
The method is documented (I get popup, can get documentation for this method using Ctrl+Q) but it seems to be inaccessible.
I'm a bit confused.
I'd like to either:
Not receive any popups for inaccessible methods in pycharm.
Or figure out why I can't see it despite it being documented.
Advice?
I'm using windows 7 64 bit, and have two python 2.7.9 installations (32bit and 64bit), with 64bit being in path 1st. Pycharm is 4.0.5 community edition.
You are correct that readall is documented for the io module, but it's complaining about file, which does not have that method. You want the read() method to read all the data in the file in one large clump. You could also use readlines() which well return a list. I have the Pro 3.4 edition of PyCharm and it does not do this. I would report this as a bug to PyCharm.
According to this: http://code.activestate.com/lists/python-list/413540/, tokenize.generate_tokens should be used and not tokenize.tokenize.
This works perfectly fine in Python 2.6. But it does not work anymore in Python 3:
>>> a = list(tokenize.generate_tokens(io.BytesIO("1\n".encode()).readline))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.2/tokenize.py", line 439, in _tokenize
if line[pos] in '#\r\n': # skip comments or blank lines
However, also in Python 3, this works (and returns also the desired output):
a = list(tokenize.tokenize(io.BytesIO("1\n".encode()).readline))
According to the documentation, it seems like tokenize.tokenize is the new way to use this module: http://docs.python.org/py3k/library/tokenize.html. tokenize.generate_tokens isn't even documented anymore.
But, why is there still a generate_tokens function in this module, if it's not documented? I haven't found any PEP regarding this.
I'm trying to maintain a code base for Python 2.5-3.2, should I call generate_tokens for Python 2 and tokenize for Python 3? Aren't there any better ways?
generate_tokens seems to be really a strange thing in Python 3. It doesn't work like in Python 2. However, tokenize.tokenize behaves like the old Python 2 tokenize.generate_tokens. Therefore I wrote a little workaround:
import tokenize
if sys.hexversion >= 0x03000000d:
tokenize_func = tokenize.tokenize
else:
tokenize_func = tokenize.generate_tokens
Now I just use tokenize_func, which works without problems.
generate_tokens in python3 is undocumented but not uncommented. it's there for backward compatibility, so you can use it, but it's probably better to use the changed tokenize instead...
I found (after another question here on StackOverflow) this interesting library written in Python which goal is the grammar parsing.
http://code.google.com/p/modgrammar/
And I also found this tutorial regarding it:
http://packages.python.org/modgrammar/tutorial.html
So, after reading all the tutorial, I understood that this is what I'm looking for! I tried to write the first example in the tutorial:
from modgrammar import *
class MyGrammar (Grammar):
grammar = (LITERAL("Hello,"), LITERAL("world!"))
but I ran into this error:
Traceback (most recent call last):
File "test.py", line 1, in <module>
from modgrammar import *
File "/Users/tesi/Desktop/Prova/modgrammar/__init__.py", line 503
class Grammar(metaclass=GrammarClass):
^
SyntaxError: invalid syntax
The problem seems to be in the metaclass declaration. Might be I have to add a "compilation flag" when I call the python interpreter? Some good news?! :) Thx
class Grammar(metaclass=GrammarClass)
is using Python3 syntax. The equivalent Python2 syntax would be
class Grammar(object):
__metaclass__=GrammarClass
but since there may be lots of other Python3-isms, you may have to use Python3 to use modgrammar.
I've backport modgrammar to Python 2.6+. Backport will also work with Python 3.x. You can find it here: https://github.com/don-ramon/modgrammar-py2.
Yeah, this is Python3 library http://code.google.com/p/modgrammar/source/browse/setup.py#32
Python2x doesn't support class Grammar(metaclass=GrammarClass)