py2neo - neo4j.GraphDatabaseService(db_string) crashes python; no error-log - python

the last couple of days I installed Python 2.7.3 and Neo4J community edition 1.8.M01. I managed to get the embedded python bindings to work, but as I need the py2neo REST bindings I've installed them like described at http://py2neo.org/. Moreover I can't download directly from git due to a "Permission denied (publickey)" error so I took the available py2neo-1.2.6.tar.gz version from the download section.
While the installation itself was not the problem, I can't get the example to work as on calling neo4j.GraphDatabaseService('http://localhost:7474/db/data') python crashes without any error message - Win7 64bit only pops up a message that the applicationn does not respond. Java, Python and Neo4J are all running on 64 bit basis and the server is accessible on http://localhost:7474. I even tried to force an output as described here: Catching a python app before it exits - but still no stacktrace or error-log.
I've installed everything from scratch or via the executables provided at www.lfd.uci.edu/~gohlke/pythonlibs/ several times now but nothing managed to get this example to work.
I have installed both tornado 2.2.1 and pycurl 7.23.1. pycurl.version_info() reveals: (3, '7.23.1', 464641, 'Windows', 28, 'OpenSSL/0.9.8s', 0, '1.2.5', ('gopher', 'http', 'https', 'imap', 'imaps', 'pop3', 'pop3s', 'rtsp', 'smtp', 'smtps'), None, 0, None) - moreover
c = pycurl.Curl()
c.setopt(c.URL, 'http://stackoverflow.com')
c.perform()
returns the content of the startpage.
I've followed the stacktrace via print-messages into tornado.IOLoop.start() and there into _run_callback() where it actually executes callback() and crashes. Not shure if the callback-function defined inside of tornado.HTTPClient.fetch() should be called here - printing the callback results in <tornado.stack_context._StackContextWrapper object at ...>
Any suggestion on how to fix this issue?
Thanks in advance,
Roman
edit: corrected port as of a typo
edit2: after a longer debug-session which narrowed the point of failure a bit, Nigel provided me with a way to deal with my issues by exchanging
self._http = http or httpclient.HTTPClient(curl_httpclient.CurlAsyncHTTPClient)
with
self._http = http or httpclient.HTTPClient()
in line 55 of rest.py. This is a workaround but does not solve the problem in the back of tornado/pycurl. The windows management console declares pycurl.pyd as the reason for the crash and as some of the nodes (after a seldom functioning intialization of the GraphDatabaseService) get stored within Neo4J and the Debug-Output below isn't shown anymore, the crash must occur between the send request and the return to the main application. I currently believe that either the Selection poll, which I fall back on Windows, is the reason for the crash or maybe the curl-handle gets shared between different threads - which should not happen (http://curl.haxx.se/libcurl/c/libcurl-tutorial.html) - and is somehow the most comprehensible reason imo

Sorry to hear that you're having issues with py2neo. I haven't carried out any testing under Windows since I only run Linux so I'm unsure whether there are any general incompatibilities there. I am also aware that error reporting is less than it should be which has been limited by the amount of time I've had to work on the project.
That said, I notice that you are running on port 4747 instead of the default 7474 - or this this a typo? Have you tried your short cURL test against the root database URI directly?
You seem to have covered all the bases looking at the layers involved so I'm unsure what else to look at here. I have considered adding an option to be able to switch between the curl_httpclient and the simple_httpclient - this may give an alternative to try. I will try to get something put up over the next few days.
Nige

Related

Ride.py shows Calling method 'start_keyword' of listener XXX failed

I'm trying to run an automated test in Ride.py. This test works on my colleague's computer but for some reason does not work on mine. The test starts but at a certain point i get the following error:
[ ERROR ] Calling method 'start_keyword' of listener 'C:\Python27\lib\site-packages\robotide\contrib\testrunner\TestRunnerAgent.py' failed: IndexError: list index out of range
The interesting part is that this error occurs on the same spot ever time, but with a different test it happens at a different time.
I tried to google several things and nothing worked. One solution suggested there was a '#' commented somewhere and this caused the crash. I looked but I don't see a '#' commented anywhere.
Another suggestion lead me to believe my testrunneragent.py file must have been installed wrong. I went online to find the file and replaced it. This did not work either (reran the test before and after a restart of ride)
We tried to re-import the test files thinking perhaps something went wrong there. This did not help either.
Googling juts the last part (IndexError: list index out of range) gave me the suggestion it does not recognize all the lines of code in the back-end file. I would have no clue how to solve this as im not a major coder.
One difference between me and my colleague could be the versions. I downloaded python version 2.7.16 and ride 1.7.3.1. My colleague uses an older version of both python and RIDE. Perhaps the problem could be here?
https://paste.fedoraproject.org/paste/TLekH3az0m4wuUyM8C2RYw
I expect the test will run without failing (it is a happy flow) I have included some screenshots with code in the previous segment that might help
downgraded to the same version of Ride.py
This seems to have fixed the issue

TIS/TSM non-main thread error; pygame script triggered by hotkey (rumps, pygame, keyboard)

I'm writing a python app, whose main purpose is to run a minigame (using the 'pygame' library), whenever I use a hotkey (which currently uses the 'keyboard' library). I want this hotkey to be recognized universally, so I'm packaging the whole thing as a status bar app (using the 'rumps' library).
So far, I can start the whole thing, select Play from the status bar dropdown, and it works! Great.
HOWEVER, if I attempt to use the bound hotkey, to run the same function that Play triggers, I get:
python[58226:599749] pid(58226)/euid(0) is calling TIS/TSM in non-main thread
environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
This shows up four times, and is followed up
python[58226:599749] WARNING: nextEventMatchingMask should only be
called from the Main Thread! This will throw an exception in the future.
To check that it wasn't the hotkey itself, I did a test: If I connect the hotkey to a simpler function, like setting an alert, it works fine. It still complains, and gives me the error:
python[60308:620099] -[NSAlert runModal] may only be invoked from the main thread. Behavior on other threads is undefined. (
0 AppKit 0x00007fff2b7f563f -[NSAlert runModal] + 178
1 _objc.cpython-36m-darwin.so 0x000000010c1358c7 ffi_call_unix64 + 79
2 ??? 0x000070000f3b2e50 0x0 + 123145557847632
)
but it does RUN. However if it connects to the Play function, it breaks (and produces the first error, above).
To summarize:
1) Dropdown => run minigame: Success!
2) Hotkey => run minigame: Incomprehensible errors!
I've googled this error, but have only seen explanations that are way over my head; is there a way that I could get around this error, by using different software, or a different approach -- but without having to leave python, or performing some deep and evil hack to an underlying system?
Could I get the hotkey/game to USE the Main Thread environment, somehow? How?
Could I use a vehicle other than a status bar app to listen for the hotkey? (Update: I tried pynput, and got the same non-main thread error.)
Could I do something inside of pygame to not make it as offensive? (At the moment, it's literally just animating a rolling sine wave.)
Could I get the rumps callback function to work? At the moment, it's just not doing ANYTHING, no matter where or how it's called.
(And unfortunately, this on a Mac, because pyhk3 is for windows only, and
wx.Window (which can have hotkeys) is also for windows only)
FURTHER failure: Can't get it to run with Keyboard Maestro either -- I've never used it before, so may well be using it wrong, but it also seems like a VERY simple command, that just flat-out has no response whatsoever.
Aaand ... the Automator script runs, but -- not with the hotkey! (EDIT: The hotkey was apparently taken. But a different one worked! See below.)
This problem is a MacOS 10.13 Sierra error. It is being reported with Processing.org 3d applications, as well as a Program called Synergy.
I'm accepting my own answer because it is the only one so far that solves the problem I was most interested in, i.e. getting the thing to run with a global shortcut, BUT, since I would ideally like to package and distribute this, I would still be very glad to see answers that solved the problem within python, and would gladly award the bounty to any answer that did that.
Finally, finally, made it work with a Service in Automator, and System Preferences > Keyboard > Shortcuts. Along the way, I had to
Explicitly call the python binary I wanted to use, because the bash shell that Automator uses in executing a text script is apparently different from the one in Terminal
Realize that the System Preferences shortcut wasn't working because of the key-combination I first chose, NOT that it was being blocked, because there were no errors or messages when I called it.
Probably all sorts of other things. Good god.

Bottle mishandling JSON data in Python3, but not Python2

I've got a bottle-based HTTP server that mostly shuffles JSON data around. When I run this in Python 2.7 it works perfectly, and in my route handlers I can access the JSON data via bottle.request.json. However, when I run it under Python 3.4 bottle.request.json is None.
I've examined the HTTP traffic, and in both cases it is exactly the same (as would expected since that's under control of the non-Python-dependent client.)
I also see that the JSON data is reaching bottle in both cases. If I print out bottle.request.params.keys(), I see the string-ified JSON as the only entry in the list in both cases. And the strings are identical in both cases. For some reason, however, the Python 2 version is recognizing the JSON data while the Python 3 version isn't.
Strangely, this used to work, but some recent change either in my code or bottle (or both) has broken things. Looking over my code, though, I can't see what I might have done to create the problem.
Does anyone know what's going on? Is this something I'm doing wrong at the client end, at the bottle configuration end, or is this a bottle defect? I searched for this problem both on google and the bottle issue tracker, but to no avail.
This turns out to have nothing to do with bottle. The ultimate cause of the problem is that the client request has two Content-Type headers due to a defect in an emacs lisp HTTP library. Embarrassingly, I've known about this defect for quite some time, but I thought I'd properly worked around it.
I'm not 100% sure why I see the variance between Python 2 and 3, but my guess right now is that it has to do with otherwise benign changes in the WSGI machinery between the versions.

pydev breakpoints not working

I am working on a project using python 2.7.2, sqlalchemy 0.7, unittest, eclipse 3.7.2 and pydev 2.4. I am setting breakpoints in python files (unit test files), but they are completely ignored (before, at some point, they worked). By now i have upgraded all related software (see above), started new projects, played around with settings, hypnotized my screen, but nothing works.
The only idea i got from some post is that it has something to de with changing some .py file names to lower case.
Does anyone have any ideas?
added: I even installed the aptana version of eclipse and copied the .py files to it => same result; breakpoints are still ignored.
still no progress: I have changed some code that might be seen as unusual and replaced it with a more straightforward solution.
some more info: it probably has something to do with module unittest:
breakpoints in my files defining test suites work,
breakpoints in the standard unittest files themselves work
breakpoints in my tests methods in classes derived from unittest.TestCase do not work
breakpoints in my code being tested in the test cases do not work
at some point before i could define working breakpoints in test methods or the code being tested
some things i changed after that are: started using test suites, changed some filenames to lowercase, ...
this problem also occurs if my code works without exceptions or test failures.
what I already tried is:
remove .pyc files
define new project and copy only .py files to it
rebooted several times in between
upgraded to eclipse 3.7.2
installed latest pydev on eclipse 3.7.2
switch to aptana (and back)
removed code that 'manually' added classes to my module
fiddled with some configurations
what I can still do is:
start new project with my code, start removing/changing code until breakpoints work and sort of black box figure out if this has something to do with some part of my code
Does anyone have any idea what might cause these problems or how they might be solved?
Is there any other place i could look for a solution?
Do pydev developers look into the questions on stackoverflow?
Is there an older version of pydev that i might try?
I have been working with pydev/eclipse for a long time and it works well for me, but without debugging i'd forced to switch IDE.
In answer to Fabio's questions below:
The python version is 2.7.2,
The sys.gettrace gives None (but I have no idea what in my code could influence that)
This is the output of the debugger after changing the suggested parameters:
pydev debugger:
starting
('Executing file ', 'D:\\.eclipse\\org.eclipse.platform_3.7.0_248562372\\plugins\\org.python.pydev.debug_2.4.0.2012020116\\pysrc\\runfiles.py')
('arguments:', "['D:\\\\.eclipse\\\\org.eclipse.platform_3.7.0_248562372\\\\plugins\\\\org.python.pydev.debug_2.4.0.2012020116\\\\pysrc\\\\runfiles.py', 'D:\\\\Documents\\\\Code\\\\Eclipse\\\\workspace\\\\sqladata\\\\src\\\\unit_test.py', '--port', '49856', '--verbosity', '0']")
('Connecting to ', '127.0.0.1', ':', '49857')
('Connected.',)
('received command ', '501\t1\t1.1')
sending cmd: CMD_VERSION 501 1 1.1
sending cmd: CMD_THREAD_CREATE 103 2 <xml><thread name="pydevd.reader" id="-1"/></xml>
sending cmd: CMD_THREAD_CREATE 103 4 <xml><thread name="pydevd.writer" id="-1"/></xml>
('received command ', '111\t3\tD:\\Documents\\Code\\Eclipse\\workspace\\sqladata\\src\\testData.py\t85\t**FUNC**testAdjacency\tNone')
Added breakpoint:d:\documents\code\eclipse\workspace\sqladata\src\testdata.py - line:85 - func_name:testAdjacency
('received command ', '122\t5\t;;')
Exceptions to hook : []
('received command ', '124\t7\t')
('received command ', '101\t9\t')
Finding files... done.
Importing test modules ... testAtomic (testTypes.TypeTest) ... ok
testCyclic (testTypes.TypeTest) ...
The rest is output of the unit test.
Continuing from Fabio's answer part 2:
I have added the code at the start of the program and the debugger stops working at the last line of following the method in sqlalchemy\orm\attributes.py (it is a descriptor, but how or whther it interferes with the debugging is beyond my current knowledge):
class InstrumentedAttribute(QueryableAttribute):
"""Class bound instrumented attribute which adds descriptor methods."""
def __set__(self, instance, value):
self.impl.set(instance_state(instance),
instance_dict(instance), value, None)
def __delete__(self, instance):
self.impl.delete(instance_state(instance), instance_dict(instance))
def __get__(self, instance, owner):
if instance is None:
return self
dict_ = instance_dict(instance)
if self._supports_population and self.key in dict_:
return dict_[self.key]
else:
return self.impl.get(instance_state(instance),dict_) #<= last line of debugging
From there the debugger steps into the __getattr__ method of one of my own classes, derived from a declarative_base() class of sqlalchemy.
Probably solved (though not understood):
The problem seemed to be that the __getattr__ mentioned above, created something similar to infinite recursion, however the program/unittest/sqlalchemy recovered without reporting any error. I do not understand the sqlalchemy code sufficiently to understand why the __getattr__ method was called.
I changed the __getattr__ method to call super for the attribute name for which the recursion occurred (most likely not my final solution) and the breakpoint problem seems gone.
If i can formulate the problem in a consise manner, i will probably try to get some more info on the google sqlalchemy newsgroup, or at least check my solution for robustness.
Thank you Fabio for your support, the trace_func() function pinpointed the problem for me.
Seems really strange... I need some more info to better diagnose the issue:
Open \plugins\org.python.pydev.debug\pysrc\pydevd_constants.py and change
DEBUG_TRACE_LEVEL = 3
DEBUG_TRACE_BREAKPOINTS = 3
run your use-case with the problem and add the output to your question...
Also, it could be that for some reason the debugging facility is reset in some library you use or in your code, so, do the following: in the same place that you'd put the breakpoint do:
import sys
print 'current trace function', sys.gettrace()
(note: when running in the debugger, it'd be expected that the trace function is something as: <bound method PyDB.trace_dispatch of <__main__.PyDB instance at 0x01D44878>> )
Also, please post which Python version you're using.
Answer part 2:
The fact that sys.gettrace() returns None is probably the real issue... I know some external libraries which mess with it (i.e.:DecoratorTools -- read: http://pydev.blogspot.com/2007/06/why-cant-pydev-debugger-work-with.html) and have even seen Python bugs and compiled extensions break it...
Still, the most common reason it breaks is probably because Python will silently disable the tracing (and thus the debugger) when a recursion throws a stack overflow error (i.e.: RuntimeError: maximum recursion depth exceeded).
You can probably put a breakpoint in the very beginning of your program and step in the debugger until it stops working.
Or maybe simpler is the following: Add the code below to the very beginning of your program and see how far it goes with the printing... The last thing printed is the code just before it broke (so, you could put a breakpoint at the last line printed knowing it should be the last line where it'd work) -- note that if it's a large program, printing may take a long time -- it may even be faster printing to a file instead of a console (such as cmd, bash or eclipse) and later opening that file (just redirect the print from the example to a file).
import sys
def trace_func(frame, event, arg):
print 'Context: ', frame.f_code.co_name, '\tFile:', frame.f_code.co_filename, '\tLine:', frame.f_lineno, '\tEvent:', event
return trace_func
sys.settrace(trace_func)
If you still can't figure it out, please post more information on the obtained results...
Note: a workaround until you don't find the actual place is using:
import pydevd;pydevd.settrace()
on the place where you'd put the breakpoint -- that way you'd have a breakpoint in code which should definitely work, as it'll force setting the tracing facility at that point (it's very similar to the remote debugging: http://pydev.org/manual_adv_remote_debugger.html except that as the debugger was already previously connected, you don't really have to start the remote debugger, just do the settrace to emulate a breakpoint)
Coming late into the conversation, but just in case it helps. I just run into a similar problem and I found that the debugger is very particular w.r.t. what lines it considers "executable" and available to break on.
If you are using line continuations, or multi-line expressions (e.g. inside a list), put the breakpoint in the last line of the statement.
I hope it helps.
Try removing the corresponding .pyc file (compiled) and then running.
Also I have sometimes realized I was running more than one instance of a program.. which confused pydev.
I've definitely seen this before too. Quite a few times.
Ran into a similar situation running a django app in Eclipse/pydev. what was happening was that the code that was running was the one installed in my virtualenv, not my source code. I removed my project from my virtual env site-packages, restarted the django up in the eclipse/pydev debugger and everything was fine.
I had similar-sounding symptoms. It turned out that my module import sequence was rexec'ing my entry-point python module because a binary (non-Python) library had to be dynamically loaded, i.e., the LD_LIBRARY_PATH was dynamically reset. I don't know why this causes the debugger to ignore subsequent breakpoints. Perhaps the rexec call is not specifying debug=true; it should specify debug=true/false based on the calling context state?
Try setting a breakpoint at your first import statement being cognizant of whether you are then s(tep)'ing into or n(ext)'ing over the imports. When I would "next" over the 3rdparty import that required the dynamic lib loading, the debug interpreter would just continue past all breakpoints.

Almost there installing Django/Python/PyISAPIe/IIS7, but message "Possible recursion detected" - configuration question

I'm installing Django onto a Windows system with Python 2.6 and IIS7 using PyISAPIe, and I'm so close I can taste it. Creating a little Django project and running it gives me:
Possible recursion detected! You probably did a passthrough with PyISAPIe configured as an application map instead of a wildcard map.
From various other postings, I get the impression it has to do with the handler mappings in IIS, and specifically how PyISAPIe is mapped to handle calls, but I'm at a loss beyond that point. One blog suggests changing PyIsapie.py to reflect a different error, but that just shifts the reaction around, not corrects the problem. In IIS7 PyISAPIe is set to * (complete wildcard) mapped to the .dll. Any ideas?
The question is kinda old but since I also ran into this prob and just solved it, maybe for future references I should mention that for the PyISAPIe script mapping someone must do the following in the properties of his virtual directory(home directory->configuration):
1) Add application extension
- executable:browse the `PyISAPIe.dll`, extension: `.py`
2) Insert wildcard application maps
- browse the `PyISAPIe.dll` (untick: Verify that file exists)
Hope it helps!

Categories

Resources