Develop Reference

uWSGI call randomly injected into Python call stack?

I'm trying to get to the bottom of a weird bug (see here) - basically I'm seeing unexpected DB disconnects. To investigate I added a call stack dump to the disconnect code in Django - now I'm more confused than ever...
What I'm seeing in the call stack is a completely unexpected call sequence, e.g.:
...
File "/deployment/v_env/lib/python3.8/site-packages/qrcode/base.py", line 326, in __mod__
for item, other_item in zip(self, other)]
File "/deployment/v_env/lib/python3.8/site-packages/qrcode/base.py", line 303, in __iter__
return iter(self.num)
File "/deployment/v_env/lib/python3.8/site-packages/django/http/response.py", line 292, in close
signals.request_finished.send(sender=self._handler_class)
File "/deployment/v_env/lib/python3.8/site-packages/django/dispatch/dispatcher.py", line 180, in send
return [
...
Here the QR code library I'm using suddenly jumps to HttpResponse.close(). Another:
...
File "/deployment/v_env/lib/python3.8/site-packages/django/db/models/query.py", line 1268, in _insert
query = sql.InsertQuery(self.model, ignore_conflicts=ignore_conflicts)
File "/deployment/v_env/lib/python3.8/site-packages/django/db/models/sql/subqueries.py", line 141, in __init__
super().__init__(*args, **kwargs)
File "/deployment/v_env/lib/python3.8/site-packages/django/http/response.py", line 292, in close
signals.request_finished.send(sender=self._handler_class)
File "/deployment/v_env/lib/python3.8/site-packages/django/dispatch/dispatcher.py", line 180, in send
return [
...
Here some Django DB query code suddenly jumps to HttpResponse.close().
There's no way that this call stack is right - the code immediately above close() simply does not call that code, nor does it call a close() method that may somehow be bound to the wrong object.
Now HttpResponse.close() can be called by uWSGI from the C runtime - the only thing I can figure is that this is happening during the handling of a request, and somehow this is showing up in the call stack.
So my question is, can this is really the case? Does a C call into python really just jump into the call stack like this, or is uWSGI failing to protect the call to Python correctly? I understand the GIL must be acquired before any call to python - maybe that isn't happening. Or maybe traceback.print_stack() doesn't handle calls from C properly?
Any insight apeciated.

Thinking some more, I think this injection into the call stack is correct. How else could it work? The GIL could be obtained at any time, and Python isn’t going to start a new interpreter per call, so it’s got to just run in the current context.

Pyinstaller is not working with pygame, making an NotImplemented Error?

So i have a problem, whenever i try to use Pyinstaller with Pygame, it won't work, instead after a second, it says that it stopped working. I ran a test program before (consisting of a block moving up and down and a block following your cursor) and it worked just fine. However, i tried implementing some text using pygame.font.Font function and suddenly it stopped working. I tried using a try function to see if it could catch it, and nothing. I tried using no font, nothing. This also works as a regular .py file as well, just not as an executable. I'll include further info as needed if need be.
Thanks in advance!
EDIT:i was able to produce an error thanks to kaktus_car letting me know that there was an update to pygame, here it is:
Traceback (Most recent call last):
"test.py", line 20, in <module>
File "site-packages\pygame\pkgdata.py", line 50, in getResource
File "site-packages\pkg_resources\__init__.py", line 1134, in resource_exists
File "site-packages\pkg_resources\__init__.py", line 1404, in has_resource
File "site-packages\pkg_resources\__init__.py", line 1472, in _has
NotImplementedError: Can't perform this operation for unregistered loader type
[1276] Failed to execute script test

So, i managed to figure it out! What i did was i instaed used a different application than pyinstaller. Instead, i used cx_Freeze. If you are having this kind of problem as well, the link is Here the directions are rather clear and you have the option to include files, so if you have a game with made graphics, it can include them.

How to diagnose why Ctrl + C not stopping pserve

I'm experimenting porting a project from Cherrypy to Pyramid web framework. I have a small part converted and notice that Ctrl+C will not stop the Pyramid application. A cookiecutter version will stop with Ctrl+C. I end up needing to kill the process every time.
I am serving using the pserve command that uses the waitress WSGI server in both cases...
pserve development.ini
I should also note: I am running Debian Stretch in a VirtualBox VM.
Is there a way to know why the behavior has changed or how to restore Ctrl+C shutdown? How could I know if something is now blocking this from happening?
-- Additional Information asked for in comments --
Using grep Sig /proc/process_id/status yields the following:
SigQ: 0/15735
SigPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000001001000
SigCgt: 0000000180004002 hex/binary 110000000000000000100000000000010
Using GDB and getting a py-bt
(gdb) py-bt
Traceback (most recent call first):
<built-in method select of module object at remote 0x7f914f837e58>
File "/usr/lib/python3.5/asyncore.py", line 144, in poll
r, w, e = select.select(r, w, e, timeout)
File "/usr/lib/python3.5/asyncore.py", line 203, in loop
poll_fun(timeout, map)
File "/home/clutton/programs/python/webapps_pyramid/env/lib/python3.5/site-packages/waitress/server.py", line 131, in run
use_poll=self.adj.asyncore_use_poll,
File "/home/clutton/programs/python/webapps_pyramid/env/lib/python3.5/site-packages/waitress/__init__.py", line 17, in serve
server.run()
File "/home/clutton/programs/python/webapps_pyramid/env/lib/python3.5/site-packages/waitress/__init__.py", line 20, in serve_paste
serve(app, **kw)
File "/home/clutton/programs/python/webapps_pyramid/env/lib/python3.5/site-packages/paste/deploy/util.py", line 55, in fix_call
val = callable(*args, **kw)
File "/home/clutton/programs/python/webapps_pyramid/env/lib/python3.5/site-packages/paste/deploy/loadwsgi.py", line 189, in server_wrapper
**context.local_conf)
File "/home/clutton/programs/python/webapps_pyramid/env/lib/python3.5/site-packages/pyramid/scripts/pserve.py", line 239, in run
server(app)
File "/home/clutton/programs/python/webapps_pyramid/env/lib/python3.5/site-packages/pyramid/scripts/pserve.py", line 32, in main
return command.run()
File "/home/clutton/programs/python/webapps_pyramid/env/bin/pserve", line 11, in <module>
sys.exit(main())

In order to diagnose where I was running into issues I took the following steps guided by many of the comments made on the question.
I inspected the process to ensure that signals were indeed being caught.
grep Sig /proc/process_id/status
Which yields the following information:
SigQ: 0/15735
SigPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000001001000
SigCgt: 0000000180004002 hex/binary 110000000000000000100000000000010
The SigCgt indicates signals that are indeed being listened to, in the above the hex value converted to binary shows that (from right to left) signal 2 and 15 were indeed bound.
At this point we need to diagnose why there would be a handler yet it appeared to not be working. So the remaining question was what was the handler. To find that out I used the Python signal module and added some code where I could see it in a debugger...
import signal
s = signal.getsignal(2)
Once I did that I found that the handler referenced a function from a standalone script that is part of the project. I was overwriting the default signal handlers in order to do cleanup before terminating the process but... I was importing it as well in part of this project that had it's own process. Since the project is normally developed on Windows, I was probably dealing with different signals when using Ctrl-C previously so this bug has existed for a long time and doing some Linux development work for the project brought it to light.

Pythons Console Module has made it impossible to type the tab key

I started using it on one of my programs a while back. Ever since, whenever I type the tab key on a console (cmd.exe instance) with python running, I get a readline internal error. Full traceback is as follows (note I haven't imported the cmd module in this context or even imported a script using it. I've simply started python, pressed tab and voila an exception):
<pre>
Traceback (most recent call last):
File "C:\SP_CI_PROGRAMS\Languages\Python\3.6.1\lib\site-packages\pyreadline\console\console.py", line 768, in hook_wrapper_23
res = ensure_str(readline_hook(prompt))
File "C:\SP_CI_PROGRAMS\Languages\Python\3.6.1\lib\site-packages\pyreadline\rlmain.py", line 571, in readline
self._readline_from_keyboard()
File "C:\SP_CI_PROGRAMS\Languages\Python\3.6.1\lib\site-packages\pyreadline\rlmain.py", line 536, in _readline_from_keyboard
if self._readline_from_keyboard_poll():
File "C:\SP_CI_PROGRAMS\Languages\Python\3.6.1\lib\site-packages\pyreadline\rlmain.py", line 556, in _readline_from_keyboard_poll
result = self.mode.process_keyevent(event.keyinfo)
File "C:\SP_CI_PROGRAMS\Languages\Python\3.6.1\lib\site-packages\pyreadline\modes\emacs.py", line 243, in process_keyevent
r = self.process_keyevent_queue[-1](keyinfo)
File "C:\SP_CI_PROGRAMS\Languages\Python\3.6.1\lib\site-packages\pyreadline\modes\emacs.py", line 286, in _process_keyevent
r = dispatch_func(keyinfo)
File "C:\SP_CI_PROGRAMS\Languages\Python\3.6.1\lib\site-packages\pyreadline\modes\basemode.py", line 257, in complete
completions = self._get_completions()
File "C:\SP_CI_PROGRAMS\Languages\Python\3.6.1\lib\site-packages\pyreadline\modes\basemode.py", line 200, in _get_completions
r = self.completer(ensure_unicode(text), i)
File "C:\SP_CI_PROGRAMS\Languages\Python\3.6.1\Lib\rlcompleter.py", line 80, in complete
readline.redisplay()
AttributeError: module 'readline' has no attribute 'redisplay'
</pre>
Before u ask, I installed python to the directory "C:\SP_CI_PROGRAMS\Languages\Python\3.6.1". It's accessible from the path variable. Also any scripts I design I place in a directory u can also access from the path variable (including one using the cmd module for python).
This may not seem like a pressing concern, especially seeing as I can just type 4 spaces instead, however using tabs is something I've become especially accustomed to and the second I type the tab key, anything I've written in a previous block is immediately lost as the traceback is printed. Please, can someone tell me how to fix this.
Edit: This is only within the python interpreter. Typing tab within a running program or something else doesn't pose any problems.

Tested solution for Windows 10 (17 January 2020)
Copy last traceback file path C:\SP_CI_PROGRAMS\Languages\Python\3.6.1\Lib\rlcompleter.py
Open it with any text editor
If has VsCode use cmd and copy this
code C:\SP_CI_PROGRAMS\Languages\Python\3.6.1\Lib\rlcompleter.py
Look the line 80 which traceback tell us
Change these line (starts 79) like bellow and It will works
There will no error message and unnecessary tab more
...
if _readline_available: ## The old one is ##
if hasattr(readline, 'redisplay'): # if _readline_available:
readline.insert_text('\t') # readline.insert_text('\t')
readline.redisplay() # readline.redisplay()
return '' # return ''
...
Do not forget to relaunch your python terminal

Seems to be a continuing issue for Windows machines as seen on Github. A workaround seems to be uninstalling the pyreadline package.

Stop using pyreadline. It's been abandoned. What you're seeing is a known issue, but unless someone takes over pyreadline development, it's unlikely to ever be fixed.

pyreadline can be uninstalled by typing pip uninstall pyreadline in the command prompt. I was experiencing the same issue, but after uninstalling pyreadline, Tab key is working for me.
NOTE: To see the installed packages, type pip list in command prompt.

Well if you don't have permission on the machine. You won't be able to do those uninstall solutions.
So set the completer function. It will solve the problem and won't have weird behaviors. But the auto-completion won't work anymore. If you want it to work have a look at the documentation and implement it.
readline.set_completer(lambda t, s: [None][s])

What has worked for me (while the others did not for some unbeknownst reason) is the following:
In line 78:
if _readline_available:
if hasattr(readline, 'redisplay'):
readline.insert_text('\t')
readline.redisplay()
return ' ' * 4

Using Ladon in Python 2.6 on windows

I have been trying to create a web service out some python scripts, and haven't seemed to have had much luck. I am new to web services in general, but would really like to get this figured out. I'm on Windows 7 and use IIS7. The service also needs to be SOAP.
I've read through most posts that have anything to do with python and SOAP and tried out pretty much all the different libraries, but most of them just seem over my head (especially ZSI/SOAPpy). The Ladon Library seems like it would be best (and simplest) for what I need, but the tutorial http://www.youtube.com/watch?v=D_JYjEBedk4&feature=feedlik loses me at 5:10 when he brings it to the server. When I type the ladon2.6ctl in the cmd, it seems like windows gets quite confused. I'm guessing it is a little different because he is running on Linux and using Apache.
With that, any tips on how to get a python web service running on Microsoft 'stuff' would be greatly appreciated, as I have been trying to figure this stuff out for way too long.
One thing to note is the reason things are so specific (and seemingly strange) is because the scripts I have do a lot of geoprocessing with ESRI's "arcpy".
--Addition--
Traceback on localhost:8080/TestService:
Traceback (most recent call last):
<br>File "c:\Python26\ArcGIS10.0\lib\site-packages\ladon-0.5.1-py2.6.egg\ladon\server\wsgi_application.py", line 229, in __call__
exec("import %s" % ','.join(self.service_list))
File "<string>", line 1, in <module>
File "c:\Users\r\workspace\ladon\src\testspace.py", line 3, in <module>
class TestService2(object):
File "c:\Users\r\workspace\ladon\src\testspace.py", line 4, in TestService2
#ladonize(int,int,rtype=int)
File "c:\Python26\ArcGIS10.0\lib\site-packages\ladon-0.5.1-py2.6.egg\ladon\ladonizer\decorator.py", line 87, in decorator
ladon_method_info = global_service_collection().add_service_method(f,*def_args,**def_kw)
File "c:\Python26\ArcGIS10.0\lib\site-packages\ladon-0.5.1-py2.6.egg\ladon\ladonizer\collection.py", line 119, in add_service_method
sinfo = self.source_info(src_fname)
File "c:\Python26\ArcGIS10.0\lib\site-packages\ladon-0.5.1-py2.6.egg\ladon\ladonizer\collection.py", line 79, in source_info
a = ast.parse(src)
File "c:\Python26\ArcGIS10.0\lib\ast.py", line 37, in parse
return compile(expr, filename, mode, PyCF_ONLY_AST)
File "<unknown>", line 1
from ladon.ladonizer import ladonize
^
SyntaxError: invalid syntax
sample code:
from ladon.ladonizer import ladonize
class TestService2(object):
#ladonize(int,int,rtype=int)
def sum(self,a,b):
'''add two numbers<br>
param a: number 1
param b: number 2
rtype: sum of result
'''
return a+b

I must admit I normally use Linux for almost everything and I haven't tried Ladon on Windows for a while. I will spin up my windows installation later today and see if there is any trouble.
You wrote that ladon2.6ctl get's confused. Do you have an exception Traceback?

To summarize the fix for anyone else interested, delete the "syslog import" from these 3 ladon modules: ladon/interfaces/jsonwsp.py - line 6
ladon/dispatcher/dispatcher.py - line 7
ladon/server/wsgi_application.py - line 37
Then, you need to change the linefeed from window's default of /r/n to /n. In Eclipse, go to Window -> Preferences -> General, then select (not drop down) the Workspace tab. On the bottom right, select "other" under New text file line delimiter and change it to Unix.