Develop Reference

Why I`m getting OSError: [Errno 7] Argument list too long: b'/usr/local/bin/git'?

I'm doing a programming course (CS50W) and it has a command to submit the project to github automatically. It's called submit50.
When trying to push my code to github through this method, I do:
submit50 --verbose web50/projects/2020/x/capstone
And I'm getting the following error:
OSError: [Errno 7] Argument list too long: b'/usr/local/bin/git'
I have literally no idea what is happening here. If you could help me out I would be grateful.
Full error with traceback:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.8/bin/submit50", line 11, in <module>
load_entry_point('submit50==3.0.2', 'console_scripts', 'submit50')()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/submit50/__main__.py", line 147, in main
user_name, commit_hash, message = lib50.push("submit50", args.slug, CONFIG_LOADER, prompt=prompt)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/lib50/_api.py", line 69, in push
with authenticate(remote["org"], repo=repo) as user, prepare(tool, slug, user, included):
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/contextlib.py", line 113, in __enter__
return next(self.gen)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/lib50/_api.py", line 326, in prepare
_run(git(f"add -f {' '.join(shlex.quote(f) for f in included)}"))
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/lib50/_api.py", line 703, in _run
with _spawn(command, quiet, timeout) as child:
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/contextlib.py", line 113, in __enter__
return next(self.gen)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/lib50/_api.py", line 674, in _spawn
child = pexpect.spawn(
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pexpect/pty_spawn.py", line 205, in __init__
self._spawn(command, args, preexec_fn, dimensions)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pexpect/pty_spawn.py", line 303, in _spawn
self.ptyproc = self._spawnpty(self.args, env=self.env,
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pexpect/pty_spawn.py", line 315, in _spawnpty
return ptyprocess.PtyProcess.spawn(args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ptyprocess/ptyprocess.py", line 330, in spawn
raise exception
OSError: [Errno 7] Argument list too long: b'/usr/local/bin/git'
Submission cancelled.

"Argument list too long" is a Unix-style1 error that indicates that the arguments to the exec-like function are, well, too long. (The b'/usr/local/bin/git' part here is misleading: that's not the part that is too long.) The /Library/Frameworks/Python.framework/ prefix strongly suggests that you're on MacOS, which has a Mach-ish kernel that is Unix-based and has relatively small argument limits:
sh-3.2$ sysctl kern.argmax
kern.argmax: 262144
This means that there's a maximum of 256 kiB of argument characters, including the command itself and all the bytes of arguments, including the \0 byte that terminates each one. There's often a limit on the length of argv itself as well. The details vary from one system to another. See also this question on the unix.stackexchange.com site, for instance.
It's not immediately clear to me what to do about this. Looking at the stack trace, we find this pair of lines (which I've split up further for posting purposes here):
File "/Library/Frameworks/Python.framework/Versions/3.8/
lib/python3.8/site-packages/lib50/_api.py", line 326, in prepare
_run(git(f"add -f {' '.join(shlex.quote(f) for f in included)}"))
which points to the main culprit: a function named prepare in the lib50 library that is part of what your course provided. This is trying to run a Git command with many file names passed to its git add command. Where did it get the included list? That, we don't know. If this list were shorter, the problem would not occur, but if there is no way for you to shorten the list, knowing this won't help.
Raising kern.argmax would help, but it is probably not something you can set.
Using a different OS, with a different (or raise-able) maximum, could help.
The "right answer", though, is probably to have the software itself be smarter. There's no need to invoke git add this particular way, with every file listed as one big argv vector. Indeed, instead of invoking git add directly, a program should probably be invoking git update-index instead. A Python program should probably be using update-index with the --stdin and -z flags (and any other flags as appropriate for this particular function's intended usage).
1Other systems now provide Unix-like error numbers as well, even if they're not based on Unix. This particular one is E2BIG or error #7, on MacOS.

How to diagnose why Ctrl + C not stopping pserve

I'm experimenting porting a project from Cherrypy to Pyramid web framework. I have a small part converted and notice that Ctrl+C will not stop the Pyramid application. A cookiecutter version will stop with Ctrl+C. I end up needing to kill the process every time.
I am serving using the pserve command that uses the waitress WSGI server in both cases...
pserve development.ini
I should also note: I am running Debian Stretch in a VirtualBox VM.
Is there a way to know why the behavior has changed or how to restore Ctrl+C shutdown? How could I know if something is now blocking this from happening?
-- Additional Information asked for in comments --
Using grep Sig /proc/process_id/status yields the following:
SigQ: 0/15735
SigPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000001001000
SigCgt: 0000000180004002 hex/binary 110000000000000000100000000000010
Using GDB and getting a py-bt
(gdb) py-bt
Traceback (most recent call first):
<built-in method select of module object at remote 0x7f914f837e58>
File "/usr/lib/python3.5/asyncore.py", line 144, in poll
r, w, e = select.select(r, w, e, timeout)
File "/usr/lib/python3.5/asyncore.py", line 203, in loop
poll_fun(timeout, map)
File "/home/clutton/programs/python/webapps_pyramid/env/lib/python3.5/site-packages/waitress/server.py", line 131, in run
use_poll=self.adj.asyncore_use_poll,
File "/home/clutton/programs/python/webapps_pyramid/env/lib/python3.5/site-packages/waitress/__init__.py", line 17, in serve
server.run()
File "/home/clutton/programs/python/webapps_pyramid/env/lib/python3.5/site-packages/waitress/__init__.py", line 20, in serve_paste
serve(app, **kw)
File "/home/clutton/programs/python/webapps_pyramid/env/lib/python3.5/site-packages/paste/deploy/util.py", line 55, in fix_call
val = callable(*args, **kw)
File "/home/clutton/programs/python/webapps_pyramid/env/lib/python3.5/site-packages/paste/deploy/loadwsgi.py", line 189, in server_wrapper
**context.local_conf)
File "/home/clutton/programs/python/webapps_pyramid/env/lib/python3.5/site-packages/pyramid/scripts/pserve.py", line 239, in run
server(app)
File "/home/clutton/programs/python/webapps_pyramid/env/lib/python3.5/site-packages/pyramid/scripts/pserve.py", line 32, in main
return command.run()
File "/home/clutton/programs/python/webapps_pyramid/env/bin/pserve", line 11, in <module>
sys.exit(main())

In order to diagnose where I was running into issues I took the following steps guided by many of the comments made on the question.
I inspected the process to ensure that signals were indeed being caught.
grep Sig /proc/process_id/status
Which yields the following information:
SigQ: 0/15735
SigPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000001001000
SigCgt: 0000000180004002 hex/binary 110000000000000000100000000000010
The SigCgt indicates signals that are indeed being listened to, in the above the hex value converted to binary shows that (from right to left) signal 2 and 15 were indeed bound.
At this point we need to diagnose why there would be a handler yet it appeared to not be working. So the remaining question was what was the handler. To find that out I used the Python signal module and added some code where I could see it in a debugger...
import signal
s = signal.getsignal(2)
Once I did that I found that the handler referenced a function from a standalone script that is part of the project. I was overwriting the default signal handlers in order to do cleanup before terminating the process but... I was importing it as well in part of this project that had it's own process. Since the project is normally developed on Windows, I was probably dealing with different signals when using Ctrl-C previously so this bug has existed for a long time and doing some Linux development work for the project brought it to light.

BlockingSwitchOutError in debugger after upgrading to PyCharm 2016.2

After upgrading from PyCharm 2016.1.4 to 2016.2, when running the debugger and setting any breakpoint, PyCharm halts in various places where I have no breakpoint set, and logs this to stderr:
Traceback (most recent call last):
File "/usr/local/pycharm/debug-eggs/pycharm-debug.egg/_pydevd_bundle/pydevd_frame.py", line 539, in trace_dispatch
self.do_wait_suspend(thread, frame, event, arg)
File "/usr/local/pycharm/debug-eggs/pycharm-debug.egg/_pydevd_bundle/pydevd_frame.py", line 71, in do_wait_suspend
self._args[0].do_wait_suspend(*args, **kwargs)
File "/usr/local/pycharm/debug-eggs/pycharm-debug.egg/pydevd.py", line 714, in do_wait_suspend
time.sleep(0.01)
File "/home/jaza/mypyapp/mypyfile.py", line 999, in mypyfunc
gevent.sleep(seconds)
File "/usr/local/lib/python2.7/dist-packages/gevent/hub.py", line 194, in sleep
hub.wait(loop.timer(seconds, ref=ref))
File "/usr/local/lib/python2.7/dist-packages/gevent/hub.py", line 630, in wait
result = waiter.get()
File "/usr/local/lib/python2.7/dist-packages/gevent/hub.py", line 878, in get
return self.hub.switch()
File "/usr/local/lib/python2.7/dist-packages/gevent/hub.py", line 608, in switch
switch_out()
File "/usr/local/lib/python2.7/dist-packages/gevent/hub.py", line 612, in switch_out
raise BlockingSwitchOutError('Impossible to call blocking function in the event loop callback')
BlockingSwitchOutError: Impossible to call blocking function in the event loop callback
OS: Linux Mint 17.3 (i.e. almost identical to Ubuntu 14.04). Using latest gevent (1.1.2).
If I open my old PyCharm (i.e. 2016.1.4), and do the same thing - i.e. start the debugger, set a breakpoint, run my app - I don't get these errors, and PyCharm doesn't halt anywhere in the code except at my breakpoint.
I also tried just "downgrading" the debugger, by renaming the debug-eggs directory and replacing it with a symlink to the old debug-eggs path, and then running the rest of PyCharm on the latest version. This didn't fix the problem, i.e. it still resulted in BlockingSwitchOutError being raised numerous times.
Seems likely that this is a bug in PyCharm 2016.2. I've submitted a bug report to JetBrains for this, see https://youtrack.jetbrains.com/issue/PY-20183 . But posting here on SO as well, in case anyone sees a problem with the code in my app (the use of gevent.sleep(seconds) ?), meaning that the code happened to work before, but was going to break sooner or later.

The solution posted by Elizaveta Shashkova on the PyCharm issue tracker at https://youtrack.jetbrains.com/issue/PY-20183 worked for me:
The new feature has appeared in PyCharm: breakpoint thread suspend policy. You should go Run | View breakpoints, select the breakpoint and change its threads suspend policy: "Thread" or "All". Also you can set the default policy for all your breakpoints.
After changing suspend policy from "All" to "Thread", debugger is no longer breaking outside of my breakpoints nor throwing BlockingSwitchOutError.
And, re:
Also do you have the setting "gevent compatible" turned on? https://www.jetbrains.com/help/pycharm/2016.1/python-debugger.html
No, I don't have this turned on, and I fixed my issue without turning it on. But will try turning it on if I have similar issues in future.

Image layering in Tkinter, Python [figured it out (smacks forehead)]

I am working with a graphics module called cTurtle, which builds upon TKinter. Each controllable 'Turtle' has an image assigned to it with TKinter. I am trying to add a method to the module that will allow me to change the layering position of a given 'Turtle'. Or to put it another way, i want to add a method that will allow me to put a given image on top of the displaylist. When i call the method i have made, i get this error
Traceback (most recent call last):
File "C:\Documents and Settings\login\Desktop\pYTHON EXPERIMENTS\canvastest.py", line 22, in <module>
bob.visUp()
File "C:\Documents and Settings\login\Desktop\pYTHON EXPERIMENTS\jTurtle.py", line 2017, in visUp
titem = self.turtle._item
File "C:\Documents and Settings\login\Desktop\pYTHON EXPERIMENTS\jTurtle.py", line 542, in _visUp
def _visUp(self, item):
File "<string>", line 1, in type
File "C:\Python32\lib\tkinter\__init__.py", line 2325, in type
return self.tk.call(self._w, 'type', tagOrId) or None
_tkinter.TclError: invalid command name ".16847312"
Does someone recognize why the function is rejecting what i passed to it? The number in quotes appears to be an object id, which should work.
EDIT
well, i did myself a great disservice here by not pasting everything involved. I coded these new methods several days ago, and have been trying to debug since. Turns out i coded them right the first time. The problem was in how i called them. I added the visup() test line after exitOnClick(), meaning that the widget had, in fact, already been destroyed.
But, hey, i coded it right the first time! sorta...

Using Ladon in Python 2.6 on windows

I have been trying to create a web service out some python scripts, and haven't seemed to have had much luck. I am new to web services in general, but would really like to get this figured out. I'm on Windows 7 and use IIS7. The service also needs to be SOAP.
I've read through most posts that have anything to do with python and SOAP and tried out pretty much all the different libraries, but most of them just seem over my head (especially ZSI/SOAPpy). The Ladon Library seems like it would be best (and simplest) for what I need, but the tutorial http://www.youtube.com/watch?v=D_JYjEBedk4&feature=feedlik loses me at 5:10 when he brings it to the server. When I type the ladon2.6ctl in the cmd, it seems like windows gets quite confused. I'm guessing it is a little different because he is running on Linux and using Apache.
With that, any tips on how to get a python web service running on Microsoft 'stuff' would be greatly appreciated, as I have been trying to figure this stuff out for way too long.
One thing to note is the reason things are so specific (and seemingly strange) is because the scripts I have do a lot of geoprocessing with ESRI's "arcpy".
--Addition--
Traceback on localhost:8080/TestService:
Traceback (most recent call last):
<br>File "c:\Python26\ArcGIS10.0\lib\site-packages\ladon-0.5.1-py2.6.egg\ladon\server\wsgi_application.py", line 229, in __call__
exec("import %s" % ','.join(self.service_list))
File "<string>", line 1, in <module>
File "c:\Users\r\workspace\ladon\src\testspace.py", line 3, in <module>
class TestService2(object):
File "c:\Users\r\workspace\ladon\src\testspace.py", line 4, in TestService2
#ladonize(int,int,rtype=int)
File "c:\Python26\ArcGIS10.0\lib\site-packages\ladon-0.5.1-py2.6.egg\ladon\ladonizer\decorator.py", line 87, in decorator
ladon_method_info = global_service_collection().add_service_method(f,*def_args,**def_kw)
File "c:\Python26\ArcGIS10.0\lib\site-packages\ladon-0.5.1-py2.6.egg\ladon\ladonizer\collection.py", line 119, in add_service_method
sinfo = self.source_info(src_fname)
File "c:\Python26\ArcGIS10.0\lib\site-packages\ladon-0.5.1-py2.6.egg\ladon\ladonizer\collection.py", line 79, in source_info
a = ast.parse(src)
File "c:\Python26\ArcGIS10.0\lib\ast.py", line 37, in parse
return compile(expr, filename, mode, PyCF_ONLY_AST)
File "<unknown>", line 1
from ladon.ladonizer import ladonize
^
SyntaxError: invalid syntax
sample code:
from ladon.ladonizer import ladonize
class TestService2(object):
#ladonize(int,int,rtype=int)
def sum(self,a,b):
'''add two numbers<br>
param a: number 1
param b: number 2
rtype: sum of result
'''
return a+b

I must admit I normally use Linux for almost everything and I haven't tried Ladon on Windows for a while. I will spin up my windows installation later today and see if there is any trouble.
You wrote that ladon2.6ctl get's confused. Do you have an exception Traceback?

To summarize the fix for anyone else interested, delete the "syslog import" from these 3 ladon modules: ladon/interfaces/jsonwsp.py - line 6
ladon/dispatcher/dispatcher.py - line 7
ladon/server/wsgi_application.py - line 37
Then, you need to change the linefeed from window's default of /r/n to /n. In Eclipse, go to Window -> Preferences -> General, then select (not drop down) the Workspace tab. On the bottom right, select "other" under New text file line delimiter and change it to Unix.