I'm running a high traffic ssl website with apache/mod_wsgi/python. Very occasionally (around 10x in 3 months) I've seen some extra garbage characters in post data.
Usually it's been in the form of a extra char at the end.
('access.uid', 'allow\xba')
('checksum', 'b219d6a006ebd95691d0d7b468a94510496c5dd8\xff')
Once though it was in the middle of someone's password. Something like:
('login_password', 'samplepass\xe7word')
I've tried to reconstruct the request with all the same headers but I haven't been able to duplicate the error. Anyone have any ideas about what could be causing this or any ideas of how I could go about reproducing and fixing this problem?
(Copied from below):
I'm using apache-2.2.17_1 – Peter Mar 15 at 18:09
I'm using mod_wsgi-3.3_1 on one machine and mod_wsgi-2.8_1 on another. I've seen this error on both.
What version of Apache are you using? From memory, somewhere around Apache 2.2.12-2.2.15 there were various SSL fixes. You might want to ensure you are using Apache 2.2.15 or later.
what happens if you print eval("u'%s'"%garbled_text)? does the output look likely (I understand that you may not be able to post sensitive data)
It looks to me like somewhere it's assuming you're reading ASCII even though you've told it to read utf-8.
Can we see the code that reads this POST data into python, or where it is specified and from what input form?
Since you said all errors occurred in IE 7 or 8 I'm starting to suspect the error occurs client-side in the browser. I've never heard of anything like this error and I have no clue what otherwise could cause it server-side except for hardware failure (though that seems weird too since only one character is added). Perhaps you should suggest your users to upgrade to a decent browser?
This looks very much like chunked HTTP/1.1.
Use an appropriate handler to un-chunk it prior to parsing. See [1], [2].
Another option is to only accept HTTP/1.0 which doesn't have chunking at all, but this may have downsides.
Related
I am currently working on a python project - windows application / software that has GUI where you can put patient information and so on and all that is going to a local mysql database / table. all that works but my IDE VS code gives 4 error messages and one of the errors is related to "len()" function. I will provide Github link of the entire code plus the error messages so if someone can help, please do so! Thank you!
https://github.com/ethicalduty/Hospital_Management_Software_Windows/blob/main/main_Hospital_Management_Software_Windows_3.11.0.py
I am new to programming so I cannot do much besides trying to find similar solution on google. I have not found anything already resolved so far, thus asking for help here!
The "errors" you are seeing are type checking issues, not real Python errors that are occurring when you run your code. They're warnings that there might be real errors in the code, but you should be able to run it regardless and the code may work fine. Whether there's a real issue may depend on the data the code is processing (it might work for some kinds of data but not others), or there might be inaccuracies in how the type checker is interpreting the code (and so there is no real issue in the code at all).
The issue that has to do with len seems to be saying that your rows = my_cursor.fetchall() statement may assign None to rows, which would cause an error below where you do len(rows). I don't know MySQL's Python bindings, so I'm not sure if the type checker's assumption is correct that a query could cause fetchall to return None (rather than an empty list), but if that's the case, you can easily fix the type checking issue (and the possibly small chance of a real error when you run the code) by checking that rows is not None before you check its length:
rows = my_cursor.fetchall()
if rows is not None and len(rows) != 0:
...
When using python dnslib.server example and trying make queries from bind9 server python dns server get these error's:
Mar 5 20:07:23 mx1 named[1868]: success resolving '85.31.199.1.domain.example/A' (in 'domain.example'?) after disabling EDNS
Mar 5 20:41:29 mx1 named[1868]: success resolving '141.36.123.190.domain.example/A' (in 'domain.example'?) after reducing the advertised EDNS UDP packet size to 512 octets
Any hint or solution how this can solve with python dnslib.server library ?
EDNS is a specific feature of DNS nameservers, albeit not a new one. But it needs specific support (code to handle it).
What you see is that bind9 as a client detects that the server it speaks too lacks EDNS support and hence fallback to some previous sane setup.
The changelog of your library has this:
0.7 2012-10-20 Add initial EDNS0 support (untested)
and nothing after, so it does not look good.
The code source shows two promising classes:
EDNS0
EDNSOption
It seems to be used when parsing the record in RR.parse that the server uses, but it may be working or not, we can not judge since you do not show your code and how it uses the library.
So, in short, either the library has a problem and its EDNS code will need to be improved or there is some problem in your program using it.
The related question is why you use this library, is it just to experiment/learn or to really build a product on top of it? In the last case, depending on what you are doing, lack of EDNS support may create you problems (for example for DNSSEC).
new to python (4 months)
got thru the first steps of basic programming skills, I believe, (having passed edX MIT 6001x and 60002x)
having big problems in the world of new libraries...
here an example:
r= requests.get ('URL',timeout=x)
works well with certain URL, keeps waiting with some other URL and I am getting
HTTPSConnectionPool(host='URL', port=443): Read timed out. (read timeout=x)
and without the timeout parameter, the jupyter notebook keeps turning the sand-watch.
I am not trying to handle the exception but to get it work.
Is there a simple way out or is requests.get too short for these kind of tasks?
And a more general question here,if you have the time: learning from the official docs (especially for larger and more complex modules) is getting too abstract for me, where it makes me feel hopeless. 'Straight diving' produces problems such as this one where you even cant figure out the simplest..
What would be an efficient way to deal with state of the art libraries? How did/do you go forward?
Try to check a file "robots.txt" of the website whose content you're trying to scrape (just type something like www.example.com/robots.txt). It's plausible that the website simply does not allow robots to use it. If this is the case, you may try to trick it by passing a custom header:
import requests
headers={'user-agent':'Chrome/41.0.2228.0'}
url='...'
r=requests.get(url, headers=headers, timeout=x)
But, of course, if you make thousands of queries to a website which does not allow robots, you'll still be blocked after a while.
I've got a bottle-based HTTP server that mostly shuffles JSON data around. When I run this in Python 2.7 it works perfectly, and in my route handlers I can access the JSON data via bottle.request.json. However, when I run it under Python 3.4 bottle.request.json is None.
I've examined the HTTP traffic, and in both cases it is exactly the same (as would expected since that's under control of the non-Python-dependent client.)
I also see that the JSON data is reaching bottle in both cases. If I print out bottle.request.params.keys(), I see the string-ified JSON as the only entry in the list in both cases. And the strings are identical in both cases. For some reason, however, the Python 2 version is recognizing the JSON data while the Python 3 version isn't.
Strangely, this used to work, but some recent change either in my code or bottle (or both) has broken things. Looking over my code, though, I can't see what I might have done to create the problem.
Does anyone know what's going on? Is this something I'm doing wrong at the client end, at the bottle configuration end, or is this a bottle defect? I searched for this problem both on google and the bottle issue tracker, but to no avail.
This turns out to have nothing to do with bottle. The ultimate cause of the problem is that the client request has two Content-Type headers due to a defect in an emacs lisp HTTP library. Embarrassingly, I've known about this defect for quite some time, but I thought I'd properly worked around it.
I'm not 100% sure why I see the variance between Python 2 and 3, but my guess right now is that it has to do with otherwise benign changes in the WSGI machinery between the versions.
I am using quickfix, compiled from the source on a linux box, setup to use the python headers. Everything 'seems' fine when I run my code, but I can't log on to my FIX server, and I noticed that the messages I'm sending have no field/tag delimiters, all the fields and values are just mashed together...
What might be causing this? Am I missing some setup in 'FIX_Settings.txt'?
Thanks!
I would comment with this, but I don't have enough reputation. So - I'm not sure why you can't log into your server, but are you sure that you don't have delimiters? Because if you're using \x01 as a delimiter in FIX, the tag-values pairs will usually just be displayed as "all mashed together," but the hex dump of it reveals otherwise (coming from personal experience).
Also, you might be getting downvoted because you haven't provided much context. If you provided the relevant bit of code or what your FIX output looks like, that might help.