I am trying to upload a file to the server in a simple way and I am getting the following error:
HTTP/1.1 411 Length Required
Content-Type: text/html
Date: Wed, 01 Jul 2015 03:05:33 GMT
Connection: close
Content-Length: 24
Length Required
I tried to insert length in different parts and looks like is not working, any suggestions?
import socket
import httplib
import os.path
target_host = "192.168.1.1"
target_port = 80
total_size = os.path.getsize('/root/test.html')
client = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
client.connect((target_host,target_port))
client.send("PUT /root/test.html HTTP/1.1\r\nHost:192.168.1.1\r\n\r\n" )
response = client.recv(4096)
print response
Have you tried implementing this with requests?
You could easily specify the missing content-length header that is causing your error.
Check out this answer: Is there any way to do HTTP PUT in python
Here's an example using the requests library:
payload = {'username': 'me', 'email': 'me#me.com'}
>>> r = requests.put("http://somedomain.org/endpoint", data=payload)
Related
I'm trying to use a Python script to call the API detailed in the link below:
https://developer.wmata.com/docs/services/gtfs/operations/5cdc51ea7a6be320cab064fe?
When I use the code below, it always returns a 404 error:
import requests
import json
def _url(path):
return "http://api.wmata.com" + path
def pull_data():
return requests.get(_url("/gtfs/bus-gtfsrt-tripupdates.pb"), params=params)
def jprint(obj):
# create a formatted string of the Python JSON object
text = json.dumps(obj, sort_keys=True, indent=4)
print(text)
# authenticate with your api key
params = {
"apiKey": "mykey",
}
response = pull_data()
print(response)
jprint(response.json())
I have also tried using the python code provided in the link, but it returns meaningless response content as shown below. Any attempts to decode the content have been unsuccessful.
Request-Context: appId=cid-v1:2833aead-1a1f-4ffd-874e-ef3a5ceb1de8
Cache-Control: public, must-revalidate, max-age=5
Date: Thu, 11 Feb 2021 22:05:31 GMT
ETag: 0x8D8CED90CC8419C
Content-Length: 625753
Content-MD5: fspEFl7LJ8QbZPgf677WqQ==
Content-Type: application/octet-stream
Expires: Thu, 11 Feb 2021 22:05:37 GMT
Last-Modified: Thu, 11 Feb 2021 22:04:49 GMT
1.0�Ԗ��
1818410080�
181841008020210211*52!�Ԗ�"19032("�Ԗ�"18173(#�Ԗ�"7779($�Ֆ�"18174(%�Ֆ�"7909(&�Ֆ�"7986('�Ֆ�"8039((�Ֆ�"8130()�֖�"8276(+�֖�"8313(,�ז�"8403(-�ז�"8452(.�ז�"8520(/�ؖ�"8604(1�ؖ�"8676(
7070 �Ӗ�(����������
1814174080�
181417408020210211*P129�Ԗ�"2373(:�Ԗ�"2387(;�Ԗ�"17296(=�Ֆ�"17212(>�֖�"2444(?�֖�"2493(#�֖�"2607(A�֖�"14633(B�֖�"2784(C�ז�"2832(D�ז�"2843(E�ז�"2848(F�ז�"2875(G�ؖ�"2945(H�ؖ�"2987(I�ؖ�"21946(K�ٖ�"14636(L�ٖ�"3122(M�ٖ�"3227(N�ٖ�"3308(O�ٖ�"3411(P�ٖ�"3500(Q�ٖ�"3539(R�ٖ�"14637(S�ږ�"3685(T�ږ�"15195(U�ږ�"15196(V�ۖ�"4243(W�ۖ�"4443(X�ۖ�"4517(Y�ۖ�"4631([�ܖ�"11962(
8002 �Ӗ�(/�
1825989080�
182598908020210211*7Y�Ӗ�"2158(�Ԗ�"2215(�Ԗ�"2259(�Ԗ�"2292(�Ԗ�"2299(�Ֆ�"18701(�Ֆ�"2310( �Ֆ�"2245(!�Ֆ�"2174("�Ֆ�"1987(#�֖�"1937(%�֖�"1864(
3191 �Ӗ�(��
1819988080�
Any guidance or direction would be greatly appreciated!
Change as pull_data function as follows:
def pull_data():
return requests.get(_url("/gtfs/bus-gtfsrt-tripupdates.pb"), headers=headers)
Then rename params module global variable to headers .
headers = {"apiKey": "mykey"}
WMATA looks for a apiKey in the headers, not in the query params.
Update: I noticed they use api_key for some samples, and apiKey for another ones. For example see:
https://developer.wmata.com/docs/services/gtfs/operations/5cdc51ea7a6be320cab064fe
Update 2: Notice the content type in the response headers :
print(response.headers['content-type'])
# application/octet-stream
it is not a JSON. You can get contents as follows:
print(response.content)
Worked example:
import requests
API_URL = 'https://api.wmata.com'
def _prepare_url(path):
return f'{API_URL}/{path.lstrip("/")}'
def pull_data(**options):
url = _prepare_url('/gtfs/bus-gtfsrt-tripupdates.pb')
return requests.get(url, **options)
response = pull_data(headers={'api_key': 'secret'})
print(response.content)
Try changing your URL so that it is using https instead of http. The documentation that you have linked at Bus RT Trip Updates seems to indicate that https is required.
Change this:
def _url(path):
return "http://api.wmata.com" + path
to make it this:
def _url(path):
return "https://api.wmata.com" + path
I am trying to query a website in Python. I need to use a POST method (according to what is happening in my browser when I monitor it with the developer tools).
If I query the website with cURL, it works well:
curl -i --data "param1=var1¶m2=var2" http://www.test.com
I get this header:
HTTP/1.1 200 OK
Date: Tue, 26 Sep 2017 08:46:18 GMT
Server: Apache/1.3.33 (Unix) mod_gzip/1.3.26.1a mod_fastcgi/2.4.2 PHP/4.3.11
Transfer-Encoding: chunked
Content-Type: text/html
But when I do it in Python 3, I get an error 104.
Here is what I tried so far. First, with urllib (getting inspiration from this thread to manage to use a POST method instead of GET):
import re
from urllib import request as ur
# URL to handle request
url = "http://www.test.com"
data = "param1=var1¶m2=var2"
# Build a request dictionary
preq = [re.findall("[^=]+", i) for i in re.findall("[^\&]+", data)]
dreq = {i[0]: i[1] if len(i) == 2 else "" for i in preq}
# Initiate request & add method
ureq = ur.Request(url)
ureq.get_method = lambda: "POST"
# Send request
req = ur.urlopen(ureq, data=str(dreq).encode())
I did basically the same with requests:
import re
import requests
# URL to handle request
url = "http://www.test.com"
data = "param1=var1¶m2=var2"
# Build a request dictionary
preq = [re.findall("[^=]+", i) for i in re.findall("[^\&]+", data)]
dreq = {i[0]: i[1] if len(i) == 2 else "" for i in preq}
# Initiate request & add method
req = requests.post(url, data=dreq)
In both cases, I get a HTTP 104 error:
ConnectionResetError: [Errno 104] Connection reset by peer
That I don't understand since the same request is working with cURL. I guess I misunderstood something with Python request but so far I'm stuck. Any hint would be appreciated!
I've just figured out I did not pass data in the right format. I thought it needed to be store in a dict; that is not the case and it is therefore much more simple that what I tried previously.
With urllib:
req = ur.urlopen(ureq, data=str(data).encode())
With requests:
req = requests.post(url, data=data)
I am having a device which is sending the following http message to my RaspberryPi:
POST /sinvertwebmonitor/InverterService/InverterService.asmx/CollectInverterData HTTP/1.1
Host: www.automation.siemens.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 349
xmlData=<rd><m>xxxxx</m><s>yyyyyy</s><d t="1483019400" l="600"><p i="1">460380AE</p><p i="2">43655DE7</p><p i="3">4212C986</p><p i="4">424805BC</p><p i="5">4604E3D1</p><p i="6">441F616A</p><p i="7">4155E7F5</p><p i="8">E1</p><p i="9">112</p><p i="C">153</p><p i="D">4</p><p i="E">11ABAC</p><p i="F">22A48C</p><p i="10">0</p></d></rd>
I cannot change anything on the device.
On the RaspberryPi im running a script to listen and receive the message from a socket.
This works so far and the received message is the one above.
Now, I would like to create a HTTP object from this message and then extract comfortably the header, content and so on.
Similar to the following example:
r = requests.get('https://www.google.com')
r.status_code
However, without "getting" an url. I just want to read the string I already have.
Pseudo-example:
r = requests.read(hereComesTheString)
r.status_code
I hope the problem became understandable.
Would be glad to get some hints.
Thanks and best regards,
Christoph
You use the status_code property in your example, but what you are receiving is a request not a response. However you can still create a simple object for accessing the data in the request.
It is probably easiest to create your own custom class:
import mimetools
from StringIO import StringIO
request = """POST /sinvertwebmonitor/InverterService/InverterService.asmx/CollectInverterData HTTP/1.1
Host: www.automation.siemens.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 349
xmlData=<rd><m>xxxxx</m><s>yyyyyy</s><d t="1483019400" l="600"><p i="1">460380AE</p><p i="2">43655DE7</p><p i="3">4212C986</p><p i="4">424805BC</p><p i="5">4604E3D1</p><p i="6">441F616A</p><p i="7">4155E7F5</p><p i="8">E1</p><p i="9">112</p><p i="C">153</p><p i="D">4</p><p i="E">11ABAC</p><p i="F">22A48C</p><p i="10">0</p></d></rd>"""
class Request:
def __init__(self, request):
stream = StringIO(request)
request = stream.readline()
words = request.split()
[self.command, self.path, self.version] = words
self.headers = mimetools.Message(stream, 0)
self.content = stream.read()
def __getitem__(self, key):
return self.headers.get(key, '')
r = Request(request)
print(r.command)
print(r.path)
print(r.version)
for header in r.headers:
print(header, r[header])
print(r.content)
This outputs:
POST
/sinvertwebmonitor/InverterService/InverterService.asmx/CollectInverterData
HTTP/1.1
('host', 'www.automation.siemens.com')
('content-type', 'application/x-www-form-urlencoded')
('content-length', '349')
xmlData=<rd><m>xxxxx</m><s>yyyyyy</s><d t="1483019400" l="600"><p i="1">460380AE</p><p i="2">43655DE7</p><p i="3">4212C986</p><p i="4">424805BC</p><p i="5">4604E3D1</p><p i="6">441F616A</p><p i="7">4155E7F5</p><p i="8">E1</p><p i="9">112</p><p i="C">153</p><p i="D">4</p><p i="E">11ABAC</p><p i="F">22A48C</p><p i="10">0</p></d></rd>
If you're using plain socket server, then you need to implement an HTTP server so that you can split the request and respond according to the protocol.
It's probably easier just to use an existing HTTP server and app server. Flask is ideal for this:
from flask import Flask
from flask import request
app = Flask(__name__)
#app.route("/sinvertwebmonitor/InverterService/InverterService.asmx/CollectInverterData", methods=['POST'])
def dataCollector():
data = request.form['xmlData']
print(data)
# parseData. Take a look at ElementTree
if __name__ == "__main__":
app.run(host=0.0.0.0, port=80)
Thanks Alden. Below your code with a few changes so it works with Python3.
import email
from io import StringIO
request = """POST /sinvertwebmonitor/InverterService/InverterService.asmx/CollectInverterData HTTP/1.1
Host: www.automation.siemens.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 349
xmlData=<rd><m>xxxxx</m><s>yyyyyy</s><d t="1483019400" l="600"><p i="1">460380AE</p><p i="2">43655DE7</p><p i="3">4212C986</p><p i="4">424805BC</p><p i="5">4604E3D1</p><p i="6">441F616A</p><p i="7">4155E7F5</p><p i="8">E1</p><p i="9">112</p><p i="C">153</p><p i="D">4</p><p i="E">11ABAC</p><p i="F">22A48C</p><p i="10">0</p></d></rd>"""
class Request:
def __init__(self, request):
stream = StringIO(request)
request = stream.readline()
words = request.split()
[self.command, self.path, self.version] = words
self.headers = email.message_from_string(request)
self.content = stream.read()
def __getitem__(self, key):
return self.headers.get(key, '')
r = Request(request)
print(r.command)
print(r.path)
print(r.version)
for header in r.headers:
print(header, r[header])
print(r.content)
Iam trying to accsess OAuth2 in Python (the code is the same as http://code.google.com/p/google-api-ads-python/source/browse/trunk/examples/adspygoogle/adwords/v201302/misc/use_oauth2.py?spec=svn139&r=139):
flow = OAuth2WebServerFlow(client_id='XXX',
client_secret='YYY',
scope='https://adwords.google.com/api/adwords',
user_agent='ZZZ')
authorize_url = flow.step1_get_authorize_url('urn:ietf:wg:oauth:2.0:oob')
code = raw_input('Code: ').strip()
credential = None
try:
credential = flow.step2_exchange(code) #<- error
except FlowExchangeError, e:
sys.exit('Authentication has failed: %s' % e)
This produces a "socket.error: [Errno 10054]" error at the step2_exchange and Python cuts off the excact message.
So after checking the key with OAuthplayground (to get an better errormsg) i get this Error:
HTTP/1.1 400 Bad Request
Content-length: 37
X-xss-protection: 1; mode=block
X-content-type-options: nosniff
X-google-cache-control: remote-fetch
-content-encoding: gzip
Server: GSE
Via: HTTP/1.1 GWA
Pragma: no-cache
Cache-control: no-cache, no-store, max-age=0, must-revalidate
Date: Thu, 06 Jun 2013 13:54:29 GMT
X-frame-options: SAMEORIGIN
Content-type: application/json
Expires: Fri, 01 Jan 1990 00:00:00 GMT
{
"error" : "unauthorized_client"
}
I checked that client_id (for installed Apps) and client_secret are Identical with the one specified in the Google API Console (https://code.google.com/apis/console/).
If i do the whole proces over OAuthPlayground it will work but if i try to use a Token created by playground the App fails also.
Anyone knows how to fix it?
Fixed it. Iam behind a proxy which makes lets the step1 Auth through but apparently not the step2 auth. So a simple
h = httplib2.Http(proxy_info = httplib2.ProxyInfo PROXY DATA .....)
flow.step2_exchange(code, h)
fixed it.
There is an example of how to configure the proxy_info in httplib2 is in https://code.google.com/p/httplib2/wiki/Examples
which says:
import httplib2
import socks
httplib2.debuglevel=4
h = httplib2.Http(proxy_info = httplib2.ProxyInfo(socks.PROXY_TYPE_HTTP, 'localhost', 8000))
r,c = h.request("http://bitworking.org/news/")
However, I've found with the latest httplib2, a cleaned-up socks module comes with it, such that you really want to do something more like:
import httplib2
ht = httplib2.Http(proxy_info = httplib2.ProxyInfo(httplib2.socks.PROXY_TYPE_HTTP, 'name_or_ip_of_the_proxy_box', proxy_port)
flow.step2_exchange(code, ht)
also, you want to be using a version of oauth2client >= 1.0beta8 which requires the version of httplib2 >= 0.7.4 which is where the support for HTTP proxy was cleaned up in both packages.
I have the following pycurl code:
curl = pycurl.Curl()
foo = StringIO()
curl.setopt(pycurl.WRITEFUNCTION, foo.write)
curl.setopt(pycurl.POST, 1)
curl.setopt(pycurl.URL, finalURL)
curl.setopt(pycurl.POSTFIELDS, encodedArgs)
curl.perform()
responseCode = curl.getinfo(pycurl.RESPONSE_CODE)
effectiveURL = curl.getinfo(pycurl.EFFECTIVE_URL)
curl.close()
When the command line curl command comes back I see:
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: text/xml;charset=UTF-8
Content-Length: 216
Date: Thu, 06 Jan 2011 15:49:36 GMT
Some XML Error Here: Something you are trying to do is not permitted.
But I don't see this from pycurl.
How can I extract this alert/error message when using pycurl?
The response from the server is written using the curl option pycurl.WRITEFUNCTION.
In your case, since you are passing it a StringIO object, the response data should be in the foo variable: foo.getvalue()
Reference: http://pycurl.sourceforge.net/doc/curlobject.html