I have a daemon which uploads json to http://localhost:9000, I can read json easily using hapi with this code:
'use strict';
const Hapi = require('hapi');
const server = new Hapi.Server();
server.connection({
host: 'localhost',
port: 9000
});
server.route({
method: 'POST',
path:'/push/',
handler: function (request, reply) {
console.log('Server running at:', request.payload);
return reply('hello world');
}
});
server.start((err) => {
if (err) {
throw err;
}
console.log('Server running at:', server.info.uri);
});
but I'm trying to handle this in python, I found a answer here, so here is my code:
import urllib, json
url = 'http://localhost:9000/push/'
response = urllib.urlopen(url)
data = json.load(response.read())
print data
but when I run it I get Exception:
Traceback (most recent call last):
File "pythonServer.py", line 3, in <module>
response = urllib.urlopen(url)
File "/usr/lib/python2.7/urllib.py", line 87, in urlopen
return opener.open(url)
File "/usr/lib/python2.7/urllib.py", line 213, in open
return getattr(self, name)(url)
File "/usr/lib/python2.7/urllib.py", line 350, in open_http
h.endheaders(data)
File "/usr/lib/python2.7/httplib.py", line 997, in endheaders
self._send_output(message_body)
File "/usr/lib/python2.7/httplib.py", line 850, in _send_output
self.send(msg)
File "/usr/lib/python2.7/httplib.py", line 812, in send
self.connect()
File "/usr/lib/python2.7/httplib.py", line 793, in connect
self.timeout, self.source_address)
File "/usr/lib/python2.7/socket.py", line 571, in create_connection
raise err
IOError: [Errno socket error] [Errno 111] Connection refused
How can I solve this?
Your server route is expecting a POST request.
Your Python code is sending a GET request.
Notice that your JavaScript code is using the following:
method: 'POST'
Your Python code by contrast will be default to a GET method, since you didn't explicitly specify it. You can look here for how to make a POST request using urllib, but I would recommend you look into using the requests library instead.
It's a simple as
import requests
r = requests.post("http://localhost:9000/push/", data = {"key":"value"})
Alternatively, just change your server (JavaScript/Node.js code) to use method: 'GET'.
Your server is sending down POST request and you are fetching a GET call.
response = urllib.urlopen(url)
urllib.urlopen opens the network URL for reading.
Instead of sending a GET request you can send POST request to server.
Here is the complete code:
import urllib, json, urllib2
url = 'http://localhost:9000/push/'
# This packages the request (it doesn't make it)
request = urllib2.Request(url)
# Sends the request and catches the response
response = urllib2.urlopen(request)
data = json.load(response.read())
print data
Related
I have the code below which works fine and brings back what I need
import requests
from requests.auth import HTTPBasicAuth
response = requests.get('https://example/answers/331', auth=HTTPBasicAuth('username', 'password'),json={"solution": "12345"})
print response.content
However when I change it to a patch method, which is accepted by the server, I get the following errors. Any idea on why?
Traceback (most recent call last):
File "auth.py", line 8, in <module>
response = requests.patch('https://example/answers/331', auth=HTTPBasicAuth('username', 'password'),json={"solution": "12345"})
File "C:\Python27\lib\site-packages\requests-2.12.0-py2.7.egg\requests\api.py", line 138, in patch
return request('patch', url, data=data, **kwargs)
File "C:\Python27\lib\site-packages\requests-2.12.0-py2.7.egg\requests\api.py", line 56, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Python27\lib\site-packages\requests-2.12.0-py2.7.egg\requests\sessions.py", line 488, in request
resp = self.send(prep, **send_kwargs)
File "C:\Python27\lib\site-packages\requests-2.12.0-py2.7.egg\requests\sessions.py", line 609, in send
r = adapter.send(request, **kwargs)
File "C:\Python27\lib\site-packages\requests-2.12.0-py2.7.egg\requests\adapters.py", line 473, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine("''",))
Thanks
Try using a POST request with the following header: X-HTTP-Method-Override: PATCH
This is unique to the Oracle Service Cloud REST API implementation and is documented.
In cases where the browser or client application does not support PATCH requests, or network intermediaries block PATCH requests, HTTP tunneling can be used with a POST request by supplying an X-HTTP-Method-Override header.
Example:
import requests
restURL = <Your REST URL>
params = {'field': 'val'}
headers = {'X-HTTP-Method-Override':'PATCH'}
try:
resp = requests.post(restURL, json=params, auth=('<uname>', '<pwd>'), headers=headers)
print resp
except requests.exceptions.RequestException as err:
errMsg = "Error: %s" % err
print errMsg
I'm trying to make a POST request script in Python which will send data to database automatically. Since my web application (made in Django) is still in progress (on localhost) I tried to send this POST request from another computer on same network via IPv4:port.
While doing this I get HTTP Error 500: Internal server error with traceback to the line "response = urllib2.urlopen(req)"
My script looks like this:
import urllib, urllib2, cookielib
import re
cookie_jar = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookie_jar))
urllib2.install_opener(opener)
url_1 = 'http://192.168.1.104:8000/legacy'
req = urllib2.Request(url_1)
rsp = urllib2.urlopen(req)
url_2 = 'http://192.168.1.104:8000/legacy'
params = {
'temperature_max' : '186',
'temperature_min' : '88',
'submit': 'Submit',
}
data = urllib.urlencode(params)
req = urllib2.Request(url_2, data)
response = urllib2.urlopen(req)
the_page = response.read()
pat = re.compile('Title:.*')
print pat.search(the_page).group()
On the other computer that is hosting the server I get the following error:
Exception happened during processing of request from ('192.168.1.65', 56996)
Traceback (most recent call last):
File "c:\python27\Lib\SocketServer.py", line 599, in process_request_thread
self.finish_request(request, client_address)
File "c:\python27\Lib\SocketServer.py", line 334, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "C:\Users\Alex\Envs\rango\lib\site-packages\django\core\servers\basehttp.
py", line 129, in __init__
super(WSGIRequestHandler, self).__init__(*args, **kwargs)
File "c:\python27\Lib\SocketServer.py", line 657, in __init__
self.finish()
File "c:\python27\Lib\SocketServer.py", line 716, in finish
self.wfile.close()
File "c:\python27\Lib\socket.py", line 283, in close
self.flush()
File "c:\python27\Lib\socket.py", line 307, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 10054] An existing connection was forcibly closed by the remote host
EDIT: I'd like to let you know that I can post data from the other computer to my database if I connect to my app with my browser.
You will need to get a session cookie and then read the generated cookie for that session.
The best way to go about that is to fetch the page with the form first, saving all cookies. You can use cookielib for that as demonstrated here. If you want to make life easier for yourself, use requests as well instead of urllib. The code then becomes a lot simpler.
To get the CSRF token, you can scrape the form page with BeautifulSoup. There are lots of tutorials for this which you can easily find on Google.
Try:
import urllib.request
from urllib.error import HTTPError
url = "<enter_URL_here>"
try:
res = urllib.request.urlopen(url)
except HTTPError as e:
content = e.read()
print("HTTP error encountered while reading URL")
I am trying to develop a small tiny application using flask and requests module. I tried to post some data to a flask web application. But I stuck at this error.
flask.py
#app.route('/add/', methods=['POST'])
def add_paths():
paths = request.form['paths']
tags = 'others'
for path in paths:
g.db.execute('insert into entries (path, tags) values(?, ?)',
[path, tags])
g.db.commit()
message = 'New paths are posted'
return jsonify(result = message)
command line file for posting the data
import json
import glob2
import requests
list = glob2.glob('/home/sheeshmohsin/*/**')
post_data = {'paths' : list }
headers = {'content-type': 'application/json'}
post_response = requests.post(url='http://localhost:9696/add/', data=json.dumps(post_data), headers=headers)
print post_response
print post_response.text
And the error i am getting is:-
File "commandline.py", line 8, in <module>
post_response = requests.post(url='http://127.0.0.1:9696/add/', data=json.dumps(post_data), headers=headers)
File "/usr/lib/python2.7/site-packages/requests/api.py", line 88, in post
return request('post', url, data=data, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 335, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 438, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 327, in send
raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=9696): Max retries exceeded with url: /add/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused)
The connection refused error message indicates it could be your firewall, or loopback:
Check that firewall isn't blocking port 9696.
I've had the loopback (127.0.0.1) cause similar issues, so replace localhost by actual IP address so that Python doesn't use loopback interface (requests.post(url='http://localhost:9696/add/',...). You might have to do same with Flask (app..run(host='192.168...', port=9696)).
I'm using urllib3 against private services that have self signed certificates. Is there any way to have urllib3 ignore the certificate errors and make the request anyways?
import urllib3
c = urllib3.HTTPSConnectionPool('10.0.3.168', port=9001)
c.request('GET', '/')
When using the following:
import urllib3
c = urllib3.HTTPSConnectionPool('10.0.3.168', port=9001, cert_reqs='CERT_NONE')
c.request('GET', '/')
The following error is raised:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3/dist-packages/urllib3/request.py", line 67, in request
**urlopen_kw)
File "/usr/lib/python3/dist-packages/urllib3/request.py", line 80, in request_encode_url
return self.urlopen(method, url, **urlopen_kw)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 415, in urlopen
body=body, headers=headers)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 267, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.3/http/client.py", line 1061, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python3.3/http/client.py", line 1099, in _send_request
self.endheaders(body)
File "/usr/lib/python3.3/http/client.py", line 1057, in endheaders
self._send_output(message_body)
File "/usr/lib/python3.3/http/client.py", line 902, in _send_output
self.send(msg)
File "/usr/lib/python3.3/http/client.py", line 840, in send
self.connect()
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 103, in connect
match_hostname(self.sock.getpeercert(), self.host)
File "/usr/lib/python3/dist-packages/urllib3/packages/ssl_match_hostname/__init__.py", line 32, in match_hostname
raise ValueError("empty or no certificate")
ValueError: empty or no certificate
Using cURL I'm able to get the expected response from the service
$ curl -k https://10.0.3.168:9001/
Please read the documentation for API endpoints
Try following code:
import urllib3
c = urllib3.HTTPSConnectionPool('10.0.3.168', port=9001, cert_reqs='CERT_NONE',
assert_hostname=False)
c.request('GET', '/')
See Setting assert_hostname to False will disable SSL hostname verification
In this question I see many answers but, IMHO, too much unnecessary information that can lead to confusion.
Just add the cert_reqs='CERT_NONE' parameter
import urllib3
http = urllib3.PoolManager(cert_reqs='CERT_NONE')
I found the answer to my problem. The urllib3 documentation does not, in fact, completely explain how to suppress SSL certificate validation. What is missing is a reference to ssl.CERT_NONE.
My code has a boolean, ssl_verify, to indicate whether or not I want SSL validation. The code now looks like this:
import ssl
import urllib3
#
#
#
if (ssl_verify):
cert_reqs = ssl.CERT_REQUIRED
else:
cert_reqs = ssl.CERT_NONE
urllib3.disable_warnings()
http = urllib3.PoolManager(cert_reqs = cert_reqs)
auth_url = f'https://{fmc_ip}/api/fmc_platform/v1/auth/generatetoken'
type = {'Content-Type': 'application/json'}
auth = urllib3.make_headers(basic_auth=f'{username}:{password}')
headers = { **type, **auth }
resp = http.request('POST',
auth_url,
headers=headers,
timeout=10.0)
Try to instanciate your connection pool this way:
HTTPSConnectionPool(self.host, self.port, cert_reqs=ssl.CERT_NONE)
or this way:
HTTPSConnectionPool(self.host, self.port, cert_reqs='CERT_NONE')
Source: https://github.com/shazow/urllib3/blob/master/test/with_dummyserver/test_https.py
EDIT (after seeing your edit):
It looks like the remote host didn't send a certificate (is it possible?).
This is the code (from urllib3) which raised an exception:
def match_hostname(cert, hostname):
"""Verify that *cert* (in decoded format as returned by
SSLSocket.getpeercert()) matches the *hostname*. RFC 2818 rules
are mostly followed, but IP addresses are not accepted for *hostname*.
CertificateError is raised on failure. On success, the function
returns nothing.
"""
if not cert:
raise ValueError("empty or no certificate")
So it looks like cert is empty, which means that self.sock.getpeercert() returned an empty string.
I've setup Kannel in Ubuntu using a USB Modem and I can send SMS via the browser using the URL as seen below
localhost:13013/cgi-bin/sendsms?username=kannel&password=kannel&to=+254781923855&text='Kid got swag'
In python, I have the following script which works only if the message to be sent does not have spaces.
import urllib.request
def send_sms(mobile_no, message):
url="http://%s:%d/cgi-bin/sendsms?username=%s&password=%s&to=%s&text=%s" \
% ('localhost', 13013, 'kannel', 'kannel', str(mobile_no), message)
f = urllib.request.urlopen(url)
print("sms sent")
If I call the function with NO spaces in the message, it works and the message is sent.
sms.send_sms('+254781923855', 'kid_got_swag')
If I have spaces in the message, it fails with the error belw
sms.send_sms('+254781923855', 'kid got swag')
Traceback (most recent call last):
File "/home/lukik/workspace/projx/src/short_message.py", line 24, in <module>
sms.send_sms('+254781923855', 'kid got swag')
File "/home/lukik/workspace/projx/src/short_message.py", line 18, in send_sms
f = urllib.request.urlopen(url)
File "/usr/lib/python3.2/urllib/request.py", line 139, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.2/urllib/request.py", line 376, in open
response = meth(req, response)
File "/usr/lib/python3.2/urllib/request.py", line 488, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.2/urllib/request.py", line 414, in error
return self._call_chain(*args)
File "/usr/lib/python3.2/urllib/request.py", line 348, in _call_chain
result = func(*args)
File "/usr/lib/python3.2/urllib/request.py", line 496, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 400: Bad Request
I've tried other variants of calling urllib but they all fail coz of the spaces in the message....
In your request you send via browser, the message is inside quotes -
&text='Kid got swag'
Try that in your request -
url="http://%s:%d/cgi-bin/sendsms?username=%s&password=%s&to=%s&text='%s'" \
% ('localhost', 13013, 'kannel', 'kannel', str(mobile_no), message)
Notice the single quotes at &text='%s'.
PS: I'd recommend using requests for requests like this. You could construct your urls better that way, like this -
>>> payload = {'key1': 'value1', 'key2': 'value2'}
>>> r = requests.get("http://httpbin.org/get", params=payload)
URLs are not permitted to contain spaces. When you tried in your browser, the browser took care of correctly encoding the URL before issuing the request. In your program you need to encode the URL. Fortunately urllib has functions built-in to take careof the details.
http://docs.python.org/3.3/library/urllib.parse.html#url-quoting
You need to URL-encode the values you pass as parameters, otherwise a broken URL gets constructed, that's why the request fails. I believe urllib.parse.urlencode does what you need.