Develop Reference

Error while making a Python API. Please have a look

https://github.com/ashwinidotx/RIOTGames-API/issues/1
I was trying to create a League of Legends API to get summoner information but I got an error related to the URL. please have a look
here's the error:
$ python main.py
Traceback (most recent call last):
File "main.py", line 9, in <module>
main()
File "main.py", line 5, in main
r=api.get_summoner_by_name('shadyjoker27')
File "/Users/macdonald/Developer/TwitterBot/RiotSumm.py", line 31, in get_summoner_by_name
return self._request(api_url)
File "/Users/macdonald/Developer/TwitterBot/RiotSumm.py", line 21, in _request
params=args
File "/Library/Python/2.7/site-packages/requests/api.py", line 72, in get
return request('get', url, params=params, **kwargs)
File "/Library/Python/2.7/site-packages/requests/api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "/Library/Python/2.7/site-packages/requests/sessions.py", line 508, in request
resp = self.send(prep, **send_kwargs)
File "/Library/Python/2.7/site-packages/requests/sessions.py", line 618, in send
r = adapter.send(request, **kwargs)
File "/Library/Python/2.7/site-packages/requests/adapters.py", line 508, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='euw.api.pvp.net', port=443): Max retries exceeded with url: /api/lol/euw/v1.4/summoner/by-name/shadyjoker27?api_key=RGAPI-3b0be3d6-1f3c-4e6f-9e52-9d9904af2c21 (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x107698a50>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known',))

You are connecting to euw.api.pvp.net/api/lol/euw/v1.4/summoner/by-name/shadyjoker27?api_key=RGAPI-3b0be3d6-1f3c-4e6f-9e52-9d9904af2c21 via HTTPS. The error message is telling you that this connection does not exist on HTTPS. Open your browser and go to: https://euw.api.pvp.net/api/lol/euw/v1.4/summoner/by-name/shadyjoker27?api_key=RGAPI-3b0be3d6-1f3c-4e6f-9e52-9d9904af2c21. You will find that no such connection exists. Before you make any http requests in python, try them in your browser, because if they don't work their, they wont be working in python. In this case the connection you are trying to make might not be available via HTTPS. Try simply making the connection to euw.api.pvp.net/api/lol/euw/v1.4/summoner/by-name/shadyjoker27?api_key=RGAPI-3b0be3d6-1f3c-4e6f-9e52-9d9904af2c21.

Python + self signed cert + HTTP not working but works in C# [duplicate]

I have an application which I'm doing self-signing certificates for using the documentation here.
The application loads that keystore into a jetty config and loads properly and I get a warning of an untrusted certification when browsing to the site.
Then I want to use python to connect to it and I've tried all variations of generating a cert, pem, etc. nothing I do gets me to connect. Simply doing the following -
import requests
requests.get('https://servername:8443', cert=('path\
\to\\cert\\app.cert', '\\\\path\\to\\keystore\\keystore'))
Gives me the following traceback -
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\requests\api.py", line 69, in get
return request('get', url, params=params, **kwargs)
File "C:\Python27\lib\site-packages\requests\api.py", line 50, in request
response = session.request(method=method, url=url, **kwargs)
File "C:\Python27\lib\site-packages\requests\sessions.py", line 465, in request
resp = self.send(prep, **send_kwargs)
File "C:\Python27\lib\site-packages\requests\sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "C:\Python27\lib\site-packages\requests\adapters.py", line 431, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL] PEM lib (_ssl.c:2580)

cert in requests is used for client site certificates, not for the CA store. Use the verify parameter to specify the CA file, e.g.:
requests.get('https://www.example.com', verify='/etc/ssl/certs/ca-certificates.crt')

not able to verify ssl-certificate from client side using any of python requests, httplib/httplib2, urllib/urlib2

this question is duplicate. but i am still asking because i am not finding any solution from given answers. so guys don't make it duplicate, instead help me to solve this problem.
i am running my server as https using bottle(paste). when trying to access from java and postman , it's verifying correctly and response coming.
versions used :
python 2.7.13
requests.version
'2.11.1'
import cryptography
cryptography.version
'2.1.3'
import OpenSSL
OpenSSL.version
'17.3.0'
but unfortunately when trying to access from python modules - it's throwing issue. ?
requests.post(url, verify=_certfile, proxies=proxies) #tried #failed to verify certificate
urllib2.urlopen(url, data=data, timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
cafile=None, capath=None, cadefault=False, context=context) #tried #failed to verify certificate
import httplib2
http = httplib2.Http(disable_ssl_certificate_validation=True)
headers, content = http.request(url, "POST", body=data, headers=headers)#tried #failed to verify certificate
httplib.HTTPSConnection(host='10.201.41.50', port=8018, key_file=key_file, cert_file=cert_file, strict=None, context=context) #tried #failed to verify certificate
and my server is running using paste server using bottle framework.
ssl_cxt = SSL.Context(SSL.SSLv23_METHOD)
...
...
ssl_cxt.set_verify(SSL.VERIFY_PEER | SSL.VERIFY_FAIL_IF_NO_PEER_CERT, verify_cb)
paste.httpserver.serve(app, host=IP, port=PORT, server_version=" ", ssl_context=ssl_cxt)
issues i am facing (different with each lib)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "Continuum\Anaconda2\lib\site-packages\requests\api.py", line 70, in get
return request('get', url, params=params, **kwargs)
File "Continuum\Anaconda2\lib\site-packages\requests\api.py", line 56, in request
return session.request(method=method, url=url, **kwargs)
File "Continuum\Anaconda2\lib\site-packages\requests\sessions.py", line 475, in request
resp = self.send(prep, **send_kwargs)
File "Continuum\Anaconda2\lib\site-packages\requests\sessions.py", line 596, in send
r = adapter.send(request, **kwargs)
File "Continuum\Anaconda2\lib\site-packages\requests\adapters.py", line 497, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: unknown error (_ssl.c:2947)
============
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "Continuum\Anaconda2\lib\site-packages\requests\api.py", line 70, in get
return request('get', url, params=params, **kwargs)
File "Continuum\Anaconda2\lib\site-packages\requests\api.py", line 56, in request
return session.request(method=method, url=url, **kwargs)
File "Continuum\Anaconda2\lib\site-packages\requests\sessions.py", line 475, in request
resp = self.send(prep, **send_kwargs)
File "Continuum\Anaconda2\lib\site-packages\requests\sessions.py", line 596, in send
r = adapter.send(request, **kwargs)
File "Continuum\Anaconda2\lib\site-packages\requests\adapters.py", line 497, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate
verify failed (_ssl.c:661)
File "Continuum\Anaconda2\lib\httplib.py", line 1038, in endheaders
self._send_output(message_body)
File "Continuum\Anaconda2\lib\httplib.py", line 882, in _send_output
self.send(msg)
File "Continuum\Anaconda2\lib\httplib.py", line 844, in send
self.connect()
File "Continuum\Anaconda2\lib\httplib.py", line 1262, in connect
self.sock = self._context.wrap_socket(self.sock,
AttributeError: 'Context' object has no attribute 'wrap_socket'

I tried to solve using urllib2, socket and ssl. i am able to hit the server. i tried with .pkcs12 but seems urllib2 expects only .pem files. so will advice try to create client certificate with .pem extension specifically when you try to access from python.
import ssl
import socket
import urllib2
import json
url = "https://localhost:port/hello/"
_certfile_pem = "ssl_client.pem"
key_file = "ssl_key.pem"
cert_file = "ssl_cert.pem"
headers = {"Content-Type":"application/json"}
data = {}
context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
context.verify_mode = ssl.CERT_REQUIRED
context.load_cert_chain(certfile=cert_file, keyfile=key_file)
context.load_verify_locations(_certfile_pem)
context.wrap_socket(socket.socket(socket.AF_INET, socket.SOCK_STREAM)) #
socket overwrite required.
try:
req = urllib2.Request(url,data=json.dumps(data))
response = urllib2.urlopen(req, context=context)
content = response.read()
print content
except urllib2.HTTPError as e:
content = e.read()
print "HTTPError : ", e

requests getting SSLError due to certificate verify failed

I am using Python 2.7.3, Requests version 2.7.0. I have setup a simple HTTPS server using a self signed certificate. I can run the following script on my Windows fine. But I am unable to do the same on an embedded device. On the embedded device I have no problem with http get or https with verify=False.
import requests
url = 'https://192.168.3.4:8002'
requests.get(url, verify='/home/root/server.crt')
The error statement:
Traceback (most recent call last):
File "test.py", line 3, in <module>
requests.get(url, verify='/home/root/server.crt')
File "/usr/lib/python2.7/site-packages/requests/api.py", line 69, in get
return request('get', url, params=params, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/api.py", line 50, in request
response = session.request(method=method, url=url, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 465, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 431, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [Errno bad handshake] [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]
Does anyone have any idea?

How can I login through 'https' login page using python3.4?

I'm trying to login certain page with python3.4 requests, bs4 module.
When I'm trying to login with below code, ssl errors occurs :
import requests
userid = "myid"
password = "mypassword"
loginURL = "https://example.com/member/login"
params = {'userid': userid, 'password':password}
res = requests.post(loginURL, params=params, verify=True)
Errors:
choejeong-us-MacBook-Pro:Desktop kris$ python a.py
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 559, in urlopen
body=body, headers=headers)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 345, in _make_request
self._validate_conn(conn)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 782, in _validate_conn
conn.connect()
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 250, in connect
ssl_version=resolved_ssl_version)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/util/ssl_.py", line 285, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ssl.py", line 365, in wrap_socket
_context=self)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ssl.py", line 583, in __init__
self.do_handshake()
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ssl.py", line 810, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/adapters.py", line 370, in send
timeout=timeout
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 588, in urlopen
raise SSLError(e)
requests.packages.urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "a.py", line 10, in <module>
res = requests.post(loginURL, params=params, verify=True)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/api.py", line 109, in post
return request('post', url, data=data, json=json, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/api.py", line 50, in request
response = session.request(method=method, url=url, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/sessions.py", line 468, in request
resp = self.send(prep, **send_kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/sessions.py", line 576, in send
r = adapter.send(request, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/adapters.py", line 433, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)
How can I login the site with ssl page?.
I'm looked some books about python requests module and web scraping, there are no books dealing this problem.
p.s I wonder if python is really strong enough to deal with 'all' web pages ? I mean, Can I crawl all sites in the web? or there are some limitations?

The last line of the traceback tells you why:
requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)
requests failed to verify the server's SSL certificate. You can disable cert verification with the verify option:
res = requests.post(loginURL, params=params, verify=False)
There are security implications in doing this, so whether you should do that is a matter for you to decide based on your application's requirements.