Develop Reference

urllib.request on Python 3 failed to get certificate

I am writing a program which can read json of ipsw.me in python3:
import urrlib.request as ur
url = "https://api.ipsw.me/v4/device/" + device
json_file = ur.urlopen(url)
with open("signedipsw.json",'wb') as output:
output.write(json_file.read())
data = json.load(open("signedipsw.json"))
i = 0
with open("signedipsw.json"):
print("signed firmwares for %s:" % device)
for i in range(0, len(data["firmwares"])):
if data["firmwares"][i]["signed"] == True :
print("%s - %s" % (data["firmwares"][i]["version"], data["firmwares"][i]["buildid"]))
i+=1
I used to write this in Python2 but I used urrlib2 module and it worked perfectly. Now, on Python3 I use urrlib.request but I received an error:
json_file = ur.urlopen(url)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 214, in urlopen
return opener.open(url, data, timeout)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 517, in open
response = self._open(req, data)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 534, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 494, in _call_chain
result = func(*args)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1385, in https_open
return self.do_open(http.client.HTTPSConnection, req,
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1345, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)>
How can I fix this? Thanks

It looks like urllib (not urrlib) is not able to verify you're talking to the real ipsw.me. They're using Cloudflare's certificates.
You'll have a much better time using the Requests module. It comes with a more recent certificate bundle, which will likely fix your problem.
import requests
device = "..."
resp = requests.get(f"https://api.ipsw.me/v4/device/{device}")
resp.raise_for_status()
data = resp.json()
print(f"signed firmwares for {device}:")
for fw in data["firmwares"]:
if fw["signed"]:
print(f"{fw['version']} - {fw['buildid']}")

Sending SMS through Python suds library

I am trying to send SMS following Python suds procedure but facing some problems related to urllib2
The API URL, username & are provided by mobile service company when i try to execute the code i receive some errors relate to ###urllib2.URLError###
from suds.client import Client
client = Client(url='https://cbs.zong.com.pk/reachcwsv2 /corporatesms.svc?wsdl',
username='9231089894269149',
password='Zo98989ng#123', cache=None)
obj_QuickSMS = client.factory.create('obj_QuickSMS')
obj_QuickSMS.loginId = '91313212304269149'
obj_QuickSMS.loginPassword = 'zqong#435'
obj_QuickSMS.Destination = '923459332997'
obj_QuickSMS.Mask = 'Edex'
obj_QuickSMS.Message = 'HELLO'
obj_QuickSMS.UniCode = '0'
obj_QuickSMS.ShortCodePrefered = 'n'
result = client.service.QuickSMS(obj_QuickSMS)
i am receiving the following error:
File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 1228, in http_open
return self.do_open(httplib.HTTPConnection, req)>
File "/usr/lib/python2.7/urllib2.py", line 1198, in do_open
raise URLError(err)
urllib2.URLError:

You have white space in your url, it should be like this:
client = Client(url='https://cbs.zong.com.pk/reachcwsv2/corporatesms.svc?wsdl',
username='9231089894269149',
password='Zo98989ng#123', cache=None)

Download a picture from certain URL with Python

I learned how to download a picture from a certain URL with python as:
import urllib
imgurl="http://www.digimouth.com/news/media/2011/09/google-logo.jpg"
resource = urllib.urlopen(imgurl)
output = open("test.jpg","wb")
output.write(resource.read())
output.close()
and it worked well, but when i changed the URL to
imgurl="http://farm1.static.flickr.com/96/242125326_607a826afe_o.jpg"
it did not work, and gave the information
File "face_down.py", line 3, in <module>
resource = urllib2.urlopen(imgurl)
File "D:\Python27\another\Lib\urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "D:\Python27\another\Lib\urllib2.py", line 431, in open
response = self._open(req, data)
File "D:\Python27\another\Lib\urllib2.py", line 449, in _open
'_open', req)
File "D:\Python27\another\Lib\urllib2.py", line 409, in _call_chain
result = func(*args)
File "D:\Python27\another\Lib\urllib2.py", line 1227, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "D:\Python27\another\Lib\urllib2.py", line 1197, in do_open
raise URLError(err)
urllib2.URLError: <urlopen error [Errno 10060] >
and I tried to open the latter image URL, and it could be shown as the former, I have no idea to solve it~~ help~~~~

You can try using requests module. The response will be some bytes. So, you can iterate over those byte chunks and write to the file.
import requests
url = "http://farm1.static.flickr.com/96/242125326_607a826afe_o.jpg"
r = requests.get(url)
path = "filename.jpg"
with open(path, 'wb') as f:
for chunk in r:
f.write(chunk)

I looked up both of the addresses and the second one does not lead anywhere. That is probably the problem.
import urllib
imgurl="webpage url"
openimg = urllib.urlopen(imgurl) #opens image (prepares it)
img = open("test.jpg","wb") #opens the img to read it
img.write(openimg.read()) #prints it to the console
img.close() #closes img
Try the link again in your webpage and if it turns up with "webpage not available" that is probably the problem.

Python socket ConnectionResetError: [Errno 54] Connection reset by peer vs socket.error: [Errno 104] Connection reset by peer

I'm having trouble debugging my code because I cannot understand the socket error being raised.
Here is the traceback.
Traceback (most recent call last):
File "clickpression.py", line 517, in <module> presser.main()
File "clickpression.py", line 391, in main
File "clickpression.py", line 121, in clickpress self.refresh_proxies(country=country)
File "clickpression.py", line 458, in refresh_proxies self.proxies = self.get_proxies(country=country)
File "helpers.py", line 72, in wrapper return func(*args, **kwargs)
File "clickpression.py", line 264, in get_proxies self.settings.SUPER_PROXY).read().decode('utf-8')
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 161, in urlopen return opener.open(url, data, timeout)
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 463, in open response = self._open(req, data)
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 481, in _open '_open', req)
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 441, in _call_chain result = func(*args)
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1210, in http_open return self.do_open(http.client.HTTPConnection, req)
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1185, in do_open r = h.getresponse()
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 1171, in getresponse response.begin()
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 351, in begin version, status, reason = self._read_status()
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 313, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socket.py", line 374, in readinto return self._sock.recv_into(b)
ConnectionResetError: [Errno 54] Connection reset by peer
According to the errno library Errno 54 is errno.EXFULL which in the python 3 documentation is explained as exchange full.
To my understanding the Connection reset by peer is Errno 104 i.e errno.ECONNRESET.
So what does errno.EXFULL mean? and why does socket raise the error with a connection reset by peer description instead of exchange full. And or how are the two errors errno.EXFULL and errno.ECONNRESET related?
PS: I read that the errno 54 might be related to http proxy (I'm using a proxy in my code). If so, how?

According to the errno library Errno 54 is errno.EXFULL
Did you determine that by examining errno.errorcode[54]? Anyway - this errno library might be at fault. You could verify the meaning of an error code on your system by looking into errno.h, e. g. with the help of gcc:
gcc -xc -imacros errno.h -Wp,-P -E <(echo ECONNRESET)
Also, the Python documentation says:
To translate a numeric error code to an error message, use
os.strerror().
It may well be that error number 54 is ECONNRESET on your system, and that os.strerror(54) will attest that.
Now that you have verified that os.strerror(54) returns 'Exchange full', I am puzzled why the error number 54 and the error string Connection reset by peer do not match. If that happens on a system with strace or something similar, I would further check which error is returned by the operating system through use of strace -e network on the affected process.
Regarding your question about EXFULL: Its meaning seems somewhat system dependent; e. g. on Linux, EXFULL is returned from only a handful places in the kernel, the only network-related place being in br_if.c concerning network bridges, when no available bridge port number is found (other places are in USB and SCSI drivers).

I tried to use python to crew coin market on OKEX.com using WebSocket,cause the url is an outer address,i used a vpn service provided by us,but it still can work. here is the code an traceback.
from ws4py.client.threadedclient import WebSocketClient
class DummyClient(WebSocketClient):
def opened(self):
# self.send("{'event': 'addChannel', 'channel': 'ok_sub_futureusd_btc_ticker_this_week'}") #发送请求数据格式
# self.send("www.baidu.com")
self.send("{'event':'addChannel','channel':'ok_sub_spot_bch_btc_ticker'}")
def closed(self, code, reason=None):
print("Closed down", code, reason)
#服务器返回消息
def received_message(self, m):
print("recv:", m)
if __name__ == '__main__':
try:
# 服务器连接地址wss://real.okex.com:10440/websocket/okexapi
# ws = DummyClient('wss://real.okcoin.cn:10440/websocket/okcoinapi', protocols=['chat'])
ws = DummyClient('wss://real.okex.com:10440/websocket/okexapi', protocols=['chat'])
ws.connect()
#ws.send("my test...")
ws.run_forever()
except KeyboardInterrupt:
ws.close()

You can try this code to your project:
import ssl
ssl._create_default_https_context = ssl._create_unverified_context
if it not work,make sure the server open TLSv1 support.

Python and proxy - urllib2.URLError: <urlopen error [Errno 110] Connection timed out>

I tried to google and search for similar question on stackOverflow, but still can't solve my problem.
I need my python script to perform http connections via proxy.
Below is my test script:
import urllib2, urllib
proxy = urllib2.ProxyHandler({'http': 'http://255.255.255.255:3128'})
opener = urllib2.build_opener(proxy, urllib2.HTTPHandler)
urllib2.install_opener(opener)
conn = urllib2.urlopen('http://www.whatismyip.com/')
return_str = conn.read()
webpage = open('webpage.html', 'w')
webpage.write(return_str)
webpage.close()
This script works absolutely fine on my local computer (Windows 7, Python 2.7.3), but when I try to run it on the server, it gives me the following error:
Traceback (most recent call last):
File "proxy_auth.py", line 18, in <module>
conn = urllib2.urlopen('http://www.whatismyip.com/')
File "/home/myusername/python/lib/python2.7/urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "/home/myusername/python/lib/python2.7/urllib2.py", line 400, in open
response = self._open(req, data)
File "/home/myusername/python/lib/python2.7/urllib2.py", line 418, in _open
'_open', req)
File "/home/myusername/python/lib/python2.7/urllib2.py", line 378, in _call_chai n
result = func(*args)
File "/home/myusername/python/lib/python2.7/urllib2.py", line 1207, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "/home/myusername/python/lib/python2.7/urllib2.py", line 1177, in do_open
raise URLError(err)
urllib2.URLError: <urlopen error [Errno 110] Connection timed out>
I also tried to use requests library, and got the same error.
# testing request library
r = requests.get('http://www.whatismyip.com/', proxies={'http':'http://255.255.255.255:3128'})
If I don't set proxy, then the program works fine.
# this works fine
conn = urllib2.urlopen('http://www.whatismyip.com/')
I think the problem is that on my shared hosting account it is not possible to set an environment variable for proxy ... or something like that.
Are there any workarounds or alternative approaches that would let me set proxies for http connections? How should I modify my test script?

The problem was in closed ports.
I had to buy a dedicated IP before tech support could open the ports I needed.
Now my script works fine.
Conclusion: when you are on a shared hosting, most ports are probably closed and you will have to contact tech support to open ports.