I'm getting the following error when trying to fetch an URL with urllib2 in the google app engine:
error: An error occured while connecting to the server: Unable to fetch URL: http://www.google.com Error: [Errno 10106] getaddrinfo failed
This is the code calling the urllib2 open read methods:
def get(self):
self.write(urllib2.urlopen("http://www.google.com").read())
self.render_index()
Nothing fancy, just a call to the library inside the main handler to ouptut the fetched text.
My PC resolves DNS correctly. I can use the urllib2 library from the python interpeter, fetching URLs successfully.
The deployed code running from the google servers work as intended, it's something with my local environment but I can't find what is it.
I also tried using urlfetch from gae with similar results (same getaddrinfo failed)
I switched to Google DNS some days before working with the urllib2 library but switching back to ISP provided DNS didn't work either.
EDIT: When calling the function with an IP address the URL is fetched:
self.write(urllib2.urlopen("http://173.194.42.34").read())
Thanks in advance!
I'm fairly certain that your DNS resolver fails to resolve the hostname. I assume that your OS, or security software prohibits the devserver from creating outbound connection. Another possibility would be that you have invalid entry in your hosts file on your operating system.
Also, there are many similar questions which could help you.
Related
When I am running a Python micro-service in a dockerized or kubernetes container it works just fine. But with Istio service mesh, it is not working.
I have added ServiceEntry for two of my outbound external http apis. It seems I can access the url content form inside the container using curl command which is inside service mesh. So, I think the service entries are fine and working.
But when I try from the micro-service which uses xml.sax parser in Python, it gives me the upstream connect error or disconnect/reset before headers though the same application works fine without Istio.
I think it is something related to Istio or Envoy or Python.
Update: I did inject the Istio-proxy side-car. I have also added ServiceEntry for external MySQL database and mysql is connected from the micro-service.
I have found the reason for this not working. My Python service is using xml.sax parser library to parse xml form the internet, which is using the legacy urllib package which initiate http/1.0 request.
Envoy doesn't support http/1.0 protocol version. Hence, it is not working. I made the workaround by setting global.proxy.includeIPRanges="10.x.0.1/16" for Istio using helm. This actually bypass the entire envoy proxy for all outgoing connections outside the given ip ranges.
But I would prefer not to globally bypass Istio.
The code below:
import urllib2
file = urllib2.urlopen("http://foo.bar.com:82")
works just fine on my mac (OS X 10.8.4 running Python 2.7.1. It opens the URL and I can parse the file with no problems.
When I try the EXACT same code (these two lines) in GoDaddy Python 2.7.3 (or 2.4) I receive an error:
urllib2.URLError: <urlopen error (111, 'Connection refused')
The problem has something to do with the port :82 that is an essential part of the address. I have tried using a forwarding address with masking, etc., and nothing works.
Any idea why it would work in one environment and not in the other (ostensibly similar) environment? Any ideas how to get around this? I also tried Mechanize to no avail. Previous posts have suggested focusing on urllib2.HTTPBasicAuthHandler, but it works fine on my OS X environment without anything special.
Ideas are welcome.
Connection refused means that your operating system tried to contact the remote host, but got a "closed port" message.
Most likely, this is because of a firewall between GoDaddy and foo.bar.com. Most likely, foo.bar.com is only reachable from your computer or your local network, but it also could be GoDaddy preventing access to strange ports.
From a quick look at the GoDaddy support forums, it looks like they only support outgoing requests to ports 80 (HTTP) and 443 (HTTPS) on their shared hosts. See e.g.
http://support.godaddy.com/groups/web-hosting/forum/topic/curl-to-ports-other-than-80/
I'm getting the following error when trying to fetch an URL with urllib2 in the google app engine:
error: An error occured while connecting to the server: Unable to fetch URL: http://www.google.com Error: [Errno 10106] getaddrinfo failed
This is the code calling the urllib2 open read methods:
def get(self):
self.write(urllib2.urlopen("http://www.google.com").read())
self.render_index()
Nothing fancy, just a call to the library inside the main handler to ouptut the fetched text.
My PC resolves DNS correctly. I can use the urllib2 library from the python interpeter, fetching URLs successfully.
The deployed code running from the google servers work as intended, it's something with my local environment but I can't find what is it.
I also tried using urlfetch from gae with similar results (same getaddrinfo failed)
I switched to Google DNS some days before working with the urllib2 library but switching back to ISP provided DNS didn't work either.
EDIT: When calling the function with an IP address the URL is fetched:
self.write(urllib2.urlopen("http://173.194.42.34").read())
Thanks in advance!
I'm fairly certain that your DNS resolver fails to resolve the hostname. I assume that your OS, or security software prohibits the devserver from creating outbound connection. Another possibility would be that you have invalid entry in your hosts file on your operating system.
Also, there are many similar questions which could help you.
I am trying to get access to a local JIRA 4.4.5 installation using it's JSON-RPC service. Therefore I wrote a python script utilizing jsonrpclib and trying to connect to http://localhost:8080/jira/rpc/json-rpc/jirasoapservice-v2 as described on https://developer.atlassian.com/display/JIRADEV/JIRA+JSON-RPC+Overview. Trying to connect from my python script as well as opening this URL in a browser gives me a 404 error.
import jsonrpclib
server = jsonrpclib.Server("http://localhost:8080/jira/rpc/json-rpc/jirasoapservice-v2")
reply = server.someMethod( param1, ... )
Calling someMethod fails with the following error:
xmlrpclib.ProtocolError: <ProtocolError for localhost:8080/jira/rpc/json-rpc/jirasoapservice-v2: 404 Not Found>
Has anyone successfully tried this the same way I did? Do I need to get access via HTTPS somehow instead of HTTP? How would I configure jira to do so?
Btw: Jira's json-rpc plugin is enabled.
I'm having a weird problem. I have this Python application and when I try to open a url in the application, for instance urllib2.urlopen("http://google.com", None) I get the following error:
IOError: [Errno socket error] [Errno 8] nodename nor servname provided, or not known
However when I do the same thing on the python command line interpreter it works fine. The same python executable is being used for both the application and the command line.
nslookup google.com seems to work fine. I opened up wireshark and it looks like when the application tries to open google.com only a mDNS query goes out for "My-Name-MacBook-Pro.local". However, when the command line tries to open google.com a regular DNS query goes out for "google.com" I found if I hardcoded Google's IP in /etc/hosts then the request from the application finally started working.
It seems something weird must be altering how the application resolves domain names, but I have no idea what could be doing this.
I'm running Mac OSX 10.6.7 and Python 2.6.
Edit: I am not using a proxy to access the internet
Just see that you don't have HTTP_PROXY environment variable set which is preventing this. (In which case, that would be a bad error message. Given the proper directory and try again, like
import urllib
r = urlib.urlopen('http://www.google.com')
print r.read()