I want to write the Python script to download from S3 bucket using Boto3. My code is working fine when not using proxy, but I have to run this under proxy. I have set proxy in the Boto3.client(config=) and I get the following error.
Proxy set as:
s3 = boto3.client('s3',
aws_access_key_id = usrkey,
aws_secret_access_key = sctkey,
region_name = 'eu-west-2',
config=Config(proxies={'http': '<proxy-server>:<port>'})
)
still end up with the error
Traceback (most recent call last):
File "C:\Users\Hello\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\connection.py", line 159, in _new_conn
conn = connection.create_connection(
File "C:\Users\Hello\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\util\connection.py", line 61, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "C:\Users\Hello\AppData\Local\Programs\Python\Python38-32\lib\socket.py", line 918, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 11001] getaddrinfo failed
During handling of the above exception, another exception occurred:
and at the end:
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL:"<URL>".
I can connect using WinSCP to AWS-S3, with proxy, that is working fine. But it pops up a warning message that I have to accept every time, by clicking ok. What is the problem here, I don't understand with Python.
Make sure your <proxy-server> is a host name:
config=Config(proxies={'http': 'example.com:port'}
Not a URL:
config=Config(proxies={'http': 'http://example.com:port'}
Related
I've built an API (with flask-restful) that stores data in its cache and exposes it to other applications. When I try to send a get request to this API from another app (also flask) it returns the following error
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/urllib3/connection.py", line 170, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "/usr/local/lib/python3.6/dist-packages/urllib3/util/connection.py", line 73, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/usr/lib/python3.6/socket.py", line 745, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 706, in urlopen
chunked=chunked,
File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 382, in _make_request
self._validate_conn(conn)
File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
conn.connect()
File "/usr/local/lib/python3.6/dist-packages/urllib3/connection.py", line 353, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.6/dist-packages/urllib3/connection.py", line 182, in _new_conn
self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f6f965d9358>: Failed to establish a new connection: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/urllib3/util/retry.py", line 574, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='data-collector.cloud', port=443): Max retries exceeded with url: /sample_url (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f6f965d9358>: Failed to establish a new connection: [Errno -2] Name or service not known',))
I thought that this error occurred because I was sending too many requests with the same url to the API. I had then limited the number of API calls by adding
decorators = [index.limiter.limit("60/minute")] to the API. The error still persisted however. Then I thought the error might be caused by the amount of calls accepted by the server. I thought I was not closing the connection properly after making an API call. So I added
from requests.packages.urllib3 import Retry, PoolManager
retries = Retry(connect=5, read=2, redirect=5)
with PoolManager(retries=retries) as http:
response = http.request('GET', url)
But this also did not solve my issue. What am I missing here? I am using python3.8
EDIT: I found out that it's not per se the query that is causing it, because if I try other queries, then the same message pops up. I'm still lost at how to debug this :/
The error says: "Name or service not known". In other words, it's a name resolution issue (DNS).
Cross check the target host ("data-collector.cloud", which isn't in the public DNS records.)
I am trying to connect to SFTP server but it returns error:
[Errno 11001] getaddrinfo failed
I am using Python 3.7.3 and Paramiko version is 2.6.0
import paramiko
host_name = "sftp://81.149.151.143"
user_name = "******"
password = "******"
ssh_client = paramiko.SSHClient()
ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh_client.connect(hostname=host_name, port=220, username=user_name, password=password)
ftp_client=ssh_client.open_sftp()
ftp_client.put('***/issue_1.docx', '/issue_1.docx')
ftp_client.close()
This is the full error:
Traceback (most recent call last):
File "/sftp/paramiko_bot.py", line 10, in <module>
ssh_client.connect(hostname=host_name, port=22, username=user_name, password=password)
File "\Local\Programs\Python\Python37\lib\site-packages\paramiko\client.py", line 340, in connect
to_try = list(self._families_and_addresses(hostname, port))
File "\Local\Programs\Python\Python37\lib\site-packages\paramiko\client.py", line 204, in _families_and_addresses
hostname, port, socket.AF_UNSPEC, socket.SOCK_STREAM
File "\AppData\Local\Programs\Python\Python37\lib\socket.py", line 748, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 11001] getaddrinfo failed
The hostname parameter of SSHClient.connect should contain a hostname only (or in your case an IP address) – not any kind of URL.
ssh_client.connect(hostname="81.149.151.143", port=220, username=..., password=...)
Obligatory warning: Do not use AutoAddPolicy this way – You are losing a protection against MITM attacks by doing so. For a correct solution, see Paramiko "Unknown Server".
I'm trying to connect to my server using SSH with port 2022 (not 22) in Python. So I wrote the following code that uses Paramiko package:
import sys
import paramiko
try:
client = paramiko.SSHClient()
client.load_system_host_keys()
client.set_missing_host_key_policy(paramiko.WarningPolicy)
client.connect('ccap#10.40.2.222', '2022', '', 'ccap')
finally:
client.close()
But when I'm running it in my IDE (PyCharm) I get the following error:
/usr/local/lib/python3.5/dist-packages/paramiko/ecdsakey.py:164: CryptographyDeprecationWarning: Support for unsafe construction of public numbers from encoded data will be removed in a future version. Please use EllipticCurvePublicKey.from_encoded_point
self.ecdsa_curve.curve_class(), pointinfo
Traceback (most recent call last):
File "/home/mshapirs/PycharmProjects/OnlineTest.py/OnlineTest.py", line 9, in
client.connect('ccap#10.40.2.222', '2022', '', 'ccap')
File "/usr/local/lib/python3.5/dist-packages/paramiko/client.py", line 334, in connect
to_try = list(self._families_and_addresses(hostname, port))
File "/usr/local/lib/python3.5/dist-packages/paramiko/client.py", line 204, in _families_and_addresses
hostname, port, socket.AF_UNSPEC, socket.SOCK_STREAM
File "/usr/lib/python3.5/socket.py", line 733, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
You should provide username as a separate parameter, not prepended to the host address.
Look at the docs for .connect. It has username and hostnamelisted separately.
i get the following error when trying a script thaat sends mail
import urllib.request
import re
import smtplib
from email.mime.text import MIMEText
from bs4 import BeautifulSoup
page=urllib.request.urlopen("http://www.crummy.com/")
soup=BeautifulSoup(page)
v=soup.findAll('a',href=re.compile('http://www.crummy.com/2012/07/24/0'))
for link in v:
w=link.get('href')
server = smtplib.SMTP( "smtp.gmail.com", 587 )
server.starttls()
server.login( 'xxxxxxxxxxx', 'xxxxxxx' )
server.sendmail( 'xxxxxxxxx', 'xxxxxxxxx', "bonus question is up" )
Traceback (most recent call last): File "C:\Python32\bonus", line 14,
in server = smtplib.SMTP( "smtp.gmail.com", 587 ) File
"C:\Python32\lib\smtplib.py", line 259, in init File
"C:\Python32\lib\smtplib.py", line 319, in connect
self.sock = self._get_socket(host, port, self.timeout) File "C:\Python32 \lib\smtplib.py", line 294, in _get_socket
return socket.create_connection((host, port), timeout) File "C:\Python32\lib\socket.py", line 386, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM): socket.gaierror: [Errno 11004] getaddrinfo failed plse advice on the
best way to go round it
The getaddrinfo function has this purpose:
The getaddrinfo function provides protocol-independent translation from an ANSI host name to an address.
If it fails it means that it cannot translate your given hostname to it's respective address. It's essentially doing a DNS query.
Your error number returned "11004" by getaddrinfo has this message associated with it:
Valid name, no data record of requested type. The requested name is
valid and was found in the database, but it does not have the correct
associated data being resolved for. The usual example for this is a
host name-to-address translation attempt (using gethostbyname or
WSAAsyncGetHostByName) which uses the DNS (Domain Name Server). An MX
record is returned but no A record—indicating the host itself exists,
but is not directly reachable.
It seems the name you're looking up has no correct data associated with it.
Are you sure your URL's are correct?
Links:
getaddrinfo: http://msdn.microsoft.com/en-us/library/windows/desktop/ms738520(v=vs.85).aspx
WinSock Error Codes: http://msdn.microsoft.com/en-us/library/windows/desktop/ms740668(v=vs.85).aspx
I am using poplib to get email from the POP3 server.
But this error occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python26\lib\site-packages\myutils.py", line 251, in dxDown
m=poplib.POP3('pop3.126.com')
File "C:\Python26\lib\poplib.py", line 83, in __init__
self.sock = socket.create_connection((host, port), timeout)
File "C:\Python26\lib\socket.py", line 500, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
socket.gaierror: [Errno 11004] getaddrinfo failed
My laptop is in an local network and using a server(ip 192.168.0.1:8080) as proxy to access internet. The error seems poplib cannot interpret the domain "pop3.126.com". How to solve this problem?Thanks!
Your proxy is for http, it doesn't effect the pop3 traffic.
A cursory glance suggests that it's probably not able to resolve the hostname to an IP address.
Can you try one of these:
pop3.126.idns.yeah.net
220.181.15.128
Or paste the output of:
nslookup pop3.126.com