Develop Reference

"WinError 10061" When trying to connect to Cloud SQL using Cloud SQL Python Connector

I have created a Cloud SQL instance, and a new database. However, I can't seem to connect to the database using Cloud SQL Python Connector. I have followed the sample code and steps in the documentation, but still failed.
Error:
Traceback (most recent call last):
File "c:\Users\my_name\my_path\gl_cloudsql_mysql_fx.py", line 149, in <module>
print(bnm_data_db.isTableExist('MY_TABLE'))
File "c:\Users\my_name\my_path\gl_cloudsql_mysql_fx.py", line 81, in isTableExist
with self.__pool.connect() as db_conn:
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\engine\base.py", line 3245, in connect
return self._connection_cls(self)
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\engine\base.py", line 145, in __init__
self._dbapi_connection = engine.raw_connection()
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\engine\base.py", line 3269, in raw_connection
return self.pool.connect()
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\pool\base.py", line 455, in connect
return _ConnectionFairy._checkout(self)
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\pool\base.py", line 1270, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\pool\base.py", line 719, in checkout
rec = pool._do_get()
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\pool\impl.py", line 168, in _do_get
with util.safe_reraise():
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\util\langhelpers.py", line 147, in __exit__
raise exc_value.with_traceback(exc_tb)
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\pool\impl.py", line 166, in _do_get
return self._create_connection()
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\pool\base.py", line 396, in _create_connection
return _ConnectionRecord(self)
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\pool\base.py", line 681, in __init__
self.__connect()
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\pool\base.py", line 905, in __connect
with util.safe_reraise():
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\util\langhelpers.py", line 147, in __exit__
raise exc_value.with_traceback(exc_tb)
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\pool\base.py", line 901, in __connect
self.dbapi_connection = connection = pool._invoke_creator(self)
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\pool\base.py", line 368, in <lambda>
return lambda rec: creator_fn()
File "c:\Users\my_name\my_path\gl_cloudsql_mysql_fx.py", line 27, in getconn
conn: pymysql.connections.Connection = self.__connector.connect(
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\google\cloud\sql\connector\connector.py", line 154, in connect
return connect_task.result()
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\concurrent\futures\_base.py", line 458, in result
return self.__get_result()
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\concurrent\futures\_base.py", line 403, in __get_result
raise self._exception
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\google\cloud\sql\connector\connector.py", line 261, in connect_async
return await asyncio.wait_for(get_connection(), timeout)
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\asyncio\tasks.py", line 445, in wait_for
return fut.result()
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\google\cloud\sql\connector\connector.py", line 257, in get_connection
return await self._loop.run_in_executor(None, connect_partial)
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\concurrent\futures\thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\google\cloud\sql\connector\pymysql.py", line 54, in connect
socket.create_connection((ip_address, SERVER_PROXY_PORT)),
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\socket.py", line 845, in create_connection
raise err
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\socket.py", line 833, in create_connection
sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it
I run this python code on locally on my machine (in a Conda environment).
Here's my code:
from google.cloud.sql.connector import Connector
import sqlalchemy
import pymysql
connector = Connector()
# function to return the database connection
def getconn() -> pymysql.connections.Connection:
conn: pymysql.connections.Connection = connector.connect(
"my_project:my_location:my_instance",
"pymysql",
user="user",
password="password",
db="my_database",
)
return conn
# create connection pool
pool = sqlalchemy.create_engine(
"mysql+pymysql://",
creator=getconn,
)
# insert statement
check_table_statement = sqlalchemy.text(
"SELECT count(*) from information_schema.tables",
)
with pool.connect() as db_conn:
result = db_conn.execute(check_table_statement)
print(result)
connector.close()
I have tried to include my IP address in the Cloud SQL instance. However, I am not sure which one is the correct address. I have check my IP address multiple ways, but all of them are different:
Google
whatsmyip.org
cmd command nslookup myip.opendns.com resolver1.opendns.com
No luck with all of them.
Note:
I did not check the Allow only SSL connections box. So, I reckon, there is no need to include the CA certificate.
The CloudSQL instance is using Public IP.
Update:
I have added full error.

To solve this, check the following -
Make sure you enabled Public IP on the Cloud SQL instance
Then, create a user with hosts allowed as % (all IPs).
Next, ensure that you have a database created by the name of my_database by clicking on the Databases button on the left navigation inside Cloud SQL dashboard.
Ensure that you are including the right service credentials in your script with Cloud SQL Client role.
You should be able to connect. Do let me know if you face any issues, will update the answer accordingly :)

A couple things for you to try in order to resolve the ConnectionRefusedError.
Make sure the Cloud SQL Admin API is enabled within your Google Cloud Project.
Make sure the IAM principal you are using for authorization within your environment has the Cloud SQL Client Role granted to it.
Double check that your instance connection name is correct (first argument being passed to the .connect method) by verifying it on the Cloud SQL instance overview page.
If you are connecting to a Cloud SQL instance Private IP make sure you are connecting from a machine within the VPC network and that your code is updated for private IP connections:
from google.cloud.sql.connector import Connector, IPTypes
import sqlalchemy
import pymysql
connector = Connector()
# function to return the database connection
def getconn() -> pymysql.connections.Connection:
conn: pymysql.connections.Connection = connector.connect(
"my_project:my_location:my_instance",
"pymysql",
user="user",
password="password",
db="my_database",
ip_type=IPTypes.PRIVATE,
)
return conn
Let me know if any of these help resolve your connection, otherwise I can take a deeper look into it for you :)

Automatically handling pymongo.errors.AutoReconnect errors?

Recently, I have been very frequently encountering the error pymongo.errors.AutoReconnect. What is strange is that I never really encountered this error before and it just started happening for some reason.
Here is the full stack trace:
File "/home/ubuntu/.local/lib/python3.9/site-packages/pymongo/command_cursor.py", line 259, in next
doc = self._try_next(True)
File "/home/ubuntu/.local/lib/python3.9/site-packages/pymongo/command_cursor.py", line 270, in _try_next
self._refresh()
File "/home/ubuntu/.local/lib/python3.9/site-packages/pymongo/command_cursor.py", line 196, in _refresh
self.__send_message(
File "/home/ubuntu/.local/lib/python3.9/site-packages/pymongo/command_cursor.py", line 139, in __send_message
response = client._run_operation_with_response(
File "/home/ubuntu/.local/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1342, in _run_operation_with_response
return self._retryable_read(
File "/home/ubuntu/.local/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1464, in _retryable_read
return func(session, server, sock_info, slave_ok)
File "/home/ubuntu/.local/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1334, in _cmd
return server.run_operation_with_response(
File "/home/ubuntu/.local/lib/python3.9/site-packages/pymongo/server.py", line 117, in run_operation_with_response
reply = sock_info.receive_message(request_id)
File "/home/ubuntu/.local/lib/python3.9/site-packages/pymongo/pool.py", line 646, in receive_message
self._raise_connection_failure(error)
File "/home/ubuntu/.local/lib/python3.9/site-packages/pymongo/pool.py", line 643, in receive_message
return receive_message(self.sock, request_id,
File "/home/ubuntu/.local/lib/python3.9/site-packages/pymongo/network.py", line 196, in receive_message
_receive_data_on_socket(sock, 16))
File "/home/ubuntu/.local/lib/python3.9/site-packages/pymongo/network.py", line 261, in _receive_data_on_socket
raise AutoReconnect("connection closed")
pymongo.errors.AutoReconnect: connection closed
This error is raised by various queries in various situations. I'm really not certain why this issue started when it wasn't a problem before.
I have tried correcting the issue at one place where the error was most common by utilizing an exponential backoff.
retry_count = 0
while retry_count < 10:
try:
collection.remove({"index" : {"$lt" : (theTime- datetime.timedelta(minutes=60)).timestamp()}})
break
except:
wait_t = 0.5 * pow(2, retry_count)
self.myLogger.error(f"An error occurred while trying to delete entries. Retry : {retry_count}, Wait Time : {wait_t}")
time.sleep( wait_t )
finally:
retry_count = retry_count + 1
So far, this appears to have worked. My question is as follows:
I have a lot of various MongoDB queries. It would be very tedious to track down each of these queries and wrap it into an exponential backoff block that I have above. Is there a way to apply this in all situations when MongoDB encounters this error, so that I don't have to manually add this code everywhere?
Adding this block of code to each MongoDB query would be excessively tedious.

Python socket ConnectionResetError: [Errno 54] Connection reset by peer vs socket.error: [Errno 104] Connection reset by peer

I'm having trouble debugging my code because I cannot understand the socket error being raised.
Here is the traceback.
Traceback (most recent call last):
File "clickpression.py", line 517, in <module> presser.main()
File "clickpression.py", line 391, in main
File "clickpression.py", line 121, in clickpress self.refresh_proxies(country=country)
File "clickpression.py", line 458, in refresh_proxies self.proxies = self.get_proxies(country=country)
File "helpers.py", line 72, in wrapper return func(*args, **kwargs)
File "clickpression.py", line 264, in get_proxies self.settings.SUPER_PROXY).read().decode('utf-8')
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 161, in urlopen return opener.open(url, data, timeout)
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 463, in open response = self._open(req, data)
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 481, in _open '_open', req)
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 441, in _call_chain result = func(*args)
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1210, in http_open return self.do_open(http.client.HTTPConnection, req)
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1185, in do_open r = h.getresponse()
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 1171, in getresponse response.begin()
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 351, in begin version, status, reason = self._read_status()
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 313, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socket.py", line 374, in readinto return self._sock.recv_into(b)
ConnectionResetError: [Errno 54] Connection reset by peer
According to the errno library Errno 54 is errno.EXFULL which in the python 3 documentation is explained as exchange full.
To my understanding the Connection reset by peer is Errno 104 i.e errno.ECONNRESET.
So what does errno.EXFULL mean? and why does socket raise the error with a connection reset by peer description instead of exchange full. And or how are the two errors errno.EXFULL and errno.ECONNRESET related?
PS: I read that the errno 54 might be related to http proxy (I'm using a proxy in my code). If so, how?

According to the errno library Errno 54 is errno.EXFULL
Did you determine that by examining errno.errorcode[54]? Anyway - this errno library might be at fault. You could verify the meaning of an error code on your system by looking into errno.h, e. g. with the help of gcc:
gcc -xc -imacros errno.h -Wp,-P -E <(echo ECONNRESET)
Also, the Python documentation says:
To translate a numeric error code to an error message, use
os.strerror().
It may well be that error number 54 is ECONNRESET on your system, and that os.strerror(54) will attest that.
Now that you have verified that os.strerror(54) returns 'Exchange full', I am puzzled why the error number 54 and the error string Connection reset by peer do not match. If that happens on a system with strace or something similar, I would further check which error is returned by the operating system through use of strace -e network on the affected process.
Regarding your question about EXFULL: Its meaning seems somewhat system dependent; e. g. on Linux, EXFULL is returned from only a handful places in the kernel, the only network-related place being in br_if.c concerning network bridges, when no available bridge port number is found (other places are in USB and SCSI drivers).

I tried to use python to crew coin market on OKEX.com using WebSocket,cause the url is an outer address,i used a vpn service provided by us,but it still can work. here is the code an traceback.
from ws4py.client.threadedclient import WebSocketClient
class DummyClient(WebSocketClient):
def opened(self):
# self.send("{'event': 'addChannel', 'channel': 'ok_sub_futureusd_btc_ticker_this_week'}") #发送请求数据格式
# self.send("www.baidu.com")
self.send("{'event':'addChannel','channel':'ok_sub_spot_bch_btc_ticker'}")
def closed(self, code, reason=None):
print("Closed down", code, reason)
#服务器返回消息
def received_message(self, m):
print("recv:", m)
if __name__ == '__main__':
try:
# 服务器连接地址wss://real.okex.com:10440/websocket/okexapi
# ws = DummyClient('wss://real.okcoin.cn:10440/websocket/okcoinapi', protocols=['chat'])
ws = DummyClient('wss://real.okex.com:10440/websocket/okexapi', protocols=['chat'])
ws.connect()
#ws.send("my test...")
ws.run_forever()
except KeyboardInterrupt:
ws.close()

You can try this code to your project:
import ssl
ssl._create_default_https_context = ssl._create_unverified_context
if it not work,make sure the server open TLSv1 support.

Redis example giving HTTP 400: Bad request error

I am trying to cache MySQL queries in my Cherrypy server.
I could not figure out how to solve the error when I was installing pylibmc, so I decided to use Redis-py.
Here I am trying a very simple example.
import redis
cache = redis.StrictRedis(host='localhost', port=8080, db=0)
...
...
cache.set('0', '1') # I also tested with other string keys, but failed with same error
and it's throwing the following error!
[05/May/2014:13:11:13] HTTP Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/cherrypy/_cprequest.py", line 656, in respond
response.body = self.handler()
File "/Library/Python/2.7/site-packages/cherrypy/lib/encoding.py", line 188, in __call__
self.body = self.oldhandler(*args, **kwargs)
File "/Library/Python/2.7/site-packages/cherrypy/_cpdispatch.py", line 34, in __call__
return self.callable(*self.args, **self.kwargs)
File "server.py", line 92, in submit_data
cache.set(str(idx), '1')#res)
File "/Library/Python/2.7/site-packages/redis/client.py", line 897, in set
return self.execute_command('SET', *pieces)
File "/Library/Python/2.7/site-packages/redis/client.py", line 461, in execute_command
return self.parse_response(connection, command_name, **options)
File "/Library/Python/2.7/site-packages/redis/client.py", line 471, in parse_response
response = connection.read_response()
File "/Library/Python/2.7/site-packages/redis/connection.py", line 339, in read_response
response = self._parser.read_response()
File "/Library/Python/2.7/site-packages/redis/connection.py", line 118, in read_response
(str(byte), str(response)))
InvalidResponse: Protocol Error: H, TTP/1.1 400 Bad Request
I could not figure out what was wrong, and my website runs without a problem on localhost at port 8080 when I am not using Redis.

Your webserver is running on port 8080, not your Redis server. Your Redis server is most likely running on port 6379, unless you changed your config for some reason. Right now you are trying to run Redis queries against your webserver, and that is not going to work. Make sure you are connecting to the correct Redis server address and port and then try again.

sending email via django send_mail

I have been using django send_mail for a long while. however on a new project which use django 1.4 and I changed my dev machine I have a very strange problem.
when I call send_mail on the standard backend with all the settings (EMAIL_HOST='localhost' EMAIL_PORT=25,etc) I am getting the error :
File "c:\python27\lib\site-packages\django\core\mail\backends\smtp.py", line 4
8, in open local_hostname=DNS_NAME.get_fqdn())
File "c:\python27\lib\smtplib.py", line 249, in __init__
(code, msg) = self.connect(host, port)
File "c:\python27\lib\smtplib.py", line 309, in connect
self.sock = self._get_socket(host, port, self.timeout)
File "c:\python27\lib\smtplib.py", line 284, in _get_socket
return socket.create_connection((port, host), timeout)
File "c:\python27\lib\socket.py", line 571, in create_connection
raise err
socket.error: [Errno 10013] An attempt was made to access a socket in a way forb
idden by its access permissions
I have no clue what I am doing wrong, I have debugged it and it seems that the connect function recieve the right port and host however its getting forbidden. I suspected its maybe my antivirus but disabling it didnt help.
I am running on administrator privilages (UAC disabled) if anyone consider this to be the problem.

Strange. I changed my email server (python SMTPd) to port 1025 instead of 25 and everything back to normal....