Develop Reference

Cannot get python module ezsheets to work

I followed the steps from the official documentation for ezsheets. Both apis were activated (sheets and drive) and I turned on a python shell and imported the ezsheets module. But, it did not open the new browser window like the documentation said it's supposed to. I then tried and successfully got google sheets working with googles' quickstart script. I have the credentials-sheets.json and token.pickle in the same folder as my python scripts. Sheets api seems to work just fine on it's own but when I try it with ezsheets module I get a following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/method/.local/share/virtualenvs/python_projects-sgt8dc05/lib/python3.6/site-packages/ezsheets/__init__.py", line 1559, in listSpreadsheets
if not IS_INITIALIZED: init()
File "/home/method/.local/share/virtualenvs/python_projects-sgt8dc05/lib/python3.6/site-packages/ezsheets/__init__.py", line 1520, in init
creds = flow.run_local_server()
File "/home/method/.local/share/virtualenvs/python_projects-sgt8dc05/lib/python3.6/site-packages/google_auth_oauthlib/flow.py", line 443, in run_local_server
host, port, wsgi_app, handler_class=_WSGIRequestHandler)
File "/usr/lib/python3.6/wsgiref/simple_server.py", line 153, in make_server
server = server_class((host, port), handler_class)
File "/usr/lib/python3.6/socketserver.py", line 456, in __init__
self.server_bind()
File "/usr/lib/python3.6/wsgiref/simple_server.py", line 50, in server_bind
HTTPServer.server_bind(self)
File "/usr/lib/python3.6/http/server.py", line 136, in server_bind
socketserver.TCPServer.server_bind(self)
File "/usr/lib/python3.6/socketserver.py", line 470, in server_bind
self.socket.bind(self.server_address)
OSError: [Errno 98] Address already in use

ezsheets is trying to open webserver on port 8000 to obtain access token. But it is already occupied by another app.
Chech what app listens on port 8000 and close it.
On Linux you can check it with
netstat -tulpn | grep 8000
On Windows:
netstat -ab

Unable to connect to Redis instance running on Azure Linux VM from Python

I created an Ubuntu VM on Azure. In its inbound networking filters, I added ports 22 (for SSHing) and 6379 (for Redis). Following this, I SSHed into an instance from my bash shell, downloaded, built and installed Redis from source. The resultant redis.conf file is located in /tmp/redis-stable, so I edited that to comment out the bind 127.0.0.1 rule.
Then I started redis-server redis.conf from the /tmp/redis-stable directory, and it started normally, following which I SSHed into another instance of the VM, started redis-cli and set some keys. Retrieved them, working correctly.
Now in Python I am running this command:
r = redis.Redis(host='same_which_I_use_for_SSHing', port=6379, password='pwd')
It connects immediately (looks weird). But then when I try a simple command like r.get("foo"), I get this error:
>>> r.get("foo")
Traceback (most recent call last):
File "/lib/python3.5/site-packages/redis/connection.py", line 484, in connect
sock = self._connect()
File "/lib/python3.5/site-packages/redis/connection.py", line 511, in _connect
socket.SOCK_STREAM):
File "/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/socket.py", line 732, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 8] nodename nor servname provided, or not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/lib/python3.5/site-packages/redis/client.py", line 667, in execute_command
connection.send_command(*args)
File "/lib/python3.5/site-packages/redis/connection.py", line 610, in send_command
self.send_packed_command(self.pack_command(*args))
File "/lib/python3.5/site-packages/redis/connection.py", line 585, in send_packed_command
self.connect()
File "/lib/python3.5/site-packages/redis/connection.py", line 489, in connect
raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 8 connecting to username#ip_address. nodename nor servname provided, or not known.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/lib/python3.5/site-packages/redis/connection.py", line 484, in connect
sock = self._connect()
File "/lib/python3.5/site-packages/redis/connection.py", line 511, in _connect
socket.SOCK_STREAM):
File "/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/socket.py", line 732, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 8] nodename nor servname provided, or not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/lib/python3.5/site-packages/redis/client.py", line 976, in get
return self.execute_command('GET', name)
File "/lib/python3.5/site-packages/redis/client.py", line 673, in execute_command
connection.send_command(*args)
File "/lib/python3.5/site-packages/redis/connection.py", line 610, in send_command
self.send_packed_command(self.pack_command(*args))
File "/lib/python3.5/site-packages/redis/connection.py", line 585, in send_packed_command
self.connect()
File "/lib/python3.5/site-packages/redis/connection.py", line 489, in connect
raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 8 connecting to username#ip_address. nodename nor servname provided, or not known.
Any idea how to fix this? FYI, the username#ip_address in the error message is the same I use for SSH from bash and connecting to Redis from Python respectively:
ssh username#ip_address
r = redis.Redis(host='username#ip_address', port=6379, password='pwd')
I also tried adding bind 0.0.0.0 in the redis.conf file after commenting out the bind 127.0.0.1 line. Same result.
Update: I tried setting protected mode to no in the config file, followed by running sudo ufw allow 6379 in the VM. Still same result. But now I get a weird error when I run redis-server redis.conf. I don't get the typical redis cube which shows up as a figure Instead I get this:
After this, if I enter redis-cli and issue a simple command like set foo boo, I get this error message:
127.0.0.1:6379> set foo 1
(error) MISCONF Redis is configured to save RDB snapshots, but it is currently not able to persist on disk. Commands that may modify the data set are disabled, because this instance is configured to report errors during writes if RDB snapshotting fails (stop-writes-on-bgsave-error option). Please check the Redis logs for details about the RDB error.
Even shutdown fails after this. I have to run grep to find the process if of redis-server and kill it manually, following which redis-server command needs to be run to start it normally. But of course, I still cannot connect from remote Mac.

I tried to create an Ubuntu VM on Azure, build & configure redis server from source code, and add a port rule for my VM NSG on Azure portal, then I connected the redis server via the same code with python redis package successfully.
Here is my steps as below.
Create an Ubuntu 18.04 VM on Azure portal.
Connect the Ubuntu VM via SSH to install the build packages (include build-essential, gcc and make), download & decompress the tar.gz file of redis source code from redis.io, and build & test the redis server with make & make test.
Configure the redis.conf via vim, to change the bind configuration from 127.0.0.1 to 0.0.0.0.
Add a new port rule of 6379 port with TCP into the NSG for the Network Interface shown in the tab Settings -> Networking of my VM on Azure portal, as the figures below.
Notes: There are two NSG in my Networking tab, the second one is related to my Network Interface which can be accessed via internet.
and
5. Install redis via pip install redis on my local machine.
6. Open a termial to type python to try to connect the redis server hosted on my Azure Ubuntu VM and get the value of the foo key successfully.
>>> import redis
>>> r = redis.Redis(host='xxx.xx.xx.xxx')
>>> r.get('foo')
b'bar'
There are two key issues I found within my testing.
For redis.conf, the binding host must be 0.0.0.0. If not, redis server will be running in protected mode to refuse the query.
For the NSG port rules, make sure the new port rule added in the NSG attached to the current network interface, not the default subnet.
The python package redis is lazy evaluation, only connect redis server when first command request happened.
Hope it helps.

Vagrant python [Errno -2] Name or service not known

I am currently having a problem with vagrant. I used it before for a number of tasks, but now when I do vagrant up and then vagrant ssh, everything seems to work ok. Then I navigate into the vagrant folder to run a python file and when I run it I get the following error:
vagrant#vagrant:/vagrant$ python project.py
Traceback (most recent call last):
File "project.py", line 75, in
app.run(host='0.0.0.0.', port=8080)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 841, in run
run_simple(host, port, self, **options)
File "/usr/local/lib/python2.7/dist-packages/werkzeug/serving.py", line 720,
in run_simple
s.bind((hostname, port))
File "/usr/lib/python2.7/socket.py", line 228, in meth
return getattr(self._sock,name)(*args)
socket.gaierror: [Errno -2] Name or service not known
I tried multiple times to restart vagrant, I also tried vagrant destroy and restarted the process. I manually deleted and powered it off a couple of times, but I always experience the same error. I would appreciate any help. Thank you.

The problem isn't vagrant... This line in the stacktrace shows there could be a typo in your python script
app.run(host='0.0.0.0.', port=8080)
Should be
app.run(host='0.0.0.0', port=8080)

Connection to AWS timed out using Python boto

I ran Python program in Cygwin to connect to AWS, but failed consistently as being timed out. But my connection to AWS using aws cli in Cygwin always works. Also if I run the same python code in Windows, it also works. I have checked all the connection credentials which are the same for all.
Here is the error msg:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/site-packages/boto-2.38.0-py2.7.egg/boto/ec2/connection.py", line 585, in get_all_instances
max_results=max_results)
File "/usr/lib/python2.7/site-packages/boto-2.38.0-py2.7.egg/boto/ec2/connection.py", line 681, in get_all_reservations
[('item', Reservation)], verb='POST')
File "/usr/lib/python2.7/site-packages/boto-2.38.0-py2.7.egg/boto/connection.py", line 1170, in get_list
response = self.make_request(action, params, path, verb)
File "/usr/lib/python2.7/site-packages/boto-2.38.0-py2.7.egg/boto/connection.py", line 1116, in make_request
return self._mexe(http_request)
File "/usr/lib/python2.7/site-packages/boto-2.38.0-py2.7.egg/boto/connection.py", line 1030, in _mexe
raise ex
socket.error: [Errno 116] Connection timed out

I have found out that the culprit lies in the proxy credentials.
I put HTTP_PROXY and HTTPS_PROXY as Windows Environment Variables. However, when run in Cygwin, boto uses 'http_proxy' to match without considering the case of the word
(see /boto/connection.py(669)handle_proxy()
line 669: if 'http_proxy' in os.environ and not self.proxy:).
When I changed the capital case HTTP_PROXY to lower case http_proxy, then it worked. Not sure why it isn't a problem if I run with Python in Windows.

redis.exceptions.ConnectionError: Error -2 connecting to localhost:6379. Name or service not known

I have this error when I run my code in server, my env is debian, and Python2.7.3
Traceback (most recent call last):
File "fetcher.py", line 4, in <module>
import mirad.fetcher_tasks as tasks
File "/home/mirad/backend/mirad/fetcher_tasks.py", line 75, in <module>
redis_keys = r.keys('*')
File "/home/mirad/backend/venv/local/lib/python2.7/site-packages/redis/client.py", line 863, in keys
return self.execute_command('KEYS', pattern)
File "/home/mirad/backend/venv/local/lib/python2.7/site-packages/redis/client.py", line 534, in execute_command
connection.send_command(*args)
File "/home/mirad/backend/venv/local/lib/python2.7/site-packages/redis/connection.py", line 532, in send_command
self.send_packed_command(self.pack_command(*args))
File "/home/mirad/backend/venv/local/lib/python2.7/site-packages/redis/connection.py", line 508, in send_packed_command
self.connect()
File "/home/mirad/backend/venv/local/lib/python2.7/site-packages/redis/connection.py", line 412, in connect
raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error -2 connecting to localhost:6379. Name or service not known.
when I run redis-cli it works correctly without any error:
$ redis-cli
127.0.0.1:6379>

It seems that you are trying connect redis with server that is unidentified by your current Debian environment.
From Traceback, I see you are trying to connect using host name as localhost ,
r_server=redis.Redis(host="localhost",port=6379)
But , your system is unable to understand "localhost" , make entry in hosts file i.e saying 127.0.0.1 is localhost. add below code in /etc/hosts
127.0.0.1 localhost
otherwise connect redis using below command ;
r_server=redis.Redis(host="127.0.0.1",port=6379)

I faced a similar problem and later on realized that I have not opened redis in the terminal by command: $ redis-server.
Maybe it is the case.