Develop Reference

How to properly install csdtoolbox-remote?

I am having trouble installing the cdstoolbox-remote on my Linux machine (tried both Ubuntu and Debian). I installed it using pip:
pip install cdstoolbox-remote
Which successfully installs the module, but I can't import the module in Python. Importing the module causes the following error:
>>> import cdstoolbox
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/an/miniconda2/envs/tools/lib/python3.7/site-packages/cdstoolbox/__init__.py",
line 8, in <module>
with open(__main__.__file__) as f:
AttributeError: module '__main__' has no attribute '__file__'
What can be done to get it working?

Do you really need cdstoolbox-remote? The standard API for downloading data from CDS would be cdsapi, which can be installed using
pip install cdsapi
or using conda (https://anaconda.org/conda-forge/cdsapi)
I also just tried installing cdstoolbox-remote and got the same problem. But as it is flagged as experimental I even consider it to be possible that the current version is not a stable version.
And cdsapi seems to work.

Actually, the CDSAPI can also be used to execute workflows as follows:
import cdsapi
c = cdsapi.Client()
with open("workflow.py") as f:
code = f.read()
r = c.workflow(code)
print(c.download(r))
For more, read this thread on the Copernicus services documentation.

python v 2.7.13 with twilio v 6.5.2 cannot find module named urllib3 when importing Client

I'm using python version 2.7.13 and twilio version 6.5.2 and I'm trying the simple example from twilio website which begins with
from twilio.rest import Client
then goes on to send a text message to a phone number. Executing that first line within idle produces this error message:
Traceback (most recent call last):
File "C:\Users\...\send_text.py", line 1, in <module>
from twilio.rest import Client
File "build\bdist.win32\egg\twilio\rest\__init__.py", line 14, in <module>
File "build\bdist.win32\egg\twilio\http\http_client.py", line 1, in <module>
File "C:\Python27\lib\site-packages\requests-2.18.4-py2.7.egg\requests\__init__.py", line 43, in <module>
import urllib3
ImportError: No module named urllib3
I have found similar questions in this forum but the details are slightly different and the solutions do not apply to my situation.

First, check this link: https://matthewhorne.me/how-to-install-python-and-pip-on-windows-10/
Second, on your console, execute: pip install urllib3
This steps maybe can help you. Good luck.

Make sure you have installed the module using pip install urllib3 before you import it.

Thank you to Jose Moreno and Colin Ricardo.
Re-installing twilio using pip rather than easy_install was step one. Then, as suggested by Jose and Colin, the next step was to pip install urrlib3.
Turned out to be two more things that were missing, but pip installing them worked.
Then I was able to get a trial phone number from twilio and their sample send_text.py program actually worked for me!

"ImportError: file_cache is unavailable" when using Python client for Google service account file_cache

I'm using a service account for G Suite with full domain delegation. I have a script with readonly access to Google Calendar. The script works just fine, but throws an error (on a background thread?) when I "build" the service. Here's the code:
from oauth2client.service_account import ServiceAccountCredentials
from httplib2 import Http
import urllib
import requests
from apiclient.discovery import build
cal_id = "my_calendar_id#group.calendar.google.com"
scopes = ['https://www.googleapis.com/auth/calendar.readonly']
credentials = ServiceAccountCredentials.from_json_keyfile_name('my_cal_key.json', scopes=scopes)
delegated_credentials = credentials.create_delegated('me#mydomain.com')
http_auth = delegated_credentials.authorize(Http())
# This is the line that throws the error
cal_service = build('calendar','v3',http=http_auth)
#Then everything continues to work normally
request = cal_service.events().list(calendarId=cal_id)
response = request.execute()
# etc...
The error thrown is:
WARNING:googleapiclient.discovery_cache:file_cache is unavailable when using oauth2client >= 4.0.0
Traceback (most recent call last):
File "/Users/myuseraccount/anaconda3/lib/python3.5/site-packages/googleapiclient/discovery_cache/__init__.py", line 36, in autodetect
from google.appengine.api import memcache
ImportError: No module named 'google'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/myuseraccount/anaconda3/lib/python3.5/site-packages/googleapiclient/discovery_cache/file_cache.py", line 33, in <module>
from oauth2client.contrib.locked_file import LockedFile
ImportError: No module named 'oauth2client.contrib.locked_file'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/myuseraccount/anaconda3/lib/python3.5/site-packages/googleapiclient/discovery_cache/file_cache.py", line 37, in <module>
from oauth2client.locked_file import LockedFile
ImportError: No module named 'oauth2client.locked_file'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/myuseraccount/anaconda3/lib/python3.5/site-packages/googleapiclient/discovery_cache/__init__.py", line 41, in autodetect
from . import file_cache
File "/Users/myuseraccount/anaconda3/lib/python3.5/site-packages/googleapiclient/discovery_cache/file_cache.py", line 41, in <module>
'file_cache is unavailable when using oauth2client >= 4.0.0')
ImportError: file_cache is unavailable when using oauth2client >= 4.0.0
What's going on here and is this something that I can fix? I've tried reinstalling and/or upgrading the google package.

I am a bit late to the party here but I had a similar problem today and found the answer here
Solution to only the error : file_cache is unavailable when using oauth2client >= 4.0.0
Solution:
change your discovery.build() to have the field cache_discovery=False
i.e
discovery.build(api, version, http=http, cache_discovery=False)

The code head of module "google-api-python-client" said...
install_requires = [
'httplib2>=0.9.2,<1dev',
'oauth2client>=1.5.0,<5.0.0dev', <<=============
'six>=1.6.1,<2dev',
'uritemplate>=3.0.0,<4dev',
]
So, I have uninstalled oauth2client version 4.0.0
Then, I have downloaded oauth2client 1.5.2 in a tar.gz file from offial python site https://pypi.python.org/pypi/oauth2client/1.5.2
I have installed this downloaded file so I have 1.5.2 version of oauth2client
Package Version
------------------------ ---------
certifi 2016.9.26
discovery 0.0.4
distribute 0.7.3
future 0.16.0
google-api-python-client 1.5.5
httplib2 0.9.2
oauth2client 1.5.2
pefile 2016.3.28
pip 9.0.1
pyasn1 0.1.9
pyasn1-modules 0.0.8
PyInstaller 3.2
pypiwin32 219
requests 2.11.1
rsa 3.4.2
setuptools 28.8.0
six 1.10.0
uritemplate 3.0.0
After that, ALL is working OK again and there is no warning message.

Tried all the solutions that were listed, but none of them worked. Until I tried the (simple) suggestion from #dtk that was listed somewhere in the comments :
Install an older version of oauth2client by running:
pip install oauth2client==3.0.0
Now everything works fine for me. Thank you #dtk !

I did not want to downgrade my oauth2client. You can use oauth2client 4.1.2 when you set cache_discovery=False for apiclient.discovery.build. This simple solution silenced the error:
service = discovery.build(api_name,
api_version,
credentials=credentials,
cache_discovery=False)
Source

To use the Google API for Python client, you need to install it first as Google API is not built-in within the Python modules. The instruction is found in Install the Library.
Installation
You can either use a package manager or download and install the Python client library manually:
Managed installation
Use pip or setuptools to manage your installation (you might need to run sudo first):
pip (preferred):
$ pip install --upgrade google-api-python-client
Setuptools: Use the easy_install tool included in the setuptools package:
$ easy_install --upgrade google-api-python-client

Fwiw this is a reposting of my answer here
Since they are technically warnings (not errors), if you don't have direct access to the call, you can do this wherever you instantiate the service to suppress them:
import logging
logging.getLogger('googleapiclient.discovery_cache').setLevel(logging.ERROR)
With thanks to theacodes who gave that answer here (but with a missing 'n' in 'client' that seems to have led to a lot of downvotes there).
See also the discussion here

to add to the selected answer, if the build function is called by a dependency, and there is no easy way to update it's code, this code can be used to force cache_discovery to be False:
import googleapiclient.discovery
_build = googleapiclient.discovery.build
def no_cache_build(*args, **kw):
kw["cache_discovery"] = False
return _build(*args, **kw)
googleapiclient.discovery.build = no_cache_build

Here is how I use google-api-python-client to build a service on Cloud Functions.
# for Cloud Functions use
def get_service():
import googleapiclient.discovery
return googleapiclient.discovery.build('compute', 'v1', cache_discovery=False)
And finally it worked.

cannot import Python-Twitter

I recently started out with python and i'm right now looking to do an app that shows my Twitter feed. So I downloaded a module(not sure if its called that) called Python-Twitter and installed it. But now everytime I try to import it I just get this error:
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
import twitter
File "build\bdist.win32\egg\twitter.py", line 38, in <module>
File "C:\Python27\lib\site-packages\requests_oauthlib-0.4.0-py2.7.egg\requests_oauthlib\__init__.py", line 1, in <module>
from .oauth1_auth import OAuth1
File "C:\Python27\lib\site-packages\requests_oauthlib-0.4.0-py2.7.egg\requests_oauthlib\oauth1_auth.py", line 4, in <module>
from requests.utils import to_native_string
ImportError: cannot import name to_native_string
Does anyone know what I might have done wrong with the install or something? Thx

In my case installing requests-oauthlib fixed it
sudo pip install requests-oauthlib

This is because of requests version.
requests.utils.to_native_string is available since requests 2.0.0
So just update requests:
pip install -U requests
See more details here on another thread

This sounds like an issue with your install. The method to_native_string should be defined in the requests\utils.py in your Python install.
I'm on Ubuntu, and installed the Twitter module, and can import it without any errors. When I look in my install, in /usr/lib/python2.7/dist-packages/requests/utils.py, there is a to_native_string method defined there.
The implication from the error you're getting is that in your installation, there is either no utils.py, or, if there is, it does not contain that method.
I would recommend checking your install to see if that is the case. If it is indeed missing, I would recommend axing your install and reinstalling. You might want to try a virtualenv environment instead, so it can act as a sandbox (in that case you can leave your current install as is, as long as it is intact enough to run virtualenv and pip).
If utils.py is actually there and does contain a method with that name, I would recommend running a debugger such as pudb or pdb (pudb isn't built in, but it's more full featured), and step through the import to see if that sheds any additional light.

installing and using python-oauth2

I installed oauth2:
$ pip install oauth2
But running the python-oauth2/example/client.py returns:
Traceback (most recent call last):
File "client.py", line 31, in <module>
import oauth.oauth as oauth
ImportError: No module named oauth.oauth
I tested pip freeze:
oauth2==1.5.211
Thanks in advance

As it turns out, the example directory in the oauth2 package should be ignored. It is broken code as far as the package is concerned.
See these issues for examples of other people discovering this:
https://github.com/simplegeo/python-oauth2/issues/33
https://github.com/simplegeo/python-oauth2/issues/12
https://github.com/simplegeo/python-oauth2/pull/64
The last one is a pull request that includes new examples to use instead.