Develop Reference

Pythonanywhere: Twitter API authenticates from bash, but fails on scheduled task

I'm using the python-twitter (not tweepy) module in my application.
I have a script set up (maketweet.py) that, when I run it from bash / console, works successfully (i.e. it makes a tweet).
The problem I'm having is that when I run the same script as a scheduled task, I get an error:
Traceback (most recent call last):
File "/home/dir1/dir2/proj/maketweet.py", line 21, in <module>
api = create_api()
File "/home/dir1/dir2/proj/config.py", line 31, in create_api
if api.VerifyCredentials():
File "/home/dir1/.local/lib/python3.8/site-packages/twitter/api.py", line 4699, in VerifyCredentials
resp = self._RequestUrl(url, 'GET', data)
File "/home/dir1/.local/lib/python3.8/site-packages/twitter/api.py", line 4959, in _RequestUrl
raise TwitterError("The twitter.Api instance must be authenticated.")
twitter.error.TwitterError: The twitter.Api instance must be authenticated.
The other problem I'm having is that I can't conceive why it would make a difference that I'm using a scheduled task, rather than running the file directly from bash. Here are the contents of config.py:
#!/usr/bin/python3.8
import twitter
import os
import logging
from dotenv import load_dotenv
logger = logging.getLogger()
# project folder is one level up from file location
project_folder = pathlib.Path(__file__).parent.absolute()
load_dotenv(os.path.join(project_folder, '.env'))
# Authenticate to Twitter and create API object
TWITTER_CONSUMER_API_KEY = os.getenv("TWITTER_CONSUMER_API_KEY")
TWITTER_CONSUMER_API_SECRET = os.getenv("TWITTER_CONSUMER_API_SECRET")
TWITTER_ACCESS_TOKEN = os.getenv("TWITTER_ACCESS_TOKEN")
TWITTER_ACCESS_SECRET = os.getenv("TWITTER_ACCESS_SECRET")
def create_api():
# Create API object
api = twitter.Api(
access_token_key = TWITTER_ACCESS_TOKEN,
access_token_secret = TWITTER_ACCESS_SECRET,
consumer_key = TWITTER_CONSUMER_API_KEY,
consumer_secret = TWITTER_CONSUMER_API_SECRET,
sleep_on_rate_limit=True)
# test API object
if api.VerifyCredentials():
pass
else:
logger.error("Error creating API", exc_info=True)
raise Exception("Twitter user authentication error")
logger.info("API created")
return api
Obviously there's an error in creating the API. I imagine this has something to do with the environment variables and how they are accessed through scheduled tasks vs. bash. Just really not sure how to figure this one out...

Status parameter not working when using python blogger api

I'm trying to use google-api-python-client 1.12.5 with Service account auth under Python 3.8. It seems to me that the when specifying the status parameter, Google responds with a 404 HTTP code. I can't figure out why. I also looked in the docs but I can't relate anything to this error.
I have pasted my code. The error is happening in the third call.
This is the code:
from google.oauth2 import service_account
from googleapiclient.discovery import build
SCOPES = ['https://www.googleapis.com/auth/blogger']
SERVICE_ACCOUNT_FILE = 'new_service_account.json'
BLOG_ID = '<your_blog_id>'
credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
service = build('blogger', 'v3', credentials=credentials)
p = service.posts()
# FIRST
promise = p.list(blogId=BLOG_ID)
result = promise.execute()
# SECOND
promise = p.list(blogId=BLOG_ID, orderBy='UPDATED')
result = promise.execute()
#THIRD
promise = p.list(blogId=BLOG_ID, orderBy='UPDATED', status='DRAFT')
result = promise.execute() # <===== ERROR HAPPENS HERE!!!!
service.close()
And this is the traceback:
Traceback (most recent call last):
File "/home/madtyn/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/202.7660.27/plugins/python/helpers/pydev/pydevd.py", line 1448, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "/home/madtyn/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/202.7660.27/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/home/madtyn/PycharmProjects/blogger/main.py", line 24, in <module>
result = promise.execute()
File "/home/madtyn/venvs/blogger/lib/python3.8/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
return wrapped(*args, **kwargs)
File "/home/madtyn/venvs/blogger/lib/python3.8/site-packages/googleapiclient/http.py", line 915, in execute
raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 404 when requesting https://blogger.googleapis.com/v3/blogs/<blog_id>/posts?orderBy=UPDATED&status=DRAFT&alt=json returned "Not Found">
python-BaseException

I can reproduce this issue... Adding status=DRAFT will return 404 but any other filter is working...
Tried with service account and your code: 404
Tried with API Key like this result = requests.get('https://blogger.googleapis.com/v3/blogs/<blog_id>/posts?status=DRAFT&orderBy=UPDATED&alt=json&key=<api_key>'): 404
Extracted "access_token" from service account (credentials.token after a call): result = requests.get('https://blogger.googleapis.com/v3/blogs/<blog_id>/posts?status=DRAFT&orderBy=UPDATED&alt=json&access_token=<extracted_service_account_token>'): 404
But very strangely if I use access_token given by "Try this API" here : https://developers.google.com/blogger/docs/3.0/reference/posts/list?apix_params={"blogId"%3A"blog_id"%2C"orderBy"%3A"UPDATED"%2C"status"%3A["DRAFT"]%2C"alt"%3A"json"} it's works !
Used that token with requests give me my blog post in draft status...
Just copy/paste raw Authorization header inside that script:
import requests
blog_id = '<blog_id>'
headers = {
'Authorization' : 'Bearer <replace_here>'
}
# Using only Authorization header
result = requests.get(
'https://blogger.googleapis.com/v3/blogs/%s/posts?status=DRAFT&orderBy=UPDATED&alt=json' % (blog_id),
headers=headers
)
print(result)
# This should print DRAFT if you have at least one draft post
print(result.json()['items'][0]['status'])
# Using "access_token" param constructed with Authorization header splited to have only token
result = requests.get('https://blogger.googleapis.com/v3/blogs/%s/posts?status=DRAFT&orderBy=UPDATED&alt=json&access_token=%s' % (blog_id, headers['Authorization'][len('Bearer '):]))
print(result)
# This should print DRAFT if you have at least one draft post
print(result.json()['items'][0]['status'])
Results I have currently:
The bug doesn't seem to come from the library but rather from the token rights...However I also used the console normally to generate accesses like you.
To conclude I think it's either a bug or it's voluntary from Google... I don't know how long the "Try this API" token is valid but it is currently the only way I found to get the draft articles... Maybe you can try to open a bug ticket but I don't know specifically where it is possible to do that.

Foursquare authentication - 'No handlers could be found for logger "foursquare"'

I'm starting with the Foursquare API, in Python, and I'm not sure why I can't authenticate.
Following the tutorial, so far I have this piece of code:
import foursquare
client = foursquare.Foursquare(client_id=myid, client_secret=mysecret,
redirect_uri='http://fondu.com/oauth/authorize')
auth_uri = client.oauth.auth_url()
access_token = client.oauth.get_token('XX_CODE_RETURNED_IN_REDIRECT_XX')
client.set_access_token(access_token)
client.venues.explore(params={'near': 'New York, NY', 'time' : date})
I've created an app here:
https://foursquare.com/developers/apps
and I'm using both:
Client id
Client secret
shown in the page.
However, when running this code, I get:
No handlers could be found for logger "foursquare"
Traceback (most recent call last):
File "noiseInference.py", line 270, in <module>
getFoursquareCheckIns(date)
File "noiseInference.py", line 156, in getFoursquareCheckIns
access_token = client.oauth.get_token('XX_CODE_RETURNED_IN_REDIRECT_XX')
File "/Library/Python/2.7/site-packages/foursquare/__init__.py", line 134, in get_token
response = _request_with_retry(url)
File "/Library/Python/2.7/site-packages/foursquare/__init__.py", line 707, in _request_with_retry
return _process_request_with_httplib2(url, headers, data)
File "/Library/Python/2.7/site-packages/foursquare/__init__.py", line 730, in _process_request_with_httplib2
return _check_response(data)
File "/Library/Python/2.7/site-packages/foursquare/__init__.py", line 763, in _check_response
raise FoursquareException(errmsg)
foursquare.FoursquareException: Response format invalid, missing meta property. data: {u'error': u'invalid_client'}
Not sure what's the problem.

The handler message is just complaining you didn't set up a logger under for the foursquare namespace.
Your real error is the message at the end of the stack trace:
foursquare.FoursquareException:
Response format invalid, missing meta property. data: {u'error': u'invalid_client'}
The message indicates that your client's credentials are incorrect. The credentials aren't fully checked until you attempt to use the client for a privileged action like client.set_access_token, so the most likely culprit here is to look at what you pass for client_secret when constructing the Foursquare client object.
client_id is probably not the problem because you'd have to go through the URL-OAuth flow in order to get the access_token you use.

Dropbox Python SDK SSL error

I am learning how to use the Dropbox Python SDK. However, I run into a problem when I try to generate my request token. Here is the code I am using (note that I have replaced my actual app key and secret with APP_KEY and APP_SECRET here, but I used my actual app key and secret when I try it out.)
from dropbox import client, rest, session
APP_KEY = 'APP_KEY'
APP_SECRET = 'APP_SECRET'
ACCESS_TYPE = 'app_folder'
print 'Creating session object'
sess = session.DropboxSession(APP_KEY, APP_SECRET, ACCESS_TYPE)
print 'Session created!\nCreating request token'
request_token = sess.obtain_request_token()
print 'Created request token!'
url = sess.build_authorize_url(request_token)
print url
raw_input()
access_token = sess.obtain_access_token(request_token)
client1 = client.DropboxClient(sess)
print client1.account_info()
I had the program print out messages as it was creating the different objects so that I could see where the error occurred. This is the output:
aaron#Aarons-Ubuntu-Computer:~/Twisted$ python example.py
Creating session object
Session created!
Creating request token
Traceback (most recent call last):
File "example.py", line 8, in <module>
request_token = sess.obtain_request_token()
File "/usr/local/lib/python2.7/dist-packages/dropbox-1.4-py2.7.egg/dropbox/session.py", line 160, in obtain_request_token
response = rest.RESTClient.POST(url, headers=headers, params=params, raw_response=True)
File "/usr/local/lib/python2.7/dist-packages/dropbox-1.4-py2.7.egg/dropbox/rest.py", line 140, in POST
return cls.request("POST", url, post_params=params, headers=headers, raw_response=raw_response)
File "/usr/local/lib/python2.7/dist-packages/dropbox-1.4-py2.7.egg/dropbox/rest.py", line 64, in request
conn = ProperHTTPSConnection(host, 443)
File "/usr/local/lib/python2.7/dist-packages/dropbox-1.4-py2.7.egg/dropbox/rest.py", line 214, in __init__
self.cert_reqs = ssl.CERT_REQUIRED
AttributeError: 'module' object has no attribute 'CERT_REQUIRED'
I had tried using this code before, and I didn't experience this problem. I have also removed the Dropbox SDK and reinstalled it, with no result. What is causing this problem, and how can I fix it?
UPDATE
After adding Kannan's code, the output looks like this:
Ready
Generating session
Session Generated!
Generating request token
['Certificate', 'CertificateOptions', 'CertificateRequest', 'Client', 'ClientContextFactory', 'Connector', 'ContextFactory', 'DN', 'DefaultOpenSSLContextFactory', 'DistinguishedName', 'KeyPair', 'Port', 'PrivateCertificate', 'SSL', 'Server', '__all__', '__builtins__', '__doc__', '__file__', '__name__', '__package__', 'implementedBy', 'implements', 'implementsOnly', 'interfaces', 'supported', 'tcp']
Then I get the error. I have tried re-created my SSL certificates with no luck.

I'm getting the same error. I tried what you suggested, and it turns out that ssl module has no attribute 'file', and only has ['doc', 'loader', 'name', 'package'] when I print dir(ssl). Also worth noting is that when I try print str(ssl) in google app engine, I see:
module 'ssl' (built-in)
but when I run python from the command line, I see:
module 'ssl' from '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.pyc'
and the ssl from the command line has a file attribute.
Not sure how ssl is getting hijacked... but I'm pretty sure it is.
EDIT: loader is google.appengine.tools.dev_appserver_import_hook.HardenedModulesHook object at 0x1142684d0
EDIT: found a related problem here: SSLError on Google App Engine (local dev-server)

This is strange. The Python 2.7 docs definitely mention ssl.CERT_REQUIRED.
Try modifying the SDK and, right before the line that fails, add in
print dir(ssl)
print ssl.__file__
That might help you figure out what's going wrong.

Pyramid on App Engine gets "InvalidResponseError: header values must be str, got 'unicode'

I am using Pyramid 1.3 with the AppEngine 1.6.4 SDK on OS X 10.7.3. I am using Python 2.7 and have threadsafe true in app.yaml.
#view_config(route_name='manager_swms', permission='manager', renderer='manager/swms.jinja2')
def manager_swms(request):
"""Generates blobstore url and passes users swms in swms table"""
# generate url for any form upload that may occur
upload_url = blobstore.create_upload_url('/upload_swm')
user = get_current_user(request)
swms = DBSession.query(SWMS).filter_by(owner_id=int(user.id)).all()
return {
"analytics_id": analytics_id,
"user": get_current_user(request),
"upload_url": upload_url,
"swms": [(x.filename, x.blob_key) for x in swms]
}
class BlobstoreUploadHandler(object):
"""Base class for creation blob upload handlers."""
def __init__(self, *args, **kwargs):
self.__uploads = None
def get_uploads(self, field_name=None):
"""Get uploads sent to this handler.
Args:
field_name: Only select uploads that were sent as a specific field.
Returns:
A list of BlobInfo records corresponding to each upload.
Empty list if there are no blob-info records for field_name.
"""
if self.__uploads is None:
self.__uploads = {}
for key, value in self.request.params.items():
if isinstance(value, cgi.FieldStorage):
if 'blob-key' in value.type_options:
self.__uploads.setdefault(key, []).append(
blobstore.parse_blob_info(value))
if field_name:
try:
return list(self.__uploads[field_name])
except KeyError:
return []
else:
results = []
for uploads in self.__uploads.itervalues():
results += uploads
return results
#view_config(route_name='upload_swm', permission='manager')
class UploadHandler(BlobstoreUploadHandler):
''' Handles redirects from Blobstore uploads. '''
def __init__(self, request):
self.request = request
super(UploadHandler, self).__init__()
def __call__(self):
user = get_current_user(self.request)
for blob_info in self.get_uploads('file'):
new_swm = SWMS(
owner_id = int(user.id),
blob_key = str(blob_info.key()),
filename = blob_info.filename,
size = blob_info.size,
)
DBSession.add(new_swm)
DBSession.flush()
# redirect to swms page
return HTTPFound(location='/manager/swms')
In the above code, manager_swms() generates a page which includes a form for uploading a file into the Blobstore. The form works OK and I can see the blob appear in the Blobstore when the form is used. The redirect from the blobstore POST is then to /upload_swm where I successfully take BlobInfo details and place them into a SQL table. All of this is good and the final thing I want to do is redirect to the first page so another file can be uploaded if need be and I can show the list of files uploaded.
As per Pyramid documentation, I am using HTTPFound(location='/manager/swms') [the original page URL] to try and redirect however I get:
ERROR 2012-03-29 22:56:38,170 wsgi.py:208]
Traceback (most recent call last):
File "/Users/tim/work/OHSPro/var/parts/google_appengine/google/appengine/runtime/wsgi.py", line 196, in Handle
result = handler(self._environ, self._StartResponse)
File "lib/dist/pyramid/router.py", line 195, in __call__
foo = response(request.environ, start_response)
File "lib/dist/pyramid/httpexceptions.py", line 291, in __call__
foo = Response.__call__(self, environ, start_response)
File "lib/dist/webob/response.py", line 922, in __call__
start_response(self.status, headerlist)
File "/Users/tim/work/OHSPro/var/parts/google_appengine/google/appengine/runtime/wsgi.py", line 150, in _StartResponse
_GetTypeName(header[1]))
InvalidResponseError: header values must be str, got 'unicode'
INFO 2012-03-29 22:56:38,174 dev_appserver_blobstore.py:408] Upload handler returned 500
INFO 2012-03-29 22:56:38,193 dev_appserver.py:2884] "POST /_ah/upload/ahJkZXZ-cHJvdG8tc2NvaHNwcm9yGwsSFV9fQmxvYlVwbG9hZFNlc3Npb25fXxgTDA HTTP/1.1" 500 -
AppEngine is clearly objecting to unicode in the HTTP header but I'm not doing anything unusual AFAIK. If I drop into pdb and take a look at the HTTPFound object, the headers are:
ResponseHeaders([('Content-Type', 'text/html; charset=UTF-8'), ('Content-Length', '0'), ('Location', '/manager/swms')])
Why would I get a unicode problem from these?

so it looks like you are overriding appengine's supported webob which is 1.1.1 on the 2.7 runtime. And pyramid 1.3 depends on webob>=1.2. This is most likely the problem because it was the Blobstore handler stuff that was keeping the sdk held back at webob==0.9 until SDK 1.6.4 was released.
FWIW, this problem is expected to be resolved by SDK 1.6.5 (late April). The only reason I know this is because I was trying to get all this crap resolved when the 2.7 runtime was deemed ready for general use, yet the SDK didn't support it. see this issue for more details.
If possible, I would suggest running it with pyramid 1.2, I know that works fine on appengine. And Just hold off on moving to 1.3 for a few weeks. :)

Yet another answer... adding a str() fixes the problem, but not the root cause. I spent hours trying to figure out why one particular redirect raised this error while others didn't, before noticing that the URL for the bad redirect was missing the initial "/".
I have no idea why this is the case - possibly an incomplete path is processed differently from a complete one. But if you hit this error try changing:
self.redirect('home.view')
to:
self.redirect('/home.view')