Develop Reference

Status parameter not working when using python blogger api

I'm trying to use google-api-python-client 1.12.5 with Service account auth under Python 3.8. It seems to me that the when specifying the status parameter, Google responds with a 404 HTTP code. I can't figure out why. I also looked in the docs but I can't relate anything to this error.
I have pasted my code. The error is happening in the third call.
This is the code:
from google.oauth2 import service_account
from googleapiclient.discovery import build
SCOPES = ['https://www.googleapis.com/auth/blogger']
SERVICE_ACCOUNT_FILE = 'new_service_account.json'
BLOG_ID = '<your_blog_id>'
credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
service = build('blogger', 'v3', credentials=credentials)
p = service.posts()
# FIRST
promise = p.list(blogId=BLOG_ID)
result = promise.execute()
# SECOND
promise = p.list(blogId=BLOG_ID, orderBy='UPDATED')
result = promise.execute()
#THIRD
promise = p.list(blogId=BLOG_ID, orderBy='UPDATED', status='DRAFT')
result = promise.execute() # <===== ERROR HAPPENS HERE!!!!
service.close()
And this is the traceback:
Traceback (most recent call last):
File "/home/madtyn/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/202.7660.27/plugins/python/helpers/pydev/pydevd.py", line 1448, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "/home/madtyn/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/202.7660.27/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/home/madtyn/PycharmProjects/blogger/main.py", line 24, in <module>
result = promise.execute()
File "/home/madtyn/venvs/blogger/lib/python3.8/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
return wrapped(*args, **kwargs)
File "/home/madtyn/venvs/blogger/lib/python3.8/site-packages/googleapiclient/http.py", line 915, in execute
raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 404 when requesting https://blogger.googleapis.com/v3/blogs/<blog_id>/posts?orderBy=UPDATED&status=DRAFT&alt=json returned "Not Found">
python-BaseException

I can reproduce this issue... Adding status=DRAFT will return 404 but any other filter is working...
Tried with service account and your code: 404
Tried with API Key like this result = requests.get('https://blogger.googleapis.com/v3/blogs/<blog_id>/posts?status=DRAFT&orderBy=UPDATED&alt=json&key=<api_key>'): 404
Extracted "access_token" from service account (credentials.token after a call): result = requests.get('https://blogger.googleapis.com/v3/blogs/<blog_id>/posts?status=DRAFT&orderBy=UPDATED&alt=json&access_token=<extracted_service_account_token>'): 404
But very strangely if I use access_token given by "Try this API" here : https://developers.google.com/blogger/docs/3.0/reference/posts/list?apix_params={"blogId"%3A"blog_id"%2C"orderBy"%3A"UPDATED"%2C"status"%3A["DRAFT"]%2C"alt"%3A"json"} it's works !
Used that token with requests give me my blog post in draft status...
Just copy/paste raw Authorization header inside that script:
import requests
blog_id = '<blog_id>'
headers = {
'Authorization' : 'Bearer <replace_here>'
}
# Using only Authorization header
result = requests.get(
'https://blogger.googleapis.com/v3/blogs/%s/posts?status=DRAFT&orderBy=UPDATED&alt=json' % (blog_id),
headers=headers
)
print(result)
# This should print DRAFT if you have at least one draft post
print(result.json()['items'][0]['status'])
# Using "access_token" param constructed with Authorization header splited to have only token
result = requests.get('https://blogger.googleapis.com/v3/blogs/%s/posts?status=DRAFT&orderBy=UPDATED&alt=json&access_token=%s' % (blog_id, headers['Authorization'][len('Bearer '):]))
print(result)
# This should print DRAFT if you have at least one draft post
print(result.json()['items'][0]['status'])
Results I have currently:
The bug doesn't seem to come from the library but rather from the token rights...However I also used the console normally to generate accesses like you.
To conclude I think it's either a bug or it's voluntary from Google... I don't know how long the "Try this API" token is valid but it is currently the only way I found to get the draft articles... Maybe you can try to open a bug ticket but I don't know specifically where it is possible to do that.

Python3, Requests: How to merge CookieJars

I am learning Python and using the Requests Lib.
I want to use a CookieJar to store cookies, but I cannot find out how to add a response's Cookies to an existing CookieJar:
CookieJar.extract_cookies requires a request object - I dont understand which request to reference and why. I want to add the Cookies to a CookieJar, not to a request...
So I tried
cj= http.cookiejar.CookieJar()
tmp= requests.utils.dict_from_cookiejar(resp.cookies)
requests.utils.add_dict_to_cookiejar(cj, tmp)
the third line Fails:
File "[...]\Python35-32\lib\site-packages\requests\utils.py", line 336, in add_dict_to_cookiejar
return cookiejar_from_dict(cookie_dict, cj)
File "[...]\Python35-32\lib\site-packages\requests\cookies.py", line 515, in cookiejar_from_dict
names_from_jar = [cookie.name for cookie in cookiejar]
File "[...]\Python35-32\lib\site-packages\requests\cookies.py", line 515, in <listcomp>
names_from_jar = [cookie.name for cookie in cookiejar]
AttributeError: 'str' object has no attribute 'name'
As the Cookiejar of Requests is a dict as well, I finally tried
requests.utils.add_dict_to_cookiejar(cj, resp.cookies)
which Fails with the same error.....
what am I doing wrong?

Try this way
# Create cookie one
one = requests.cookies.RequestsCookieJar()
# Create cookie two
two = requests.cookies.RequestsCookieJar()
# set some cookie value
one.set("one_key", "one_value")
two.set("two_key", "two_value")
print(one)
<RequestsCookieJar[<Cookie one_key=one_value for />]>
print(two)
<RequestsCookieJar[<Cookie two_key=two_value for />]>
# Now merge
one.update(two)
<RequestsCookieJar[<Cookie one_key=one_value for />, <Cookie two_key=two_value for />]>

How to deal with incorrect json format ? (simplejson.scanner.JSONDecodeError: Expecting value: line 1 column 1 (char 0))

1) A script that I had working for many weeks broke a few days ago. I can't parse the JSON properly now. So this is not net new code, it's something that has been in operation for months.
2) Something changed in the servicing website, and it's making the JSON non-compliant but I have been trying to circumvent the issue with no success. I think it may be an extra space or something, but I can't change the information returned from the servicing website.
3) I know the json is not compliant because I used a validator (https://jsonformatter.curiousconcept.com/) by putting the URL of the service I need with my credentials/format needed, and I get proper results but the validation fails with "Invalid encoding, expecting UTF-8, UTF-16 or UTF-32.[Code 29, Structure 0]". There is a way to tell the validator not to validate and the Json looks proper, but Python will not have anything to do with it. When I run my script it reports:
simplejson.scanner.JSONDecodeError: Expecting value: line 1 column 1 (char 0).
4) Below is my URL entry manually and the script. I have obfuscated all sensitive and personal information so the URL if you try won't work, but when I do the non-obfuscated format, I do get a JSON response.
5) Manual URL (obfuscated):
https://mystuff.mydevices.com/Membership/SomeOtherURLrelated?appId=BB8pQgg123450WHahgl12345nAkkX67890q2HrHD7H1nabcde5KqtN654321LB%2fi&securityToken=null&username=myemail#somedomain.com&password=mypassword&culture=en
6) If I manually opened a browser and put the previous real URL (unmodified), the browser responds with json. An example (obfuscated):
{"UserId":0,"SecurityToken":"abcdb8c3-1ef1-1110-1234-402a914f52aa","ReturnCode":"0","ErrorMessage":"","BrandId":2,"BrandName":"Mydevicebrandname","RegionId":1}
7) What can I do to overcome this ? any suggestions ? I have been reading and testing but no luck!
8) Now the script (obfuscated) that basically builds the previous URL and extracts from the JSON a one-time security token that then I can use for other purposes in a much bigger application:
import json,requests
APPID = 'BB8pQgg123450WHahgl12345nAkkX67890q2HrHD7H1nabcde5KqtN654321LB%2fi'
USERNAME = 'myemail#somedomain.com'
PASSWORD = 'mypassword'
CULTURE = 'en'
SERVICE = 'https://mystuff.mydevices.com'
def get_token_formydevices():
payload = {'appId': APPID,
'securityToken': 'null',
'username': USERNAME,
'password': PASSWORD,
'culture': CULTURE,}
login_url = SERVICE + '/Membership/SomeOtherURLrelated'
try:
r = requests.get(login_url, params=payload)
except requests.exceptions.RequestException as err:
return
data = r.json()
if data['ReturnCode'] != '0':
print(data['ErrorMessage'])
sys.exit(1)
return data['SecurityToken']
tokenneeded = get_token_formydevices()
print tokenneeded
9) When I run the previous code this is what I get back:
Traceback (most recent call last):
File "testtoken.py", line 33, in <module>
tokenneeded = get_token_formydevices()
File "testtoken.py", line 26, in get_token_formydevices
data = r.json()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/models.py", line 826, in json
return complexjson.loads(self.text, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/simplejson/__init__.py", line 516, in loads
return _default_decoder.decode(s)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/simplejson/decoder.py", line 370, in decode
obj, end = self.raw_decode(s)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/simplejson/decoder.py", line 400, in raw_decode
return self.scan_once(s, idx=_w(s, idx).end())
simplejson.scanner.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I found a solution and want to share.
So I was very puzzled by the fact that I could open the servicing URL in a web browser and get some json back, but I couldn't do it in my script or even just using cURL. I kept on getting "request denied" even though the request worked from the browser, so it had to be something else.
So I started experimenting and sending in the request user agent information in my script and voilá! the code below is working although I obfuscated the original URL and my credentials for protection.
I further want to explain that I was doing this as the servicing URL provides back a one-time token that I can then use to trigger another action. So I needed this routine and executive for as many times I need to carry on specific actions, so all I wanted was to retrieve the token from the json form that url. Hope this makes more sense now with the code below.
import json,urllib2
url='https://mystuff.mydevices.com/Membership/SomeOtherURLrelated?appId=BB8pQgg123450WHahgl12345nAkkX67890q2HrHD7H1nabcde5KqtN654321LB%2fi&securityToken=null&username=myemail#somedomain.com&password=mypassword&culture=en'
request = urllib2.Request(url)
#request.add_header('User-Agent', 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36') # <--- this works too
request.add_header('User-Agent', 'Mozilla/5.0')
data = json.loads(str(urllib2.urlopen(request).read()))
token = data["SecurityToken"]
print token

GAE Python URL Fetch throws InvalidURLError [duplicate]

This question already has an answer here:
Python GAE urlfetch credentials
(1 answer)
Closed 7 years ago.
GAE Python URL Fetch throws InvalidURLError while the same URL works perfectly with Postman ( Google Chrome App ).
CODE
url = "https://abcdefgh:28dfd95928dfd95928dfd95928dfd95928dfd95928dfd959#twilix.exotel.in/v1/Accounts/abcdefgh/Sms/send"
form_fields = {
"From": "08039511111",
"To": "+919844100000",
"Body": "message for you"
}
form_data = urllib.urlencode (form_fields)
try:
result = urlfetch.fetch(url=url,
payload=form_data,
method=urlfetch.POST,
headers={'Content-Type': 'application/x-www-form-urlencoded' }
)
logging.info ("result = ["+repr (result)+"] ")
except Exception:
logging.error ("Exception. ["+traceback.format_exc ()+"] ")
OUTPUT LOGS
2016-01-21 15:48:23.368 +0530 E Exception. [
Traceback (most recent call last): File "main.py", line 27, in get method=urlfetch.POST,
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/urlfetch.py", line 271, in fetch return rpc.get_result()
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result return self.__get_result_hook(self)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/urlfetch.py", line 389, in _get_fetch_result 'Invalid request URL: ' + url + error_detail) InvalidURLError: Invalid request URL: https://abcdefgh:28dfd95928dfd95928dfd95928dfd95928dfd95928dfd959#twilix.exotel.in/v1/Accounts/abcdefgh/Sms/send ]
For security purpose, I have replaced sensitive text in the URL with similar different characters.

The code indicates an INVALID_URL RPC error code was received from the urlfetch service.
The most common occurence seems to be due to the URL length limit (check if your unedited URL hits that): Undocumented max length for urlfetch URL?
A long time ago it was also seen for very slow URLs (in Go land, but I suspect the urlfetch service itself is the same serving all language sandboxes) - unsure if this still stands, I also see a DEADLINE_EXCEEDED error code as well which might have been introduced specifically for such case in the meantime): Google App Engine Go HTTP request to a slow page
The failure might also be related to incorrect parsing of the rather unusual "host" portion of your URL foo:blah#hostname. Check if it you're getting the same error if dropping the foo:blah# portion. If it's indeed the case you might want to file an issue with Google - the URL seems valid, works with curl as well.

I found the problem and the solution.
We need to specify the HTTP auth info using headers.
urlfetch.make_fetch_call ( rpc,
url,
method = urlfetch.POST,
headers = { "Authorization" : "Basic %s" % base64.b64encode ( URL_USERNAME+":"+URL_PASSOWRD ) },
)
Courtesy
https://stackoverflow.com/a/8454580/1443563 by raugfer

XML parser syntax error

So I'm working with a block of code which communicates with the Flickr API.
I'm getting a 'syntax error' in xml.parsers.expat.ExpatError (below). Now I can't figure out how it'd be a syntax error in a Python module.
I saw another similar question on SO regarding the Wikipedia API which seemed to return HTML intead of XML. Flickr API returns XML; and I'm also getting the same error when there shouldn't be a response from Flickr (such as flickr.galleries.addPhoto)
CODE:
def _dopost(method, auth=False, **params):
#uncomment to check you aren't killing the flickr server
#print "***** do post %s" % method
params = _prepare_params(params)
url = '%s%s/%s' % (HOST, API, _get_auth_url_suffix(method, auth, params))
payload = 'api_key=%s&method=%s&%s'% \
(API_KEY, method, urlencode(params))
#another useful debug print statement
#print url
#print payload
return _get_data(minidom.parse(urlopen(url, payload)))
TRACEBACK:
Traceback (most recent call last):
File "TESTING.py", line 30, in <module>
flickr.galleries_create('test_title', 'test_descriptionn goes here.')
File "/home/vlad/Documents/Computers/Programming/LEARNING/curatr/flickr.py", line 1006, in galleries_create
primary_photo_id=primary_photo_id)
File "/home/vlad/Documents/Computers/Programming/LEARNING/curatr/flickr.py", line 1066, in _dopost
return _get_data(minidom.parse(urlopen(url, payload)))
File "/usr/lib/python2.6/xml/dom/minidom.py", line 1918, in parse
return expatbuilder.parse(file)
File "/usr/lib/python2.6/xml/dom/expatbuilder.py", line 928, in parse
result = builder.parseFile(file)
File "/usr/lib/python2.6/xml/dom/expatbuilder.py", line 207, in parseFile
parser.Parse(buffer, 0)
xml.parsers.expat.ExpatError: syntax error: line 1, column 62
(Code from http://code.google.com/p/flickrpy/ under New BSD licence)
UPDATE:
print urlopen(url, payload) == <addinfourl at 43340936 whose fp = <socket._fileobject object at 0x29400d0>>
Doing a urlopen(url, payload).read() returns HTML which is hard to read in a terminal :P but I managed to make out a 'You are not signed in.'
The strange part is that Flickr shouldn't return anything here, or if permissions are a problem, it should return a 99: User not logged in / Insufficient permissions error as it does with the GET function (which I'd expect would be in valid XML).
I'm signed in to Flickr (in the browser) and the program is properly authenticated with delete permissions (dangerous, but I wanted to avoid permission problems.)

SyntaxError normally means an error in Python syntax, but I think here that expatbuilder is overloading it to mean an XML syntax error. Put a try:except block around it, and print out the contents of payload and to work out what's wrong with the first line of it.
My guess would be that flickr is rejecting your request for some reason and giving back a plain-text error message, which has an invalid xml character at column 62, but it could be any number of things. You probably want to check the http status code before parsing it.
Also, it's a bit strange this method is called _dopost but you seem to actually be sending an http GET. Perhaps that's why it's failing.

This seems to fix my problem:
url = '%s%s/?api_key=%s&method=%s&%s'% \
(HOST, API, API_KEY, method, _get_auth_url_suffix(method, auth, params))
payload = '%s' % (urlencode(params))
It seems that the API key and method had to be in the URL not in the payload. (Or maybe only one needed to be there, but anyways, it works :-)