Develop Reference

can't get facebook profile pic from flask-OAuth package

I am using this code for logging in to my app. It works fine, but when I try to get the url for profile pic
pic = facebook.get("/me/picture?fields=url")
I get None in response.
TypeError: must be string or buffer, not None
If I try sending this GET request from facebook Graph API, with same access-token, I get the profile link.
I checked out the version message by printing out get("/me") request, I get
connection: keep-alive
etag: "2c59a2baef08156f18055c64eaa9d9822e35e8f1"
pragma: no-cache
cache-control: private, no-cache, no-store, must-revalidate
date: Fri, 23 Jun 2017 14:20:04 GMT
**facebook-api-version: v2.9**
access-control-allow-origin: *
content-type: text/javascript; charset=UTF-8
Which shows that version is automatically converting when i send request from flask-OAuth as well. Then what is it that I am missing ?

As documented in the Graph API, "me/picture?" will return a 302 redirect to the picture image. To get access to the data about the picture, we need to include redirect=false in the end of query. So I get the url by;
pic = facebook.get('/me/picture?redirect=false').data
print 'picture:', pic['data']["url"]

JSON in post request works in HttpRequester but not in python Requests

I'm stuck in web scraping a page using Python. Basically, the following is the request from HttpRequester (in Mozilla) and it gives me the right response.
POST https://www.hpe.com/h20195/v2/Library.aspx/LoadMore
Content-Type: application/json
{"sort": "csdisplayorder", "hdnOffset": "1", "uniqueRequestId": "d6da6a30bdeb4d77b0e607a6b688de1e", "test": "", "titleSearch": "false", "facets": "wildcatsearchcategory#HPE,cshierarchycategory#No,csdocumenttype#41,csproducttype#18964"}
-- response --
200 OK
Cache-Control: private, max-age=0
Content-Length: 13701
Content-Type: application/json; charset=utf-8
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Sat, 28 May 2016 04:12:57 GMT
Connection: keep-alive
The exact same operation in python 2.7.1 using Requests, fails with an error. The following is the code snippet:
jsonContent = {"sort": "csdisplayorder", "hdnOffset": "1", "uniqueRequestId": "d6da6a30bdeb4d77b0e607a6b688de1e", "test": "", "titleSearch": "false", "facets": "wildcatsearchcategory#HPE,cshierarchycategory#No,csdocumenttype#41,csproducttype#18964"}
catResponse = requests.post('https://www.hpe.com/h20195/v2/Library.aspx/LoadMore', json = jsonContent)
The following is the error that I get:
{"Message":"Value cannot be null.\r\nParameter name: source","StackTrace":" at
System.Linq.Enumerable.Contains[TSource](IEnumerable`1 source, TSource value, I
EqualityComparer`1 comparer)\r\n
More information:
The Post request that I'm looking for is fired upon:
opening this web page: https://www.hpe.com/h20195/v2/Library.aspx?doctype=41&doccompany=HPE&footer=41&filter_doctype=no&filter_doclang=no&country=&filter_country=no&cc=us&lc=en&status=A&filter_status=rw#doctype-41&doccompany-HPE&prodtype_oid-18964&status-a&sortorder-csdisplayorder&teasers-off&isRetired-false&isRHParentNode-false&titleCheck-false
Clicking on the "Load more" grey button at the end of the page
I'm capturing the exact set of request headers and response from the browser operation and trying to mimic that in Postman, Python code and HttpRequester (Mozilla).
It flags the same error (mentioned above) with Postman and Python, but works with no headers set on my part with HttpRequester.
Can anyone think of an explanation for this?

If both Postman and requests are receiving an error, then there is more context than what HttpRequester is showing. There are a number of headers that I'd expect to be set almost always, including User-Agent and Content-Length, that are missing here.
The usual suspects are cookies (look for Set-Cookie headers in earlier requests, preserve those by using a requests.Session() object), the User-Agent header and perhaps a Referrer header, but do look for other headers like anything starting with Accept, for example.
Have HttpRequester post to http://httpbin.org/post instead for example, and inspect the returned JSON, which tells you what headers were sent. This won't include cookies (those are domain-specific), but anything else could potentially be something the server looks for. Try such headers one by one if cookies are not helping.

Testing web-tornado using Firefox's HttpRequest addon

I am testing my web-tornado application using Firefox's HttpRequest add-on but after I log in and receive my secure cookie data, I am not able to re-use it to consume protected methods.
This is my response data:
POST http://mylocalurl:8888/user/login
Content-Type: application/x-www-form-urlencoded
Login=mylogin;Pass=123
-- response -- 200 OK Content-Length: 33
Content-Type: text/html; charset=UTF-8
Server: TornadoServer/2.2.1
Set-Cookie:
IdUser="Mjk=|1395170421|ffaf0d6fecf2f91c0dccca7cab03d799ef6637a0";
expires=Thu, 17 Apr 2014 19:20:21 GMT; Path=/
{
"Success": true }
-- end response --
Now why I am trying to do is to configure HttpRequester to use this cookie for my new requests. I tried to add it using the "Headers" tab but my server keeps sending me a 403, Forbidden.
Can anyone help me on this ? It could be with another tool (for linux) too.

I really like fiddler2 for these kind of things and there's an alpha build for mono that you may wish to try out: http://www.telerik.com/download/fiddler
If you don't mind paid software you can use Charles, for which there is a free trial.
And if you are testing and already using python, why not use a simple python script with requests and its Session object with cookie-persistence..

Making Head Requests in Twisted

I am relatively new to using Twisted and I am having trouble returning the content-length header when performing a basic head request. I have set up an asynchronous client already but the trouble comes in this bit of code:
def getHeaders(url):
d = Agent(reactor).request("HEAD", url)
d.addCallbacks(handleResponse, handleError)
return d
def handleResponse(r):
print r.code, r.headers
whenFinished = twisted.internet.defer.Deffered()
r.deliverBody(PrinterClient(whenFinished))
return whenFinished
I am making a head request and passing the url. As indicated in this documentation the content-length header is not stored in self.length, but can be accessed from the self.headers response. The output is returning the status code as expected but the header output is not what is expected. Using "uhttp://www.espn.go.com" as an example it currently returns:
Set-Cookie: SWID=77638195-7A94-4DD0-92A5-348603068D58;
path=/; expires=Fri, 31-Jan-2034 00:50:09 GMT; domain=go.com;
X-Ua-Compatible: IE=edge,chrome=1
Cache-Control: max-age=15
Date: Fri, 31 Jan 2014 00:50:09 GMT
P3P: CP="CAO DSP COR CURa ADMa DEVa TAIa PSAa PSDa IVAi IVDi CONi
OUR SAMo OTRo BUS PHY ONL UNI PUR COM NAV INT DEM CNT STA PRE"
Content-Type: text/html; charset=iso-8859-1
As you can see, no content-length field is returned. If the same request is done in requests then the result will contain the content-length header:
r = requests.head("http://www.espn.go.com")
r.headers
({'content-length': '46133', 'content-encoding': 'gzip'...})
(rest omitted for readability)
What is causing this problem? I am sure it is a simple mistake on my part but I for the life of me cannot figure out what I have done wrong. Any help is appreciated.

http://www.espn.go.com/ returns one response if the client sends an Accept-Encoding: gzip header and another response if it doesn't.
One of the differences between the two responses is the inclusion of the Content-Length header.
If you want to make requests using Agent including Accept-Encoding: gzip then take a look at ContentDecoderAgent or the third-party treq package.

http allows (but does not REQUIRE) entity headers in responses to HEAD requests. The only restriction it places is that 200 responses to HEAD requests MUST NOT include an entity payload. Its up to the origin server to decide which, if any entity headers it would like to include.
In the case of Content-Length, it makes sense for this to be optional for HEAD; if the entity will be computed dynamically (as with compressing/decompressing content), it's better for the server to avoid the extra work of computing the content length when the request won't include the content anyway.

Google App Engine OAuth endpoints throwing 400 in production

I implemented the experimental OAuth support for Google App Engine using Python, and have it working locally, but the endpoints are throwing a 400 when I deploy to appspot.
For example, the url http(s)://my-app.appspot.com/_ah/OAuthGetRequestToken returns a 400, but locally that url pattern behaves as expected.
I have tried both http and https, and assumed that appspot handles the ssl cert.
UPDATE
I've been using the OAuth Playground to test my code. Despite documentation, it seems Registering your app is required. Go here for instructions on how to register. According to documentation during the registration process, certificate is not required when running on App Engine. Playground is showing more detail on the error - "signature invalid". If I understand correctly, the signature is produced from a signature base string. In this case I am using the base string 7DYB6MJ2s-IQcd7VJYJUmcct .
GET /accounts/OAuthGetRequestToken?scope=https%3A%2F%2Fmail.google.com%2Fmail%2Ffeed%2Fatom HTTP/1.1
Host: www.google.com
Accept: */*
Authorization: OAuth oauth_version="1.0", oauth_nonce="168cfd60a93a46caa38dddfdcedd9de9", oauth_timestamp="1305315895", oauth_consumer_key="xxxxxxx.appspot.com", oauth_callback="http%3A%2F%2Fgooglecodesamples.com%2Foauth_playground%2Findex.php", oauth_signature_method="HMAC-SHA1", oauth_signature="4J5faUujE0VNaybyvFCiEPY7DQ8%3D"
HTTP/1.1 400 Bad Request
Content-Type: text/plain; charset=UTF-8
Date: Fri, 13 May 2011 19:44:55 GMT
Expires: Fri, 13 May 2011 19:44:55 GMT
Cache-Control: private, max-age=0
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Content-Length: 451
Server: GSE
**signature_invalid**
base_string:GET&https%3A%2F%2Fwww.google.com%2Faccounts%2FOAuthGetRequestToken&oauth_callback%3Dhttp%253A%252F%252Fgooglecodesamples.com%252Foauth_playground%252Findex.php%26oauth_consumer_key%3Dxxxxxx.appspot.com%26oauth_nonce%3D168cf60a94caa38e2defdcedd9de9%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1305315895%26oauth_version%3D1.0%26scope%3Dhttps%253A%252F%252Fmail.google.com%252Fmail%252Ffeed%252Fatom
FINAL UPDATE
There were two things causing the 400. First, the app was not registered. Google's documentation says it's optional, but that is not the case apparently. Secondly, the Request was not properly signed. Here is an excellent debugging tool to test your OAuth Requests: Oauth Playground

You must register your domain in order to have OAuth working on production.
Although the following docs state that Registering is Optional:
http://code.google.com/apis/accounts/docs/RegistrationForWebAppsAuto.html
It doesn't work without the Registration since January.
Look at the link above and Register your domain/application. You don't need to submit a certificate, this is still optional.

Which request method it is? In dev_appserver_oauth.py I see:
if method != 'GET' and method != 'POST':
outfile.write('Status: 400\r\n')
return
So it'll only work for GET or POST requests.

First ensure you have enabled Federated Login in your Application Settings.
From your description it sounds like you might just be performing a direct GET request to /_ah/OAuthGetRequestToken without any of the other required parameters of oAuth. This will work on the dev_appserver as it is simply a mockup of oAuth to let you flesh out your code.
See the parameters listed on the OAuthGetRequestToken description page for what is required and how to deal with signing. I believe you can ignore scope for GAE though