I'm trying to use urlfetch to make a request to my application (the same application which is sending the request) however, it doesn't work.
My code is as follows;
uploadurl = 'http://myapp.appspot.com/posturl'
result = urlfetch.fetch(
url=uploadurl,
payload=data,
method=urlfetch.POST,
headers={'Content-Type': 'application/x-www-form-urlencoded'})
There is no error at all when I call this, and everything seems to work correctly, however the request never arrives. For debugging purposes, I changed the uploadurl to a different application which I own and it worked fine. Any ideas why I can't send requests using urlfetch to the same application?
The full (real) url that I would call is made by
session = str(os.urandom(16).encode('hex'))
uploadurl = blobstore.create_upload_url('/process?session=' + session)
So I can't understand how that could be incorrect as the url is made for me.
Thanks.
I don't know how you're verifying that the request "never arrives". The blobstore URLs are not handled by your application's actual code, but by the App Engine runtime itself, so if you're looking in the logs you won't see that request there.
I think it is not possible, to prevent endless loop. From the urlfetch api documentation page:
To prevent an app from causing an endless recursion of requests, a
request handler is not allowed to fetch its own URL. It is still
possible to cause an endless recursion with other means, so exercise
caution if your app can be made to fetch requests for URLs supplied by
the user.
Related
I find this code and I really don't understand it, how is it possible to send data (not query) with GET request
response = requests.get(
check_all_info_url_2, files=multipart_form_data, timeout=30)
and what is files= argument in the get request.
Since requests.get is just a wrapper function this will just call requests.request. Unless requests.session implementes any checking, it will happily send off a GET request with multipart data in it.
Is this valid? Not to my knowledge, although I'm willing to be proven wrong. No api I have ever written would accept file upload on a GET request. But not every server will even check the method, so perhaps this code is interacting with a badly written server which doesn't reject for wrong method, or perhap's it's even interacting with a worse server which expects file upload with GET. There are lots of broken servers out there ;)
In any case, the reason this works with requests is that it just passes keyword arguments through to the underlying session without performing any kind of validation.
I am running a Flask restful API behind an NGINX web server on AWS. I am hitting that with a python module from my Pi.
Everything worked fine when I was using HTTP to make calls to the api. But I just locked down my api so only HTTPS is possible. I changed the UIRL used by my python module but it now fails. The code is quite simple...here is an extract:
jsonpkg = {'subscriberID': self.api_login, 'token': self.api_token,
'content': speech_content}
headers = {'Content-Type': 'application/json'}
r = requests.post(self.api_apiurl, data=json.dumps(jsonpkg), headers=headers)
The values are being correct set by the class init section. And I am importing the requests module at the top. Error messages indicate it is using python 2.7. However when I monitor the API I can see its not even hitting the server. I can point a browser to the api and its working fine.
Am I to understand the requests module in python 2.7 does not support https?
Are there additional parameters I need to send for https?
Aha! With a little more digging into the request module docs I found the answer. If I use the following
r = requests.post(self.api_apiurl, data=json.dumps(jsonpkg), headers=headers, verify=False)
then it works. So the issue is with verifying the cert. I am not quite sure why the browser gets by without this...but perhaps it does the extra stuff automatically. So I either need to NOT verify the cert or have a local copy(?) that can be verified.
Final Update:
I finally worked out how to concatenate my site certificate with the chain certificate (and understand why). This site here was a great help. Also, once they are concatenated you will probably get a second error, which if you google it you will find is caused by the need for a carriage return after the first certificate and before the second (edit the resulting concatenated file with notepad). I then was able to return the post to using "verify=True" which made the warnings about no verification go away.
I'm currently having a hard time getting some of my Django tests to work.
What I'm trying to do is test if a given URL (the REST API I want to consume) is up and running (returning status code 200) and later on if it's responding the expected values.
However, all I get returned is a status code 404 (Page not found), even though the URL is definitely the right one. (Tried the exact string in my browser)
This is the code:
from django.test import TestCase
class RestTest(TestCase):
def test_api_test_endpoint(self):
response = self.client.get("http://ip.to.my.api:8181/test/")
self.assertEqual(response.status_code, 200, "Status code not equals 200")
It always returns a 404 instead of a 200...
Anyone knows what I do wrong here?
self.client is not a real HTTP client; it's the Django test client, which simulates requests to your own app for the purposes of testing. It doesn't make HTTP requests, and it only accepts a path, not a full URL.
If you really needed to check that an external URL was up, you would need a proper HTTP client like requests. However this doesn't seem to be an appropriate thing to do in a test case, since it depends on an external API; if that API went down, suddenly your tests would fail, which seems odd. This could be something that you do in a monitoring job, but not in a test.
I'm developing an app using an API in Eve and AngularJS.
To avoid CORS issue, I made a simple NodeJS server to serve my static files.
However, even after allowing All domains on my Python Eve API by writing 'X_DOMAINS': '*' (I tested with Curl and it worked) I'm getting issues in my browser when I want to call the API using $http.
This is the error I get in chrome:
XMLHttpRequest cannot load http://localhost:5000/regions. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8080' is therefore not allowed access.
I even wrote this in my Node app, even though I figured it would be useless:
app.all('/*', function(req, res, next) {
res.header("Access-Control-Allow-Origin", "*");
res.header("Access-Control-Allow-Headers", "X-Requested-With, Content-Type");
res.header("Access-Control-Allow-Methods", "GET, POST","PUT");
next();
});
Nothing has worked so far, thanks in advance for the help!
Ran into some similar issues with javascript. I could get it running with some tweaks on eve, but in the end it was not enough. The BadRequest responses are sent by the flask engine, not eve, disabling any javascript attempt at handling errors. It appears there is a nice flask extension CORS that covers it well:
from flask.ext.cors import CORS
....
CORS(app)
That was all that I needed in the end.
Add this in your response headers:
res.header("Access-Control-Allow-Origin", "http://localhost:8080");
Remove the following:
http://localhost:8080
Or what you can try is to get the request origin and add it to the response. Something like this:
var hostName = req.get('host');
res.header("Access-Control-Allow-Origin", hostName);
On some cases it might work, specially if it is a GET call. But if it is a POST, some browsers will block it.
Another option is to disable the web security on your browser, (though not the best thing to do.) You can read here for that.
Also go through the posts here , so that you can have a better idea of the problem.
In the following command:
res.header("Access-Control-Allow-Headers", "X-Requested-With, Content-Type");
you set allowed headers, maybe you sent other headers e.g. "Authorization", in this case you must include these headers to above command
They didn't mention this in python documentation. And recently I'm testing a website simply refreshing the site using urllib2.urlopen() to extract certain content, I notice sometimes when I update the site urllib2.urlopen() seems not get the newly added content. So I wonder it does cache stuff somewhere, right?
So I wonder it does cache stuff somewhere, right?
It doesn't.
If you don't see new data, this could have many reasons. Most bigger web services use server-side caching for performance reasons, for example using caching proxies like Varnish and Squid or application-level caching.
If the problem is caused by server-side caching, usally there's no way to force the server to give you the latest data.
For caching proxies like squid, things are different. Usually, squid adds some additional headers to the HTTP response (response().info().headers).
If you see a header field called X-Cache or X-Cache-Lookup, this means that you aren't connected to the remote server directly, but through a transparent proxy.
If you have something like: X-Cache: HIT from proxy.domain.tld, this means that the response you got is cached. The opposite is X-Cache MISS from proxy.domain.tld, which means that the response is fresh.
Very old question, but I had a similar problem which this solution did not resolve.
In my case I had to spoof the User-Agent like this:
request = urllib2.Request(url)
request.add_header('User-Agent', 'Mozilla/5.0')
content = urllib2.build_opener().open(request)
Hope this helps anyone...
Your web server or an HTTP proxy may be caching content. You can try to disable caching by adding a Pragma: no-cache request header:
request = urllib2.Request(url)
request.add_header('Pragma', 'no-cache')
content = urllib2.build_opener().open(request)
If you make changes and test the behaviour from browser and from urllib, it is easy to make a stupid mistake.
In browser you are logged in, but in urllib.urlopen your app can redirect you always to the same login page, so if you just see the page size or the top of your common layout, you could think that your changes have no effect.
I find it hard to believe that urllib2 does not do caching, because in my case, upon restart of the program the data is refreshed. If the program is not restarted, the data appears to be cached forever. Also retrieving the same data from Firefox never returns stale data.