This is my python code:
import requests
from flask import Flask
app = Flask(__name__)
#app.route("/")
def index():
url = 'https://google.co.in'
headers = {
'User-Agent':'blahblahblah'
}
r = requests.get(url, headers=headers)
return 'check terminal'
So this is the way to change request headers in python requests. But if I open the url and see developer options > Network > Request headers. I see default as user agent. Means it simply doesn't work.
The request you're making is by the server, not the client (the web browser). The index page served by flask goes to a client. The client doesn't make the requests.get request you've written here, instead the server does.
Instead, the client only requests whatever you're returning from the route, which here is 'check terminal' (which should not work, and should be something in the lines of return jsonify(result='check terminal')), and is not concerned about what the server is doing internally.
So as #brunns has commented, these two requests are different. If you want to check the headers of your request, maybe try httpbin.org.
Related
I am working on a url redirector application, in Python. The idea of this application is simple: when a user performs an http request to a certain domain (e.g. to https://google.com), the app stops the request and performs another http request (e.g. to https://github.com), thereby redirecting the user to the second page.
Unfortunately, I have looked through SO and I haven't found any question that addresses this issue directly:
Intercept and filter HTTP request doesn't redirect
Sending a HTTP request using python doesn't either
Admittedly, I only have some fabricated pseudocode to demonstrate what I wish to do, but it may prove useful:
import requests
original_site = "https://google.com"
redirect_site = "https://github.com"
def redirect_request():
if requests.get(original_site) == True:
requests.kill(request.original_site.id)
requests.get(redirect_site)
I greatly appreciate any suggestions.
EDIT - Here is a clarification of what I mean:
The user runs my python script, which I will call foobar.py, as follows:
python foobar.py
Then the user opens their web browser, and enters the url https://google.com, and as the script is running, the user will not actually visit https://google.com, but be redirected to https://github.com
One option is if you are trying to build a lightweight web app using python where you can mess with HTTP redirects, you can use Flask to accept a GET on a route and then simply do a redirect.
from flask import Flask, redirect
app = Flask(__name__)
#app.route('/')
def index():
return redirect("https://www.google.com")
But to answer your question more directly, you don't "stop" the initial request. You simply find a way to handle it and serve back a response that you specify.
I have a simple flask server
import pdb
from flask import Flask, jsonify, abort, make_response, request, send_from_directory
from flask_cors import CORS, cross_origin
from pprint import pprint
import argparse
from mylib.mylib_rest_api import ProcessRestApiRequest
DEBUG=True
app = Flask(__name__)
CORS(app)
parser = argparse.ArgumentParser(description='Run the server')
parser.add_argument('--ip-address', default='127.0.0.1', help='Server IP Address. Default: %(default)s')
parser.add_argument('--port', type=int, default=8081, help='Server port')
args = parser.parse_args()
#app.errorhandler(404)
def not_found(error):
return make_response(jsonify({'error': 'Not found'}), 404)
def do_pre_serve_actions():
if not request.json: abort(400)
# Extract the data
dictReq = request.get_json(force=True)
if DEBUG:
pprint('dictReq: '+str(dictReq))
return dictReq
def do_post_serve_actions(dictResp):
if DEBUG:
pprint("dictResp: "+str(dictResp))
dictJSONResp = jsonify(dictResp)
objRespADF = make_response(dictJSONResp)
return objRespADF
#app.route('/<target>', methods=['POST'])
def serve(target):
dictReq = do_pre_serve_actions()
dictResp = ProcessRestApiRequest(dictReq, target)
return do_post_serve_actions(dictResp)
if __name__ == '__main__':
app.run(debug=DEBUG, host=args.ip_address, port=args.port)
This is how a request looks like:
makeRequestAndSendData(xhr, dict) {
dict['Interface'] = this.getChipInterface();
var data = JSON.stringify(dict);
var url = this.server.getUrl();
console.log("url: "+url);
console.log("Request Dictionary:");
console.log(dict);
xhr.open("POST", url, true);
xhr.setRequestHeader("Access-Control-Allow-Origin", "*");
xhr.setRequestHeader("Access-Control-Allow-Headers","*");
xhr.setRequestHeader("Access-Control-Allow-Credentials", "true");
xhr.setRequestHeader("Content-type","application/json");
xhr.send(data);
}
Here is what I'm getting:
Failed to load http://192.168.0.132:8084/mychip: Response to preflight
request doesn't pass access control check: The
'Access-Control-Allow-Origin' header has a value 'null' that is not
equal to the supplied origin. Origin 'null' is therefore not allowed
access.
What am I doing wrong? I've looked everywhere online and it seems like it got everything I need. Am I missing something?
This can happen with Chrome and WebKit/Chrome based browsers (Opera, Brave, etc.) if you serve the page from the file system instead of the network. In that case there is no origin site (because it was served from file), and Chrome will send the header Origin: null in the preflight OPTIONS request. Your CORS enabled Flask server will reply with this origin in its preflight response, however, Chrome rejects it as an invalid origin, even though that is what Chrome sent in the first place.
Firefox also sends header Origin: null in the OPTIONS request, but it is OK with the response and so it does issue the actual POST request. I have also found that Chrome on Android does seem to work when loading from the file system.
As a work-around, serve your test file from a HTTP server. The easiest way (for testing purposes only) is to use Python's SimpleHTTPSever. Simply change to the directory containing your test HTML file and run this command:
$ python -m SimpleHTTPServer
It will start a HTTP server listening on port 8000 so you can load the file into your browser using URL http://localhost:8000/cors_test.html for example.
Also note that the Access-Control-Allow-* headers are meant to be sent in the preflight response, not the request. flask-cors will generally handle that for you and the client side XMLHttpRequest object should properly construct the request for you.
I have created and API using python+flask. When is try to hit the api using postman or chrome it works fine and I am able to get to the api.
On the other hand when I try to use python
import requests
requests.get("http://localhost:5050/")
I get 407. I guess that the proxy of the our environment is not allowing me to hit the localhost. But due to LAN settings in IE/Chrome the request went through.
I did try to set proxies , auth in request and now I start getting 502(bad gateway). If I see on the API side I can't see a request come through. What can I do to troubleshoot the same.
According to requests module documentation you can either provide proxy details through environment variable HTTP_PROXY (in case use Linux distribution):
$ export HTTP_PROXY="http://corporate-proxy:port"
$ python
>>> import requests
>>> requests.get('http://localhost:5050/')
Or provide proxies keyword argument to get method directly:
import requests
proxies = {
'http': 'http://coporate-proxy:port',
}
requests.get('http://localhost:5050/', proxies=proxies)
Try
import requests
from flask_cors import CORS, cross_origin
app = Flask(__name__)
cors = CORS(app, resources={r"/*": {"origins": "*"}})
requests.get("http://localhost:5050/")
I need a simple Client side that can send a bool valu if he is Alive, in a HTTP POST request. I have a Linux server running Apache.
I realized from Atermi(user who helped me), i new to use a requests library, something like this:
import requests
r = requests.post("http://127.0.0.1/post", data={'key': 'val'})
print(r.text)
But I don't know how to make my apache server receive the post requests.
In addition,when running the script I am receiving an error,
The requested URL '/' was not found on this servers.
I tried a server side, for testing in the same linux server and this is the
Code,
from flask import Flask, request
app = Flask(__name__)
#app.route('/', methods=['POST'])
def res():
print(request.form['key'])
return "Revived"
but recveing the following error:
runtimerror: working outside of the request context
Thanks!
Environment - Python 2.7.3, webpy.
I'm trying a simple oauth 3 way authentication for github using Python web.py. Per the basic oauth guide on github I'm doing something like this:
import web,requests
import oauth2,pymongo,json
from oauth2client.client import OAuth2WebServerFlow
urls=('/', 'githublogin',
'/session','session',
'/githubcallback','githubCallback');
class githublogin:
def GET(self):
new_url = 'https://github.com/login/oauth/authorize'
pay_load = {'client_id': '',
'client_secret':'',
'scope':'gist'
}
headers = {'content-type': 'application/json'}
r = requests.get(new_url, params=pay_load, headers=headers)
return r.content
This is sending me to the GH login page. Once I sign in - GH is not redirecting me to the callback. The redirect_uri parameter is configured in the github application. I've double checked to make sure that's correct.
class githubCallback:
def POST(self):
data = web.data()
print data
def GET(self):
print "callback called"
Instead in the browser I see
http://<hostname>:8080/session
and a 404 message, because I haven't configured the session URL. That's problem no 1. Problem no 2 - If I configure the session URL and print out the post message
class session:
def POST(self):
data = web.data()
print data
def GET(self):
print "callback called"
I can see some data posted to the URL with something called 'authenticity_token'.
I've tried to use the python_oauth2 library but can't get past the authorization_url call. So I've tried this much simpler requests library. Can someone please point out to me whats going wrong here.
So here's how I solved this. Thanks to #Ivanzuzak for the requestb.in tip.
I'm using Python webpy.
import web,requests
import oauth2,json
urls=('/', 'githublogin',
'/githubcallback','githubCallback');
render = web.template.render('templates/')
class githublogin:
def GET(self):
client_id = ''
url_string = "https://github.com/login/oauth/authorize?client_id=" + client_id
return render.index(url_string)
class githubCallback:
def GET(self):
data = json.loads(json.dumps(web.input()))
print data['code']
headers = {'content-type': 'application/json'}
pay_load = {'client_id': '',
'client_secret':'',
'code' : data['code'] }
r = requests.post('https://github.com/login/oauth/access_token', data=json.dumps(pay_load), headers=headers)
token_temp = r.text.split('&')
token = token_temp[0].split('=')
access_token = token[1]
repo_url = 'https://api.github.com/user?access_token=' + access_token
response = requests.get(repo_url)
final_data = response.content
print final_data
app = web.application(urls,globals())
if __name__ == "__main__":
app.run()
I was not using a html file before, but sending the request directly from the githublogin class. That didn't work. Here I'm using a html to direct the user first from where he'll login to gh. With this I added a html and rendered it using the templator.
def with (parameter)
<html>
<head>
</head>
<body>
<p>Well, hello there!</p>
<p>We're going to now talk to the GitHub API. Ready? <a href=$parameter>Click here</a> to begin!</a></p>
<p>If that link doesn't work, remember to provide your own Client ID!</p>
</body>
</html>
This file is taken straight from the dev guide, with just the client_id parameter changed.
Another point to be noted is that in the requests.post method - passing the pay_load directly doesn't work. It has to be serialized using json.dumps.
I'm not sure what the problem is at your end, but try reproducing this flow below, first manually using the browser, and then using your python library. It will help you debug the issue.
create a request bin on http://requestb.in/. A request bin is basically a service that logs all HTTP requests sent to it. You will use this instead of the callback, to log what is being sent to the callback. Copy the URL of the request bin, which is something like http://requestb.in/123a546b
Go to your OAuth application setup on GitHub (https://github.com/settings/applications), enter the setup of your specific application, and set the Callback URL to the URL of the request bin you just created.
Make a request to the GitHub OAuth page, with the client_id defined. Just enter this URL below into your browser, but change the YOUR_CLIENT_ID_HERE to be the client id of your OAuth application:
https://github.com/login/oauth/authorize?client_id=YOUR_CLIENT_ID_HERE
Enter your username and password and click Authorize. The GitHub app will then redirect you to the request bin service you created, and the URL in the browser should be something like (notice the code query parameter):
http://requestb.in/YOUR_REQUEST_BIN_ID?code=GITHUB_CODE
(for example, http://requestb.in/abc1def2?code=123a456b789cdef)
Also, the content of the page in the browser should be "ok" (this is the content returned by the request bin service).
Go to the request bin page that you created and refresh it. You will now see a log entry for the HTTP GET request that the GitHub OAuth server sent you, together with all the HTTP headers. Basically, you will see there the same code parameter that is present in the URL that you were redirected to. If you get this parameter, you are now ready to make a POST request with this code and your client secret, as described in step 2 of the guide you are using: http://developer.github.com/v3/oauth/#web-application-flow
Let me know if any of these steps are causing problems for you.