I have created and API using python+flask. When is try to hit the api using postman or chrome it works fine and I am able to get to the api.
On the other hand when I try to use python
import requests
requests.get("http://localhost:5050/")
I get 407. I guess that the proxy of the our environment is not allowing me to hit the localhost. But due to LAN settings in IE/Chrome the request went through.
I did try to set proxies , auth in request and now I start getting 502(bad gateway). If I see on the API side I can't see a request come through. What can I do to troubleshoot the same.
According to requests module documentation you can either provide proxy details through environment variable HTTP_PROXY (in case use Linux distribution):
$ export HTTP_PROXY="http://corporate-proxy:port"
$ python
>>> import requests
>>> requests.get('http://localhost:5050/')
Or provide proxies keyword argument to get method directly:
import requests
proxies = {
'http': 'http://coporate-proxy:port',
}
requests.get('http://localhost:5050/', proxies=proxies)
Try
import requests
from flask_cors import CORS, cross_origin
app = Flask(__name__)
cors = CORS(app, resources={r"/*": {"origins": "*"}})
requests.get("http://localhost:5050/")
Related
This is my python code:
import requests
from flask import Flask
app = Flask(__name__)
#app.route("/")
def index():
url = 'https://google.co.in'
headers = {
'User-Agent':'blahblahblah'
}
r = requests.get(url, headers=headers)
return 'check terminal'
So this is the way to change request headers in python requests. But if I open the url and see developer options > Network > Request headers. I see default as user agent. Means it simply doesn't work.
The request you're making is by the server, not the client (the web browser). The index page served by flask goes to a client. The client doesn't make the requests.get request you've written here, instead the server does.
Instead, the client only requests whatever you're returning from the route, which here is 'check terminal' (which should not work, and should be something in the lines of return jsonify(result='check terminal')), and is not concerned about what the server is doing internally.
So as #brunns has commented, these two requests are different. If you want to check the headers of your request, maybe try httpbin.org.
Making GET requests from urls can be done with the python requests library like this:
import requests
response = requests.get("https://someapi.io/api/data")
response.json()
I'm currently trying to make GET requests from URLs with the flask request module but i can't find any useful information.
Flask is a server, the request object you can access on flask.request is the context of the request you received on the related API request.
Please see: https://flask.palletsprojects.com/en/1.1.x/
The requests library is not related and with this one, you can do all the GET requests you want.
You can do a request in one of your API requests handled by your flask server but you would have to use directly the request module and not the request object of the flask module.
I have an API Key for a Google API that I would like to use in all my requests to it. Some of these requests will originate from within a Google App Engine (Python 2.7) application. I had planned to use the UrlFetch library to complete the POST request, basically as follows:
headers = {'Content-Type': 'application/json'}
payload = {'longUrl': request.long_url}
result = urlfetch.fetch([API_REQUEST_URL],
method=urlfetch.POST,
payload=json.dumps(payload),
headers=headers)
json_result = json.loads(result.content)
I had set a referrer restriction on my API Key to *.[my-app].appspot.com/* with the hope that this would protect my API Key from unauthorized use and negate the need to update an IP-based key restriction (as App Engine IPs change all the time).
This approach as failed me though, because it seems that urlfetch does NOT specify a value for referrer on its own. I assume I could add my own referrer, but then so could anyone else. The approach isn't very secure.
What is the best practice? How should I restrict the key given that I'm using urlfetch from within App Engine? If I do use an HTTP Referrer restriction, which address do I use?
Many thanks.
You got like this error message?
Requests from referer <empty> are blocked.
urlfetch seems not to attach Refer automatically, so you should set Refer in your request header.
headers = {'Content-Type': 'application/json','Referer': '*.[my-app].appspot.com/*'}
As you observed the referrer header can be faked, so setting a referrer restriction on your API Key is rather useless to start with.
But you can add a check based on the X-Appengine-Inbound-Appid header, which is sanitized by the GAE infrastructure and precisely identifies the app. From Issuing a request to another App Engine app:
When issuing a request to another App Engine app, your App Engine app
must assert its identity by adding the header
X-Appengine-Inbound-Appid to the request. If you instruct the URL
Fetch service to not follow redirects, App Engine will add this header
to requests automatically.
To instruct the URL Fetch service to not follow redirects, set the
fetch follow_redirects parameter to False.
Note: If you are making requests to another App Engine application, use its appspot.com domain name rather than a custom
domain for your app.
How can I use automatic NTLM authentication from python on Windows?
I want to be able to access the TFS REST API from windows without hardcoding my password, the same as I do from the web browser (firefox's network.automatic-ntlm-auth.trusted-uris, for example).
I found this answer which works great for me because:
I'm only going to run it from Windows, so portability isn't a problem
The response is a simple json document, so no need to store an open session
It's using the WinHTTP.WinHTTPRequest.5.1 COM object to handle authentication natively:
import win32com.client
URL = 'http://bigcorp/tfs/page.aspx'
COM_OBJ = win32com.client.Dispatch('WinHTTP.WinHTTPRequest.5.1')
COM_OBJ.SetAutoLogonPolicy(0)
COM_OBJ.Open('GET', URL, False)
COM_OBJ.Send()
print(COM_OBJ.ResponseText)
You can do that with https://github.com/requests/requests-kerberos. Under the hood it's using https://github.com/mongodb-labs/winkerberos. The latter is marked as Beta, I'm not sure how stable it is. But I have requests-kerberos in use for a while without any issue.
Maybe a more stable solution would be https://github.com/brandond/requests-negotiate-sspi, which is using pywin32's SSPI implementation.
I found solution here https://github.com/mullender/python-ntlm/issues/21
pip install requests
pip install requests_negotiate_sspi
import requests
from requests_negotiate_sspi import HttpNegotiateAuth
GetUrl = "http://servername/api/controller/Methodname" # Here you need to set your get Web api url
response = requests.get(GetUrl, auth=HttpNegotiateAuth())
print("Get Request Outpot:")
print("--------------------")
print(response.content)
for request by https:
import requests
from requests_negotiate_sspi import HttpNegotiateAuth
import urllib3
urllib3.disable_warnings()
GetUrl = "https://servername/api/controller/Methodname" # Here you need to set your get Web api url
response = requests.get(GetUrl, auth=HttpNegotiateAuth(), verify=False)
print("Get Request Outpot:")
print("--------------------")
print(response.content)
NTLM credentials are based on data obtained during the interactive logon process, and include a one-way hash of the password. You have to provide the credential.
Python has requests_ntlm library that allows for HTTP NTLM authentication.
You can reference this article to access the TFS REST API :
Python Script to Access Team Foundation Server (TFS) Rest API
If you are using TFS 2017 or VSTS, you can try to use Personal Access Token in a Basic Auth HTTP Header along with your REST request.
I'm working with http://robobrowser.readthedocs.org/en/latest/readme.html, (a new python library based on the beautiful soup and request libraries) within django. My django app contains :
def index(request):
p=str(request.POST.get('p', False)) # p='https://www.yahoo.com/'
pr="http://10.10.1.10:3128/"
setProxy(pr)
browser = RoboBrowser(history=True)
postedmessage = browser.open(p)
return HttpResponse(postedmessage)
I would like to add a proxy to my code but can't find a reference in the docs on how to do this. Is it possible to do this?
EDIT:
following your recommendation I've changed the code to
pr="http://10.10.1.10:3128/"
setProxy(pr)
browser = RoboBrowser(history=True)
with:
def setProxy(pr):
import os
os.environ['HTTP_PROXY'] = pr
return
I'm now getting:
Django Version: 1.6.4
Exception Type: LocationParseError
Exception Value:
Failed to parse: Failed to parse: 10.10.1.10:3128
Any ideas on what to do next? I can't find a reference to this error
After some recent API cleanup in RoboBrowser, there are now two relatively straightforward ways to control proxies. First, you can configure proxies in your requests session, and then pass that session to your browser. This will apply your proxies to all requests made through the browser.
from requests import Session
from robobrowser import RoboBrowser
session = Session()
session.proxies = {'http': 'http://my.proxy.com/'}
browser = RoboBrowser(session=session)
Second, you can set proxies on a per-request basis. The open, follow_link, and submit_form methods of RoboBrowser now accept keyword arguments for requests.Session.send. For example:
browser.open('http://stackoverflow.com/', proxies={'http': 'http://your.proxy.com'})
Since RoboBrowser uses the request library, you can try to set the proxies as mentioned in the request docs by setting the environment variables HTTP_PROXY and HTTPS_PROXY.