I have a java Google App Engine restful server implemented via Jersey. I am able to make successful GET, POST, PUT, and DELETEs using Postman, a browser based curl like program, when signed into my Google account in the browser session.
Now I am trying to make similar calls in a Python script using the Requests library.
payload = "{\"key\":1, \"dateOpened\":2341342, \"dateClosed\":0, \"description\":\"test\", \"urgency\":2, \"staus\":1}"
r = requests.post("http://localhost:8080/ticket", params=payload)
print r.status_code
When I do this I get a redirect which I am unsure how to handle.
com.google.apphosting.utils.jetty.AppEngineAuthentication$AppEngineAuthenticator authenticate
Got /ticket but no one was logged in, redirecting.
How am I able to make it log in as well using the Requests library? I would like to continue using the Requests library if possible due to its simplicity, but I am a new to using it.
Thanks.
Related
I have a Python script that should download data from a web resource using link. It so happened that resource is in Azure App Service protected by Active Directory. My user account is allowed to access the link and download data (I can do it from web browser manually, but want to automate this process). The Python script uses requests library. I can't figure out how to authenticate properly, cause when I'm trying to run the script, I get:
Error 403 - Forbidden
The web app you have attempted to reach has blocked your access.
Usual authentication with requests doesn't work (using auth parameter or session.auth or with HttpNtlmAuth).
I know one can use VS Code to authenticate to Azure and then use DefaultAzureCredential, but I can't get where you should use this DefaultAzureCredential object (cause it doesn't work with requests).
I don't need the whole Python app to be registered or somehow else recognizable by Azure resource. It's just a script to download data, that is not supposed to be productionized.
Any ideas how I can scrap the data from Azure?
Note: I'm not an admin or creator of this Azure App, so can't change any restriction settings.
In short, the part of script making request looks like:
params = {"param1": param1,
"param2": param2}
session = requests.Session()
session.auth = HttpNtlmAuth(USERNAME, PASSWORD)
url = "my-app.azurewebsites.net/the-rest-of-the-path"
response = session.get(url, params=params, verify=False)
If you want to access the Azure App Service, you have to authenticate the Azure App Service. If you don't have access for Azure App Service, we cannot access the Azure resources.
Genereally, when a web server stops you from accessing the page you're trying to open in your browser, you'll get a 403 Forbidden Error. There isn't much you can do most of the time. However, occasionally the issue is on your end.
Here are some points that can cause this error.
If you have an open public API and public access is not allowed on Azure App Service.
Your app's IP address, which you're using to call the app service, isn't whitelisted.
If you have a gateway in the middle, it's possible that it's also blocking your calls.
Here are the possible solutions that you can try:
Remove the access restrictions from your web app's Networking page.
Try adding 0.0.0.0/0 to give access to all. You can later add restrictions based on your needs.
The order of the restrictions is important, so double-check it. It may have an impact if you have a blocked call before an approved call.
You can also have restrictions based on http-headers like X-Forwarded-For. Please double-check that. This can also happen in code, depending on how you handle errors.
Protocol support for HTTP headers in Azure Front Door | Microsoft Docs
Chech this, if your API is behind the Gateway Application Gateway integration with service endpoints
I am trying to work on a project that collects all my monthly utility amounts and disperses the amounts owed to my roommates. I have managed to programmatically log into two websites but I am having trouble with the last, as they are using SAML (https://www.blackhillsenergy.com/). I have inspected the web requests with Chrome's Developer Tools but I am not getting any breakthroughs. I attempted to use requests_ecp but I am not having any luck with that either. I get the idea of SAML but having a hard time understanding their implementation and how I can use it in my script. Below is my sample code? Any ideas?
def get_bh_bill():
url = 'https://www.blackhillsenergy.com/cpm/v1/user/accounts?username={fill here}'
bh_login = ''
bh_pass = ‘'
# Start a session so we can have persistent cookies
session = requests.session()
session.auth = HTTPECPAuth('https://sso.blackhillsenergy.com', username=bh_login, password=bh_pass)
acc_res = session.get(url)
acc_soup = BeautifulSoup(acc_res.text, "html.parser")
print(acc_soup.prettify())
return '0000'
SAML, typically, works like this.
You hit the desired site, they see you are not authenticated, so they create a SAML request, route it through your browser, and send you to an IdP, Identity Provider.
The IdP reads the SAML request, and then asks you for credentials. Once authenticated, it creates a SAML response, and routes that back to the original site, through your browser.
The routing is done by presenting a simple HTML form containing the SAML Request/Response, and a teeny bit of javascript to submit it. This is how it moves information across domains (SAML is typically done across domains, this is why it doesn't use cookies.)
What your script needs to do is basically follow the workflow, submit the forms automatically, login when asked, and submit the forms back. It's a multi step workflow. There may well be a bunch of redirects involved as well.
I am fairly proficient in Python and have started exploring the requests library to formulate simple HTTP requests. I have also taken a look at Sessions objects that allow me to login to a website and -using the session key- continue to interact with the website through my account.
Here comes my problem: I am trying to build a simple API in Python to perform certain actions that I would be able to do via the website. However, I do not know how certain HTTP requests need to look like in order to implement them via the requests library.
In general, when I know how to perform a task via the website, how can I identify:
the type of HTTP request (GET or POST will suffice in my case)
the URL, i.e where the resource is located on the server
the body parameters that I need to specify for the request to be successful
This has nothing to do with python, but you can use a network proxy to examine your requests.
Download a network proxy like Burpsuite
Setup your browser to route all traffic through Burpsuite (default is localhost:8080)
Deactivate packet interception (in the Proxy tab)
Browse to your target website normally
Examine the request history in Burpsuite. You will find every information you need
All,
I am trying to build a library for onedrive.
Not to give too much detail but I have constructed the request and if I call this from a browser everything works fine, and i eventually get a json response.
https://login.live.com/oauth20_authorize.srf?client_id=CLIENT_ID&scope=wl.signin%20wl.basic&response_type=code&redirect_uri=http://someaddress.com/redirect.html
My problem is that I cannot get the authorization code in python using urllib2 and geturl() method as the redirect that yields the code is not seen by urllib2.
When I call the geturl() method I get the original url, not the redirect url containing the code.
Regards,
Frage
With the release of the OneDrive API there is pretty solid documentation on how to implement the OAuth 2.0 standard from scratch. Following what that doc lays out should make the authentication flow in your application pretty straight-forward.
I am trying to find the easiest way how to use Facebook Graph API using my favorite Requests library. The problem is, all examples I found are about getting user access token, about redirects and user interaction.
All I need is only application access token. I do not handle any non-public data, so I need no user interaction and as my final app is supposed to be command-line script, no redirects are desired.
I found something similar here, but it seems to be everything but elegant. Moreover, I would prefer something using Requests or Requests-OAuth2. Or maybe there is library for that? I found Requests-Facebook and Facepy (both Requests based), but again, all examples are with redirection, etc. Facepy does not handle authorization at all, it just accepts your token and it is up to you to get it somehow.
Could someone, please, provide a short, sane, working example how to get just the application access token?
Following https://developers.facebook.com/docs/technical-guides/opengraph/publishing-with-app-token/:
import requests
r = requests.get('https://graph.facebook.com/oauth/access_token?grant_type=client_credentials&client_id=123&client_secret=XXX')
access_token = r.text.split('=')[1]
print access_token
(using the correct values for client_id and client_secret) gives me something that looks like an access token.
If you just need a quick/small request, you can manually cut and paste the access token from here into you code: https://developers.facebook.com/tools/explorer
Note: Unlike Richard Barnett's answer, you'll need to regenerate the code manually from the graph api explorer every time you use it.