I am making python wrapper for some web api service.
I have faced a problem, which I do not how to solve and quite can not understand.
I need to save authorized session object. But I get error:
import pickle
from oauthlib.oauth2 import BackendApplicationClient
from requests_oauthlib import OAuth2Session
client = BackendApplicationClient(client_id="some_id")
oauth_session = OAuth2Session(client=client)
with open("./oauth_session.pkl", "wb") as file:
pickle.dump(oauth_session, file)
# AttributeError: Can't pickle local object
# 'OAuth2Session.__init__.<locals>.<lambda>'
Oauth session according to documentation inherits from requests.Session, which is pickled fine
import pickle
from requests import Session
session = Session()
with open("./session.pkl", "wb") as file:
pickle.dump(session, file)
# Process finished with exit code 0
I have two questions:
Why OAuth session can not be pickled? What's wrong with lambdas?
What is way around it?
My way around was to store only token from session, but it have showed itself unreliable and not working properly with Authorization Code Flow.
Also i tried to use dill, it helped to dump session object by when I load session object back, it seems to be losing session attributes:
like that AttributeError: 'OAuth2Session' object has no attribute '_client'
I found on SO similliar question but answer is for specific external library and not explains anything.
Related
I am using Django with requests_oauthlib to access the Xero API using the xero-python SDK.
Xero Starter Demo
requests-oauthlib
I am trying to use request.session to store the oauth token information. This works well on the initial call and callback, but in subsequent views when I want to load the correct token from the request.session I am hitting a problem.
The Xero ApiClient (xero_python.api_client) wants me to configure two functions it can call to get and save the token in the future (oauth2_token_getter, oauth2_token_saver) (decorators are used in the demo). Those functions are called by the Xero API and take no parameters. I can create the functions no problem, and they get called correctly, but once inside the functions I have no request object so I can't access request.session to get or save the token to the session.
The Xero demonstration projects use Flask which imports a root level session object (from flask import session) which can be accessed in any function, so this works fine for Flask. What is the correct way to solve this problem in Django?
I have tried creating a SessionStore object but it was empty despite me knowing the data is in the request.session
I have also tried importing requests.session but this doesn't seem to be what I am looking for as I can't get it to read/write any data.
Some Code:
Saving the token in the callback works as I have a request.session:
def xero_callback(request):
...
request.session["oauth_token"] = token
...
The Xero ApiClient is configured with the methods for getting and saving:
api_client = ApiClient(
Configuration(
debug=True,
oauth2_token=OAuth2Token(client_id=config.XERO_CLIENT_ID, client_secret=config.XERO_CLIENT_SECRET),
),
pool_threads=1,
oauth2_token_getter=oauth2_token_getter,
oauth2_token_saver=oauth2_token_saver)
This doesn't work as I have no request object and can't pass one in:
def oauth2_token_getter():
return request.session.get("token") #! FAIL, I don't have access to request here
The error is thrown after calling identity_api.get_connections() as it tries to get the token using oauth2_token_getter and fails
def xero_tenants(request):
identity_api = IdentityApi(api_client)
accounting_api = AccountingApi(api_client)
available_tenants = []
for connection in identity_api.get_connections():
...
Any help appreciated, Thanks
Are you able to store the token in the django cache? (django.core.cache import cache) Then instead of specifying oauth2_token_getter=oauth2_token_getter in the ApiClient constructor you can do:
#api_client.oauth2_token_getter
def obtain_xero_oauth2_token():
return cache.get('token')
and something similar for the setter/saver.
I'm trying to set the environment variable from a dict but getting and error when connecting.
#service account pulls in airflow variable that contains the json dict with service_account credentials
service_account = Variable.get('google_cloud_credentials')
os.environ["GOOGLE_APPLICATION_CREDENTIALS"]=str(service_account)
error
PermissionDeniedError: Error executing an HTTP request: HTTP response code 403 with body '<?xml version='1.0' encoding='UTF-8'?><Error><Code>AccessDenied</Code><Message>Access denied.</Message><Details>Anonymous caller does not have storage.objects.get access to the Google Cloud Storage object.</Details></Error>'
when reading if I use and point to file then there are no issues.
os.environ["GOOGLE_APPLICATION_CREDENTIALS"]=/file/path/service_account.json
I'm wondering is there a way to convert the dict object to an os path like object? I don't want to store the json file on the container and airflow/google documentation isn't clear at all.
The Python stringio package lets you create a file-like object backed by a string, but that won't help here because the consumer of this environment variable is expecting a file path, not a file-like object. I don't think it's possible to do what you're trying to do. Is there a reason you don't want to just put the credentials in a file?
There is a way to do it, but the Google documentation is terrible. So I wrote a Github gist to document the recipe that I and a colleague (Imee Cuison) developed to use the key securely. Sample code below:
import json
from google.oauth2.service_account import Credentials
from google.cloud import secretmanager
def access_secret(project_id:str, secret_id:str, version_id:str="latest")->str:
"""Return the secret in string format"""
# Create the Secret Manager client.
client = secretmanager.SecretManagerServiceClient()
# Build the resource name of the secret version.
name = f"projects/{project_id}/secrets/{secret_id}/versions/{version_id}"
# Access the secret version.
response = client.access_secret_version(name=name)
# Return the decoded payload.
return response.payload.data.decode('UTF-8')
def get_credentials_from_token(token:str)->Credentials:
"""Given an authentication token, return a Credentials object"""
credential_dict = json.loads(secret_payload)
return Credentials.from_service_account_info(credential_dict)
credentials_secret = access_secret("my_project", "my_secret")
creds = get_credentials_from_token(credentials_secret)
# And now you can use the `creds` Credentials object to authenticate to an API
Putting the service account into the repository is not a good practice. As a best practice; You need to use authentication propagating from the default google auth within your application.
For instance, using Google Cloud Kubernetes you can use the following python code :
from google.cloud.container_v1 import ClusterManagerClient
credentials, project = google.auth.default(
scopes=['https://www.googleapis.com/auth/cloud-platform', ])
credentials.refresh(google.auth.transport.requests.Request())
cluster_manager = ClusterManagerClient(credentials=credentials)
I am trying to use the Spotify API (using Spotipy), however I am having issues with the authentication step. I am following along a youtube playlist to learn it. To do that, I am just following along with the code that is shown below. However, when it opens into my web browser for authentication, I get an "Illegal redirect_uri" error.
I tried searching the web and came across this answer that says that it is probably a typo in the redirect_URI on the spotify website or that has been set in my environment variable, however, I have quadruple checked to make sure there was no typo. An image is attached that shows what my environment variable is and what the redirect_URI is set as in spotify.
Is there another reason that I could be getting this error?
Thank you for the help. Spotify Redirect_URI
import os
import sys
import json
import spotipy
import webbrowser
import spotipy.util as util
from json.decoder import JSONDecodeError
#Get the username from terminal
username = sys.argv[1]
# Erase cache and prompt for user permission
try:
token = util.prompt_for_user_token(username)
except:
os.remove(f".cache-{username}")
token = util.prompt_for_user_token(username)
#Create our spotifyObject
spotifyObject = spotipy.Spotify(auth=token)
It is probably a security issue, since google.com will not understand the parameters of the request sent by the Spotify API. It looks like you are not trying to intercept the request (since you are using google.com), so you could try to use https://localhost:8080/ as the redirect url. Since there is (probably) no server running locally, no page will open (and you will get an error), but you only need to copy the url in the address bar ;-)
I tried to create a zeep client and use it by two methods.
1) I tried to keep everything in a single module . i created the zeep client object and it was working fine while using a payload.
2) I created a method which returns a zeep client object for a wsdl. I tried to use this a way as method 1) But getting the below error.
zeep.exception.Fault : Incoming message could not be authenticated. No valid credentials found
Can someone please advise what I am missing here which causes this error. My second approach is like this.
\\
def zeepClient(wsdl):
## do all here and return zeep client object.
return client
#Now in another module I do call that above method like this
Client=othermodule.zeepClient(mywsdl)
Payload={my payload}
Client.service.myservice(**Payload)
\\
If I do this , I get above error.But if my above piece of code and my method for zeepClient are all in same place. I am not getting error.
Not sure . What that Returned Client object is missing.
You must provide credentials to the session and then make a request like here
from requests import Session
from requests.auth import HTTPBasicAuth # or HTTPDigestAuth, or OAuth1, etc.
from zeep import Client
from zeep.transports import Transport
session = Session()
session.auth = HTTPBasicAuth(user, password)
client = Client('http://my-endpoint.com/production.svc?wsdl',
transport=Transport(session=session))
I'm currently playing around with tipfy on Google's Appengine and just recently ran into a problem: I can't for the life of me find any documentation on how to use GET variables in my application, I've tried sifting through both tipfy and Werkzeug's documentations with no success. I know that I can use request.form.get('variable') to get POST variables and **kwargs in my handlers for URL variables, but that's as much as the documentation will tell me. Any ideas?
request.args.get('variable') should work for what I think you mean by "GET data".
Source: http://www.tipfy.org/wiki/guide/request/
The Request object contains all the information transmitted by the client of the application. You will retrieve from it GET and POST values, uploaded files, cookies and header information and more. All these things are so common that you will be very used to it.
To access the Request object, simply import the request variable from tipfy:
from tipfy import request
# GET
request.args.get('foo')
# POST
request.form.get('bar')
# FILES
image = request.files.get('image_upload')
if image:
# User uploaded a file. Process it.
# This is the filename as uploaded by the user.
filename = image.filename
# This is the file data to process and/or save.
filedata = image.read()
else:
# User didn't select any file. Show an error if it is required.
pass
this works for me (tipfy 0.6):
from tipfy import RequestHandler, Response
from tipfy.ext.session import SessionMiddleware, SessionMixin
from tipfy.ext.jinja2 import render_response
from tipfy import Tipfy
class I18nHandler(RequestHandler, SessionMixin):
middleware = [SessionMiddleware]
def get(self):
language = Tipfy.request.args.get('lang')
return render_response('hello_world.html', message=language)