I'm trying to enable authentication in Apache SuperSet through Oauth2.
It shoud be straightforward due to the fact that it's built upon Flask AppBuilder which supports OAuth and is extremely easy to setup and use.
I managed to make both the following examples work seamlessy with Twitter Oauth configuration:
FAB OAuth example
flask-oauthlib examples
Now I'm trying to apply the same configuration to SuperSet.
Docker
As I can't manually build the project for several mysterious python errors (tried on Windows 7/Ubuntu Linux and with Python versions 2.7 and 3.6), I decided to use this Superset docker image (that installs and works fine) and inject my configuration as suggested by docs:
Follow the instructions provided by Apache Superset for writing your own superset_config.py. Place this file in a local directory and mount this directory to /home/superset/.superset inside the container.
I added a superset_config.py (in a folder and alone) and mounted it by adding to the Dockerfile the following:
ADD config .superset/config
(config is the name of the folder) or (for the single file):
COPY superset_config.py .superset
In both cases the files end up in the right place in the container (I check with docker exec /bin/bash) but the web application shows no difference: no traces of Twitter authentication.
Can somebody figure out what I am doing wrong?
You have to change the superset_config.py. Look at this example config, it works for me.
import os
from flask_appbuilder.security.manager import AUTH_OID,
AUTH_REMOTE_USER,
AUTH_DB, AUTH_LDAP, AUTH_OAUTH
basedir = os.path.abspath(os.path.dirname(__file__))
ROW_LIMIT = 5000
SUPERSET_WORKERS = 4
SECRET_KEY = 'a long and random secret key'
SQLALCHEMY_DATABASE_URI = ‘postgresql://username:pass#host:port/dbname’
CSRF_ENABLED = True
AUTH_TYPE = AUTH_OAUTH
AUTH_USER_REGISTRATION = True
AUTH_USER_REGISTRATION_ROLE = "Public"
OAUTH_PROVIDERS = [
{
'name': 'google',
'whitelist': ['#company.com'],
'icon': 'fa-google',
'token_key': 'access_token',
'remote_app': {
'base_url': 'https://www.googleapis.com/oauth2/v2/',
'request_token_params': {
'scope': 'email profile'
},
'request_token_url': None,
'access_token_url':
'https://accounts.google.com/o/oauth2/token',
'authorize_url': 'https://accounts.google.com/o/oauth2/auth',
'consumer_key': 'GOOGLE_AUTH_KEY',
'consumer_secret': 'GOOGLE_AUTH_SECRET'
}
}
]
2021 update: The FAB OAuth provider schema seems like it changed a bit since this answer. If you're trying to do this with Superset >= 1.1.0, try this instead:
OAUTH_PROVIDERS = [
{
'name': 'google',
'icon': 'fa-google',
'token_key': 'access_token',
'remote_app': {
'client_id': 'GOOGLE_KEY',
'client_secret': 'GOOGLE_SECRET',
'api_base_url': 'https://www.googleapis.com/oauth2/v2/',
'client_kwargs':{
'scope': 'email profile'
},
'request_token_url': None,
'access_token_url': 'https://accounts.google.com/o/oauth2/token',
'authorize_url': 'https://accounts.google.com/o/oauth2/auth'
}
}
]
Of course, sub out GOOGLE_KEY and GOOGLE_SECRET. The rest should be fine. This was cribbed from the FAB security docs for the next time there is drift.
Related
I want to Test my azure function using the Azure Apps feature to Run/Test mode but it is throwing the '500 internal server error'.
I am able to debug the same code in my local environment but when to trigger the same code on the azure portal then it is getting failed without any proper error logs.
This Azure function will read the json format data from the event hub and write the same to the blob storage. I am using python for the azure function development.
Here is the code:
init.py
from typing import List
import logging
import os
import azure.functions as func
from azure.storage.blob import BlobClient
import datetime
import json
storage_connection_string = os.getenv('storage_connection_string_FromKeyVault')
container_name = os.getenv('storage_container_name_FromKeyVault')
today = datetime.datetime.today()
def main(events: List[func.EventHubEvent]):
for event in events:
a = event.get_body().decode('utf-8')
json.loads(a)
logging.info('Python EventHub trigger processed an event: %s', a)
logging.info(f' SequenceNumber = {event.sequence_number}')
logging.info(f' Offset = {event.offset}')
blob_client = BlobClient.from_connection_string(storage_connection_string, container_name, str(today.year) +"/" + str(today.month) + "/" + str(today.day) + "/" + str(event.sequence_number) + ".json")
blob_client.upload_blob(event.get_body().decode(),blob_type="AppendBlob")
local.settings.json
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "<Endpoint1>",
"FUNCTIONS_WORKER_RUNTIME": "python",
"storage_connection_string_FromKeyVault": "<connectionString",
"storage_container_name_FromKeyVault": "<container_name>",
"EventHubReceiverPolicy_FromKeyVault": "<Endpoint2>"
}
}
function.json
{
"scriptFile": "__init__.py",
"bindings": [
{
"type": "eventHubTrigger",
"name": "events",
"direction": "in",
"eventHubName": "pwo-events",
"connection": "EventHubReceiverPolicy_FromKeyVault",
"cardinality": "many",
"consumerGroup": "$Default",
"dataType": "binary"
}
]
}
Please note that this error is throwing when I am clicking on Run/Test on the portal. but the same code is also running fine after deployment.
The 500 error is not helpful to solve this problem, you need to check the specific error of the azure function. You can use application insights to get the details error. The function must configure the corresponding application insights before you can view the log on the portal.
So you need to configure an application insights to your function app like this:
Then your function app will restart.
Of course, you can also go to kudu to view:
First, go to advanced tools, then click 'GO',
Then After you go to kudu, click Debug Console -> CMD -> LogFiles -> Application -> Functions -> yourtriggername. You will find log file there.
If you are based on linux OS, after go to kudu, just click 'log stream'(this is not supportted to consumption plan for linux.).
I had this problem and I found that problem was with dependencies. Removing unexisting libraries (or using Microsoft's bring dependency document) will solve the issue.
Adding third-party dependencies in the Azure portal is currently not supported for Linux Consumption Function Apps. Click here to setup local environment. Learn more
If you need dependencies, to solve this problem, you can refer to this Microsoft Document
I am trying to inject Environment Variables into an Azure "App Services" Flask resource.
Note: I am aware that some use files to set up environment variables. I may look into that in the future, but for now I'm trying to do this without managing files.
Per the manual, I added the environment variables as "app settings," on the portal page.
And I can see that they have been set correctly with the Azure CLI command:
az webapp config appsettings list --name <redacted> --resource-group <redacted>
which outputs:
{
"name": "DB.DATABASE",
"slotSetting": false,
"value": "<redacted>"
},
{
"name": "DB.DRIVER",
"slotSetting": false,
"value": "{SQL Server Native Client 11.0}"
},
...
My Python code references the variables, which works locally.
from os import environ
driver = environ.get('DB.DRIVER')
server = environ.get('DB.SERVER')
user_id = environ.get('DB.USER_ID')
password = environ.get('DB.PASSWORD')
database = environ.get('DB.DATABASE')
trusted_connection = environ.get('DB.TRUSTED_CONNECTION')
print(f'driver: {driver}')
print(f'server: {server}')
print(f'user_id: {user_id}')
and the output, in the Azure log stream is:
2020-10-05T17:08:01.172742838Z driver: None
2020-10-05T17:08:01.172767338Z server: None
2020-10-05T17:08:01.172772538Z user_id: None
What, please, am I missing from this procedure? It seemed so simple, but it just fails with no error message.
I want to use the securityPolicies API for the Google Cloud Platform in Linux in a script written in Python.
To do this:
I installed google-api-python-client:
pip install google-api-python-client
I generated and downloaded from GCP this credential information (private key) in JSON format and exported the path to this file under the environmental variable:
export GOOGLE_APPLICATION_CREDENTIALS='[PATH]'
Now I have a doubt about how to properly use the google-api-python-client library to achieve my goal.
Using the addRule method according to the documentation I write the following script (of course with valid values for project keys and securityPolicy), but when I execute it, although the interpreter doesn't return any error, but the script doesn't give the expected effect:
from googleapiclient import discovery
compute_service = discovery.build('compute', 'v1')
security_policies = compute_service.securityPolicies()
security_policies.addRule(
project='existed_project_name',
securityPolicy='existed_security_policy_name',
body={
'kind': 'compute#securityPolicyRule',
'priority': 303,
'action': 'deny(403)',
'preview': False,
'match': {
'config': {
'srcIpRanges': [
'192.0.2.0/24',
'198.51.100.0/24',
'203.0.113.0/24'
]
},
'versionedExpr': 'SRC_IPS_V1'
}
}
)
So I have the following questions:
1. What should I improve or change?
2. Is my approach to authentication correct?
Any ideas?
You're missing the .execute() at the end of your addRule method
So your code should look like this:
from googleapiclient import discovery
compute_service = discovery.build('compute', 'v1')
security_policies = compute_service.securityPolicies()
security_policies.addRule(
project='existed_project_name',
securityPolicy='existed_security_policy_name',
body={
'kind': 'compute#securityPolicyRule',
'priority': 303,
'action': 'deny(403)',
'preview': False,
'match': {
'config': {
'srcIpRanges': [
'192.0.2.0/24',
'198.51.100.0/24',
'203.0.113.0/24'
]
},
'versionedExpr': 'SRC_IPS_V1'
}
}
).execute()
Assuming I have gotten the ecr credentials from boto already in an object called creds, when I do:
client = from_env()
client.login(creds.username, password=creds.password, registry=creds.endpoint)
I get:
{u'IdentityToken': u'', u'Status': u'Login Succeeded'}
Great so far! And I inspect:
client.api.__dict__
I get:
{'_auth_configs': {'auths': {'registry_i_just_logged_into': {'email': None,
'password': 'xxxxxxxxxxxxx',
'serveraddress': 'registry_i_just_logged_into',
'username': 'xxxxxxx'},
u'some_other_registry': {},
'credsStore': u'osxkeychain'}
.... (etc, etc)
Still so far, so good. But when I then do:
client.images.pull("registry_i_just_logged_into/some_repo", tag="latest")
Or when I do (from a command line):
docker pull registry_i_just_logged_into/some_repo:latest
I always get:
Error response from daemon: pull access denied for some_repo, repository does not exist or may require 'docker login'
Despite the fact that, if I do (with the same username and password I used to log in):
client.images.pull("registry_i_just_logged_into/some_repo", tag="latest", auth_config={'username': creds.username, 'password': creds.password})
It works no problems.
So I am assuming this is a problem with the order for resolving which registry to use, but it seems like the docker sdk should handle this if the key already exists within _auth_configs.
What am I doing wrong?
Thanks!
Short:
rm -rf ~/.docker/config.json
Long:
Remove credsStore, auths and credSstore properties from ~/.docker/config.json
Explanation:
Probably, you have already tried to login. So your Docker config.json has credsStore, auths and credSstore properties.
E.g.:
"credSstore" : "osxkeychain",
"auths" : {
"acc_id_1.dkr.ecr.us-east-1.amazonaws.com" : {
},
"acc_id_2.dkr.ecr.us-east-1.amazonaws.com" : {
},
"https://acc_id_1.dkr.ecr.us-east-1.amazonaws.com" : {
},
"https://acc_id_2.dkr.ecr.us-east-1.amazonaws.com" : {
}
},
"HttpHeaders" : {
"User-Agent" : "Docker-Client/18.06.1-ce (darwin)"
},
"credsStore" : "osxkeychain"
}
token = client.get_authorization_token() returns base64 encoded token. So to successfully login you need to decode it.
import docker
import boto3
import base64
docker_client = docker.from_env()
client = boto3.client('ecr', aws_access_key_id="xyz", aws_secret_access_key="abc", region_name="ap-south-1")
token = client.get_authorization_token()
docker_client.login(username="AWS", password=base64.b64decode(token["authorizationData"][0]["authorizationToken"]).decode().split(":")[1], registry="xxxx.dkr.ecr.ap-south-1.amazonaws.com")
will return
{'IdentityToken': '', 'Status': 'Login Succeeded'}
I'm trying to upload my Python script to authorize the user for the Spotify iOS SDK. Honestly, I dont know what I'm doing but the documentation is really poor. I'm using Heroku as web server but when I use foreman start I only get this on localhost:5000:
Not Found
The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
spotify_token_swap.py looks as following:
import cherrypy
from cherrypy import tools
import simplejson as json
import requests
import os
from flask import Flask
app = Flask(__name__)
# CHANGE these values to your own
k_client_id = "spotify-ios-sdk-beta"
k_client_secret = "ba95c775e4b39b8d60b27bcfced57ba473c10046"
k_client_callback_url = "spotify-ios-sdk-beta://callback"
verbose = True
class SpotifyTokenSwap(object):
#cherrypy.expose
#tools.json_out()
def swap(self, code=None):
params = {
'grant_type': 'authorization_code',
'client_id': k_client_id,
'client_secret': k_client_secret,
'redirect_uri': k_client_callback_url,
'code' : code
}
r = requests.post('https://ws.spotify.com/oauth/token', params)
cherrypy.response.status = r.status_code
if verbose:
print
print code
print r.status_code
print r.text
print
return r.json()
def CORS():
cherrypy.response.headers["Access-Control-Allow-Origin"] = "*"
if __name__ == '__main__':
cherrypy.tools.CORS = cherrypy.Tool('before_handler', CORS)
root = SpotifyTokenSwap()
config = {
'global' : {
'server.socket_host' : '0.0.0.0',
'server.socket_port' : 5000,
'server.thread_pool' : 10,
# 'environment' : 'production',
},
'/' : {
'tools.CORS.on' : True,
}
}
cherrypy.quickstart(root, '/', config=config)
and I start the foreman webserver using this in my Procfile:
web: gunicorn spotify_token_swap:app
I'm pretty sure you are pointing to the wrong wsgi application. Pointing to app from the Procfile meant that flask was serving the page. You registered and built everything with cherrypy, and did not include any routes in flask. So the app object had no routes, ie no '/'. So you need to switch to serving the cherrypy app.
Since you're removing the flask app part, you should remove the if __name__ == '__main__': line and change the rest to
config = {
'global' : {
'server.socket_host' : '0.0.0.0',
'server.socket_port' : 5000,
'server.thread_pool' : 10,
# 'environment' : 'production',
},
'/' : {
'tools.CORS.on' : True,
}
}
wsgiapp = cherrypy.Application(SpotifyTokenSwap(), '/', config=config)
And then use this in the ProcFile
web: gunicorn spotify_token_swap:wsgiapp
I'm not used to Foreman or cherrypy, but I think this is what you need to do.
You can use this python service instead:
Download Google App engine here
Install the launcher
Go to ChirsmLarssons GitHub and download the project, it will have everything you need.
In Google app engine launcher, Press add excisting project
Go Google app engines website and create a project, here you will get an app-id
in app.yaml , replace spotifyauth with the app-id
Press deploy
Done, you can now access it on the web at app-id.appspot.com/swap
Before I got this solution, I've spend hours in the jungle of Python and Ruby, Cheers!