I'm trying to interface Firebase with the official Admin SDK for Python (https://firebase.google.com/docs/database/admin/start).
However, I'm doing something wrong, as I'm not authorized somehow
This is my code:
import firebase_admin
from firebase_admin import credentials
from firebase_admin import db
# Fetch the service account key JSON file contents
cred = credentials.Certificate('./ServiceAccountKey.json')
# Initialize the app with a None auth variable, limiting the server's access
firebase_admin.initialize_app(cred, {
'databaseURL': 'https://*[database name]*.firebaseio.com',
'databaseAuthVariableOverride': None
})
# The app only has access to public data as defined in the Security Rules
ref = db.reference('/public_resource')
print(ref.get())
This is the error I get:
python3 firebase_test.py
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/firebase_admin/db.py", line 943, in request
return super(_Client, self).request(method, url, **kwargs)
File "/usr/local/lib/python3.6/site-packages/firebase_admin/_http_client.py", line 117, in request
resp.raise_for_status()
File "/usr/local/lib/python3.6/site-packages/requests/models.py", line 941, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://*[database name]*.firebaseio.com/public_resource.json?auth_variable_override=null
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "firebase_test.py", line 16, in <module>
print(ref.get())
File "/usr/local/lib/python3.6/site-packages/firebase_admin/db.py", line 222, in get
return self._client.body('get', self._add_suffix(), params=params)
File "/usr/local/lib/python3.6/site-packages/firebase_admin/_http_client.py", line 129, in body
resp = self.request(method, url, **kwargs)
File "/usr/local/lib/python3.6/site-packages/firebase_admin/db.py", line 945, in request
raise _Client.handle_rtdb_error(error)
firebase_admin.exceptions.UnauthenticatedError: Unauthorized request.
I'm using a Raspberry Pi 4 B with installed Python 3.6.
Can somebody point me into the right direction of why this happens and how to fix it?
Found it out!
Under database, go to rules and change it to:
{
/* Visit https://firebase.google.com/docs/database/security to learn more about security rules. */
"rules": {
".read": true,
".write": true
}
}
It was automatically set to false after I've waited for a too long period since my last access.
Related
I have four files main.py, jobs.zip, libs.zip & params.yaml and these I have stored on Azure Storage Account Container.
Now I have this code which is making a payload and will try to run a spark job using that payload. And that payload will be having the location link of these 4 files.
hook = AzureSynapseHook(
azure_synapse_conn_id=self.azure_synapse_conn_id, spark_pool=self.spark_pool
)
payload = SparkBatchJobOptions(
name=f"{self.job_name}_{self.app_id}",
file=f"abfss://{Variable.get('ARTIFACT_BUCKET')}#{Variable.get('ARTIFACT_ACCOUNT')}.dfs.core.windows.net/{self.env}/{SPARK_DIR}/main.py",
arguments=self.job_args,
python_files=[
f"abfss://{Variable.get('ARTIFACT_BUCKET')}#{Variable.get('ARTIFACT_ACCOUNT')}.dfs.core.windows.net/{self.env}/{SPARK_DIR}/jobs.zip",
f"abfss://{Variable.get('ARTIFACT_BUCKET')}#{Variable.get('ARTIFACT_ACCOUNT')}.dfs.core.windows.net/{self.env}/{SPARK_DIR}/libs.zip",
],
files=[
f"abfss://{Variable.get('ARTIFACT_BUCKET')}#{Variable.get('ARTIFACT_ACCOUNT')}.dfs.core.windows.net/{self.env}/{SPARK_DIR}/params.yaml"
],
)
self.log.info("Executing the Synapse spark job.")
response = hook.run_spark_job(payload=payload)
I have checked the location link that is correct but when I run this on airflow it throws an error related to the payload which I think it is trying to say that it is not able to grab the links.
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/azure/core/pipeline/transport/_base.py", line 579, in format_url
base = self._base_url.format(**kwargs).rstrip("/")
KeyError: 'endpoint'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/airflow/dags/operators/spark/__init__.py", line 36, in execute
return self.executor.execute()
File "/usr/local/airflow/dags/operators/spark/azure.py", line 60, in execute
response = hook.run_spark_job(payload=payload)
File "/usr/local/lib/python3.9/site-packages/airflow/providers/microsoft/azure/hooks/synapse.py", line 144, in run_spark_job
job = self.get_conn().spark_batch.create_spark_batch_job(payload)
File "/usr/local/lib/python3.9/site-packages/azure/synapse/spark/operations/_spark_batch_operations.py", line 163, in create_spark_batch_job
request = self._client.post(url, query_parameters, header_parameters, **body_content_kwargs)
File "/usr/local/lib/python3.9/site-packages/azure/core/pipeline/transport/_base.py", line 659, in post
request = self._request(
File "/usr/local/lib/python3.9/site-packages/azure/core/pipeline/transport/_base.py", line 535, in _request
request = HttpRequest(method, self.format_url(url))
File "/usr/local/lib/python3.9/site-packages/azure/core/pipeline/transport/_base.py", line 582, in format_url
raise ValueError(err_msg.format(key.args[0]))
ValueError: The value provided for the url part endpoint was incorrect, and resulted in an invalid url
I also want to know the difference of abfss and wasbs and where should i upload my files so that the code will be able to grab the links ?
Maybe I am uploading the files at wrong place.
You have something wrong in the connection self.azure_synapse_conn_id, where the host (Synapse Workspace URL) is not valid, here is an example of the connection:
Connection(
conn_id=DEFAULT_CONNECTION_CLIENT_SECRET,
conn_type="azure_synapse",
host="https://testsynapse.dev.azuresynapse.net",
login="clientId",
password="clientSecret",
extra=json.dumps(
{
"extra__azure_synapse__tenantId": "tenantId",
"extra__azure_synapse__subscriptionId": "subscriptionId",
}
),
)
For the difference between abfss and wasbs, here is a detailed answer about the topic.
I am trying to learn firebase to build apps more easily, but I've been having slight issues with adding databases to the server.
Here is the code:
import pyrebase as pb
fbconfig = {"apiKey": "AIzaSyCy5R9rbQWt4AzBo5sZeGBKcoyfHZ1fD-s",
"authDomain": "fir-fb8e9.firebaseapp.com",
"databaseURL": "https://fir-fb8e9.firebaseio.com",
"projectId": "fir-fb8e9",
"storageBucket": "fir-fb8e9.appspot.com",
"messagingSenderId": "816508440796",
"appId": "1:816508440796:web:e03ef3c4810606f8faa419",
"measurementId": "G-0MWJ5YES8M"}
fb = pb.initialize_app(fbconfig)
db = fb.database()
d1={"age":16,"Name":"Ebraheem Alrawi","Group":"For Fun"}
db.push(d1)
I tried changing the databaseURL links over and over again but it would give me the same error
Traceback (most recent call last):
File "C:\Users\Test\PycharmProjects\speedplate\venv\lib\site-packages\pyrebase\pyrebase.py", line 503, in raise_detailed_error
request_object.raise_for_status()
File "C:\Users\Test\PycharmProjects\speedplate\venv\lib\site-packages\requests\models.py", line 943, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://fir-fb8e9.firebaseio.com/.json
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/Users/Test/PycharmProjects/speedplate/firebasetest/main.py", line 34, in <module>
db.push(d1)
File "C:\Users\Test\PycharmProjects\speedplate\venv\lib\site-packages\pyrebase\pyrebase.py", line 310, in push
raise_detailed_error(request_object)
File "C:\Users\Test\PycharmProjects\speedplate\venv\lib\site-packages\pyrebase\pyrebase.py", line 507, in raise_detailed_error
raise HTTPError(e, request_object.text)
requests.exceptions.HTTPError: [Errno 404 Client Error: Not Found for url: https://fir-fb8e9.firebaseio.com/.json] {
"error" : "404 Not Found"
}
Process finished with exit code 1
I tried searching in google but there were no results
According to the Pyrebase documentation found https://github.com/thisbejim/Pyrebase#database
Create a database reference location.
db = firebase.database()
db.child("users").child("Morty")
Save data with an auto-generated key.
data = {"name": "Mortimer 'Morty' Smith"}
db.child("users").push(data)
Define your own keys use the set() method.
data = {"name": "Mortimer 'Morty' Smith"}
db.child("users").child("Morty").set(data)
Update data for an existing entry.
db.child("users").child("Morty").update({"name": "Mortiest Morty"})
I am using below python authentication script to connect to ADLS using service principal details but it keeps throwing exception:azure.core.exceptions.HttpResponseError: (AuthorizationPermissionMismatch) This request is not authorized to perform this operation using this permission.
The role assigned to me is "Storage blob data owner" and not sure what is missing?
Python Code:
from azure.storage.filedatalake import DataLakeServiceClient
from azure.identity import ClientSecretCredential
TENANT_ID = 'XXXXXXXXXX'
CLIENT_ID = 'XXXXXXXXXX'
CLIENT_SECRET = 'XXXXXXXXXX'
STORAGE_ACCOUNT_NAME = 'XXXXXXXXXX'
credential = ClientSecretCredential(TENANT_ID, CLIENT_ID, CLIENT_SECRET)
service_client = DataLakeServiceClient(account_url="{}://{}.dfs.core.windows.net".format(
"https", STORAGE_ACCOUNT_NAME), credential=credential)
print(service_client.primary_endpoint) # Can see the primary endpoint.
file_system_client = service_client.get_file_system_client("my-container")
file_system_client.create_directory("test-dir") #Throwing the (AuthorizationPermissionMismatch) error.
print("test directory created.")
Complete Trace:
Traceback (most recent call last):
File "/home//lib/python3.5/site-packages/azure/storage/filedatalake/_path_client.py", line 200, in _create
return self._client.path.create(**options)
File "/home/lib/python3.5/site-packages/azure/storage/filedatalake/_generated/operations/_path_operations.py", line 248, in create
raise HttpResponseError(response=response, model=error)
azure.core.exceptions.HttpResponseError: (AuthorizationPermissionMismatch) This request is not authorized to perform this operation using this permission.
RequestId:80605399-e01f-0038-2cd2-0a4210000000
Time:2021-02-24T17:25:49.0969802Z
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "adls_client.py", line 30, in <module>
file_system_client.create_directory("test-dir")
File "/home/lib/python3.5/site-packages/azure/storage/filedatalake/_file_system_client.py", line 540, in create_directory
directory_client.create_directory(metadata=metadata, **kwargs)
File "/home/lib/python3.5/site-packages/azure/storage/filedatalake/_data_lake_directory_client.py", line 160, in create_directory
return self._create('directory', metadata=metadata, **kwargs)
File "/home/lib/python3.5/site-packages/azure/storage/filedatalake/_path_client.py", line 202, in _create
process_storage_error(error)
File "/home/lib/python3.5/site-packages/azure/storage/filedatalake/_deserialize.py", line 150, in process_storage_error
raise error
azure.core.exceptions.HttpResponseError: (AuthorizationPermissionMismatch) This request is not authorized to perform this operation using this permission.
I can reproduce your error:
I am pretty sure your code is no problem, and, Storage blob data owner is the right RBAC role of your AD app. I think maybe the problem comes from the RBAC role does not take effect immediately, you need to wait a while. And then it should work.
I'm trying to build an app using the spotipy python library to access the spotify api.
My oauth code looks like this and it seems to work except for initialising the client with the right auth parameter.
self.sp_auth=spotipy.oauth2.SpotifyOAuth(secrets.sp_auth_id,
secrets.sp_auth_pw, secrets.sp_callback_url,
scope="playlist-modify-public user-library-read", state=state)
...
url = self.sp_auth.get_authorize_url()
send url to user.
after user said she/he has given permission:
auth code is fetched from webserver and used to generate a token.
self.auth_token=self.sp_auth.get_access_token(self.auth_code)
self.auth_token then looks like this:
{'access_token' : 'BQD ... qE7K3PBZKB6iZFU3_4p',
'token_type' : 'Bearer',
'expires_in' : 3600,
'refresh_token' : 'AQCOS2Xo ... MK09ry7-a-fl61OwhuO1Q',
'scope' : 'playlist-modify-public user-library-read',
'expires_at' : 1548247835}
then I initialize the spotipy client module like this:
self.sp = spotipy.Spotify(auth=self.auth_token)
then I try the following:
playlists = self.sp.current_user_playlists(limit=10)
which raises this Exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/spotipy/client.py", line 119, in _internal_call
r.raise_for_status()
File "/usr/local/lib/python3.6/dist-packages/requests/models.py", line 940, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://api.spotify.com/v1/me/playlists?limit=10&offset=0
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/telegram/ext/dispatcher.py", line 279, in process_update
handler.handle_update(update, self)
File "/usr/local/lib/python3.6/dist-packages/telegram/ext/callbackqueryhandler.py", line 143, in handle_update
return self.callback(dispatcher.bot, update, **optional_args)
File "spotify_playlist_bot_v2.py", line 140, in button_auth_done
User.data[user_id].msg_start(bot, update)
File "spotify_playlist_bot_v2.py", line 84, in msg_start
self.msg_choose_playlist()
File "spotify_playlist_bot_v2.py", line 90, in msg_choose_playlist
playlists = self.sp.current_user_playlists(limit=10)
File "/usr/local/lib/python3.6/dist-packages/spotipy/client.py", line 355, in current_user_playlists
return self._get("me/playlists", limit=limit, offset=offset)
File "/usr/local/lib/python3.6/dist-packages/spotipy/client.py", line 146, in _get
return self._internal_call('GET', url, payload, kwargs)
File "/usr/local/lib/python3.6/dist-packages/spotipy/client.py", line 124, in _internal_call
headers=r.headers)
spotipy.client.SpotifyException: http status: 400, code:-1 - https://api.spotify.com/v1/me/playlists?limit=10&offset=0:
Only valid bearer authentication supported
It looks like I'm not passing the token to the spotipy client correctly. For example self.sp = spotipy.Spotify(auth="random_bullshit") gives me the same Exception. I also tried passing the token like this auth=self.auth_token['access_token'] with the same result. The documentation doesn't say anything about what the auth parameter should be exactly and I'm not really understanding the source code. But I'd say it suggests that auth=self.auth_token['access_token'] is the right thing to do.
Thanks!
As I already suggested in my last edit auth=self.auth_token['access_token'] was the right thing I just had a typo in it. Anyway since the Documentation doesn't say a lot about the auth parameter this might help some people.
When I run this simple code from Spotify's Docs:
import spotipy
birdy_uri = 'spotify:artist:2WX2uTcsvV5OnS0inACecP'
spotify = spotipy.Spotify()
results = spotify.artist_albums(birdy_uri, album_type='album')
albums = results['items']
while results['next']:
results = spotify.next(results)
albums.extend(results['items'])
for album in albums:
print(album['name'])
I got this exception:
Traceback (most recent call last):
File "/Users/haodong/Documents/Projects/python-workspace/env3/lib/python3.4/site-packages/spotipy/client.py", line 119, in _internal_call
r.raise_for_status()
File "/Users/haodong/Documents/Projects/python-workspace/env3/lib/python3.4/site-packages/requests/models.py", line 844, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://api.spotify.com/v1/search?q=artist%3ARadiohead&offset=0&type=artist&limit=10
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "test.py", line 19, in <module>
results = spotify.search(q='artist:' + name, type='artist')
File "/Users/haodong/Documents/Projects/python-workspace/env3/lib/python3.4/site-packages/spotipy/client.py", line 339, in search
return self._get('search', q=q, limit=limit, offset=offset, type=type, market=market)
File "/Users/haodong/Documents/Projects/python-workspace/env3/lib/python3.4/site-packages/spotipy/client.py", line 146, in _get
return self._internal_call('GET', url, payload, kwargs)
File "/Users/haodong/Documents/Projects/python-workspace/env3/lib/python3.4/site-packages/spotipy/client.py", line 124, in _internal_call
headers=r.headers)
spotipy.client.SpotifyException: http status: 401, code:-1 - https://api.spotify.com/v1/search?q=artist%3ARadiohead&offset=0&type=artist&limit=10:
No token provided
You need to use your Spotify app credentials (Client ID and Client Secret) from www.developer.spotify.com, assign it to a variable and use that as your object.
import spotipy
from spotipy.oauth2 import SpotifyClientCredentials
cid ="Your-client-ID"
secret = "Your-client-secret"
client_credentials_manager = SpotifyClientCredentials(client_id=cid, client_secret=secret)
sp = spotipy.Spotify(client_credentials_manager=client_credentials_manager)
#Then run your query
results = sp.artist_albums(birdy_uri, album_type='album'))
#### ETC ######
More info here: Client Credential Flow
It seems that Spotify Web API has been updated recently and requires authorization for all kinds of requests.
Use authorized requests would solve this problem.