I created a Shared Access Signature(SAS) token on my Azure storage account using the web interface. The token looks like
?sv=xxxx-xx-xx&ss=b&srt=sco&sp=rl&se=xxxx-xx-xxTxx:xx:xxZ&st=xxxx-xx-xxTxx:xx:xxZ&spr=https&sig=xxxxxxxxxxxxxxxxxxxxxx
The SAS token here is missing the sr field for Service Resource. I have to manually prepend the sr=b to the query string to get things to work. I must be doing something wrong, because this seems extremely finicky.
from azure.storage.blob import BlockBlobService
sas_token = "?sv=xxxx-xx-xx&ss=b&srt=sco&sp=rl&se=xxxx-xx-xxTxx:xx:xxZ&st=xxxx-xx-xxTxx:xx:xxZ&spr=https&sig=xxxxxxxxxxxxxxxxxxxxxx"
sas_token = "?sr=b&" + sas_token[1:]
serv = BlockBlobService(account_name='myaccount', sas_token=sas_token)
for cont in serv.list_containers():
print cont.name
Without the sas_token = "?sr=b&" + sas_token[1:] I get the error:
sr is mandatory. Cannot be empty
And if the sr=b field is not first in the query, I get an authentication error like
Access without signed identifier cannot have time window more than 1 hour
Access without signed identifier cannot have time window more than 1 hour
Based on this error message, you may need to set expiry time less than 1 hour from now. See Windows Azure Shared Access Signature always gives: Forbidden 403.
I took your code with Python v2.7.12 and #azure-storage-python v0.34.3 (the latest version). And it worked well on my site. So, I'd recommend you upgrade to latest version and try it again.
UPDATE:
I traced the code of Azure Storage SDK for Python and here's what I found. The SDK is a REST API warpper which assumes that the SAS token looks like this:
sv=2015-04-05&ss=bfqt&srt=sco&sp=rl&se=2015-09-20T08:49Z&sip=168.1.5.60-168.1.5.70&sig=a39%2BYozJhGp6miujGymjRpN8tsrQfLo9Z3i8IRyIpnQ%3d
As you can see, the token doesn't include ?. And the SDK will append ? before the SAS token when it makes a GET request to the Azure Storage REST service.
This would cause that the key of the signed version was parsed as ?sv, then it raised the issue. So, to avoid this, we should remove the ? from the SAS token.
Related
I have created an Azure function which is trigered when a new file is added to my Blob Storage. This part works well !
BUT, now I would like to start the "Speech-To-Text" Azure service using the API. So I try to create my URI leading to my new blob and then add it to the API call. To do so I created an SAS Token (From Azure Portal) and I add it to my new Blob Path .
https://myblobstorage...../my/new/blob.wav?[SAS Token generated]
By doing so I get an error which says :
Authentification failed Invalid URI
What am I missing here ?
N.B : When I generate manually the SAS token from the "Azure Storage Explorer" everything is working well. Plus my token is not expired in my test
Thank you for your help !
You might generate the SAS token with wrong authentication.
Make sure the Object option is checked.
Here is the reason in docs:
Service (s): Access to service-level APIs (e.g., Get/Set Service Properties, Get Service Stats, List Containers/Queues/Tables/Shares)
Container (c): Access to container-level APIs (e.g., Create/Delete Container, Create/Delete Queue, Create/Delete Table, Create/Delete
Share, List Blobs/Files and Directories)
Object (o): Access to object-level APIs for blobs, queue messages, table entities, and files(e.g. Put Blob, Query Entity, Get Messages,
Create File, etc.)
I'm trying to access Azure databricks spark cluster by a python script which takes token as an input generated via databricks user settings and calling a Get method to get the details of the cluster alongwith the cluster-id.
The below is the code snippet. As shown, I have created a cluster in southcentralus zone.
import requests
headers = {"Authorization":"Bearer dapiad************************"}
data=requests.get("https://southcentralus.azuredatabricks.net/api/2.0/clusters/get?cluster_id=**************",headers=headers).text
print data
Expected result should give the full detail of the cluster eg.
{"cluster_id":"0128-******","spark_context_id":3850138716505089853,"cluster_name":"abcdxyz","spark_version":"5.1.x-scala2.11","spark_conf":{"spark.databricks.delta.preview.enabled":"true"},"node_type_id" and so on .....}
The above code is working when I execute the code on google colaboratory whereas the same is not working with my local IDE i.e. idle. It gives the error of HTTP 403 stating as below:
<p>Problem accessing /api/2.0/clusters/get. Reason:
<pre> Invalid access token.</pre></p>
Can anyone help me resolve the issue? I'm stuck on this part and not able to access the cluster through APIs.
It could be due to encoding issue when you pass the secret. Please look into this issue and how to resolve it. Even though the resolution they have given for AWS,it could be similar one for Azure as well. Your secret might be having "/", which you have to replace.
There is a known problem in the last update related to the '+'
character in secret keys. In particular, we no longer support escaping
'+' into '%2B', which some URL-encoding libraries do.
The current best-practice way of encoding your AWS secret key is
simply
secretKey.replace("/","%2F")
sample python script is given below:
New_Secret_key = "MySecret/".replace("/","%2F")
https://forums.databricks.com/questions/6590/s3serviceexception-raised-when-accessing-via-mount.html
https://forums.databricks.com/questions/6621/responsecode403-responsemessageforbidden.html
I have problems with the authentication in the Python Library of Google Cloud API.
At first it worked for some days without problem, but suddenly the API calls are not showing up in the API Overview of the Google CloudPlatform.
I created a service account and stored the json file locally. Then I set the environment variable GCLOUD_PROJECT to the project ID and GOOGLE_APPLICATION_CREDENTIALS to the path of the json file.
from google.cloud import speech
client = speech.Client()
print(client._credentials.service_account_email)
prints the correct service account email.
The following code transcribes the audio_file successfully, but the Dashboard for my Google Cloud project doesn't show anything for the activated Speech API Graph.
import io
with io.open(audio_file, 'rb') as f:
audio = client.sample(f.read(), source_uri=None, sample_rate=48000, encoding=speech.encoding.Encoding.FLAC)
alternatives = audio.sync_recognize(language_code='de-DE')
At some point the code also ran in some errors, regarding the usage limit. I guess due to the unsuccessful authentication, the free/limited option is used somehow.
I also tried the alternative option for authentication by installing the Google Cloud SDK and gcloud auth application-default login, but without success.
I have no idea where to start troubleshooting the problem.
Any help is appreciated!
(My system is running Windows 7 with Anaconda)
EDIT:
The error count (Fehler) is increasing with calls to the API. How can I get detailed information about the error?!
Make sure you are using an absolute path when setting the GOOGLE_APPLICATION_CREDENTIALS environment variable. Also, you might want to try inspecting the access token using OAuth2 tokeninfo and make sure it has "scope": "https://www.googleapis.com/auth/cloud-platform" in its response.
Sometimes you will get different error information if you initialize the client with GRPC enabled:
0.24.0:
speech_client = speech.Client(_use_grpc=True)
0.23.0:
speech_client = speech.Client(use_gax=True)
Usually it's an encoding issue, can you try with the sample audio or try generating LINEAR16 samples using something like the Unix rec tool:
rec --channels=1 --bits=16 --rate=44100 audio.wav trim 0 5
...
with io.open(speech_file, 'rb') as audio_file:
content = audio_file.read()
audio_sample = speech_client.sample(
content,
source_uri=None,
encoding='LINEAR16',
sample_rate=44100)
Other notes:
Sync Recognize is limited to 60 seconds of audio, you must use async for longer audio
If you haven't already, set up billing for your account
With regards to the usage problem, the issue is in fact that when you use the new google-cloud library to access ML APIs, it seems everyone authenticates to a project shared by everyone (hence it says you've used up your limit even though you've not used anything). To check and confirm this, you can call an ML API that you have not enabled by using the python client library, which will give you a result even though it shouldn't. This problem persists to other language client libraries and OS, so I suspect it's an issue with their grpc.
Because of this, to ensure consistency I always use the older googleapiclient that uses my API key. Here is an example to use the translate API:
from googleapiclient import discovery
service = discovery.build('translate', 'v2', developerKey='')
service_request = service.translations().list(q='hello world', target='zh')
result = service_request.execute()
print(result)
For the speech API, it's something along the lines of:
from googleapiclient import discovery
service = discovery.build('speech', 'v1beta1', developerKey='')
service_request = service.speech().syncrecognize()
result = service_request.execute()
print(result)
You can get the list of the discovery APIs at https://developers.google.com/api-client-library/python/apis/ with the speech one located in https://developers.google.com/resources/api-libraries/documentation/speech/v1beta1/python/latest/.
One of the other benefits of using the discovery library is that you get a lot more options compared to the current library, although often times it's a bit more of a pain to implement.
I'm having difficulty signing GET requests for Google Cloud Storage (GCS) when specifying a 'generation' (version number) on a the object. Signing the URL without the generation works like a charm and GET requests work fine. However, when I prepend #generation to the path, the GCS server always returns "access denied" when attempting to GET the signed URL.
For example, signing this URL path works fine:
https://storage.googleapis.com/BUCKET/OBJECT
signing this URL path gives me access denied:
https://storage.googleapis.com/BUCKET/OBJECT#1360887697105000
Note that for brevity and privacy, I've omitted what the actual signed URL with Signature, Expires, GoogleAccessId parameters. Also note, that I have also verified the bucket, object, and generation are correct using gsutil.
Does GCS allow for Signed URL access to specific object versions by 'generation' number? Is the URL signing procedure different when accessing a specific version?
The URL you're using is gsutil-compatible, but the XML API requires that you denote generation with a query parameter (which would look like storage.googleapis.com/BUCKET/OBJECT?generation=1360887697105000).
Documentation is here for reference: developers.google.com/storage/docs/reference-headers#generation
I'm using facepy for retrieving fb comments and posts (This is the objective). I've given a valid access token (generated from http://developers.facebook.com/tools/explorer/?method=GET&path=me)
The error generated is:
NameError: name 'AAACEdEose0cBAHshwZCGJ6dHPb0x68c.......D' is not defined. And/OR
facepy.exceptions.OAuthError: [190] Error validating access token: Session has expired at unix time 1359752400.
I believe I've generated the correct access token (with the correct permissions) I thought I would test the facepy library for retrieving photos first. Is their a faster way to retrieve comments, respective user ids, and time stamp from a public fan page (I am guessing most of my target pages have 'public information')
Can someone help me on this?
from facepy import GraphAPI
graph = GraphAPI(AAACEdEose0cBAHshwZCGJ6dHPb0x68c.......D)
graph.get('me/posts')
graph.post(
path = 'me/photos',
source = open('parrot.jpg')
Never tried with the FB but some of the solutions which worked with other API's.
Error validating access token: Session has expired at unix time 1359752400.
Says All. However mostly occurs when I did connected it to some DB for storing some information retrieved from the API.I am sure you have made a right key and password so don't worry about it. Make sure the other connection is still open.
Second, it happened again when I did not accessed the API for weeks so make sure to log in once or twice.
Your API needs to be in quotes...
also generate an extended token via the below