I am trying to use Google speech-to-text service, according to https://googleapis.github.io/google-cloud-python/latest/speech/index.html
I have created project, uploaded audio to gs: cloud, added permissions, downloaded json file named My First Project-7bb85a480131.json. https://console.cloud.google.com/storage/browser/mybucket?project=my-project
that is my file:
import os
os.environ["GOOGLE_APPLICATION_CREDENTIALS"]="/home/joo/Документы/LocalRepository/robotze/My First Project-7bb85a480131.json"
from google.cloud import speech
client = speech.SpeechClient()
audio = speech.types.RecognitionAudio(
uri='gs://zaudio/audio.mp3')
config = speech.types.RecognitionConfig(
encoding=speech.enums.RecognitionConfig.AudioEncoding.LINEAR16,
language_code='ru-RU',
sample_rate_hertz=44100)
operation = client.long_running_recognize(config=config, audio=audio)
op_result = operation.result()
for result in op_result.results:
for alternative in result.alternatives:
print('=' * 20)
print(alternative.transcript)
print(alternative.confidence)
Issue: i got
google.api_core.exceptions.PermissionDenied: 403 my-service-account#my-project.iam.gserviceaccount.com does not have storage.objects.get access to mybucket/audio.mp3.
Full traceback
/home/joo/anaconda3/bin/python /home/joo/Документы/LocalRepository/robotze/speech-to-text-googlecloud.py
Traceback (most recent call last):
File "/home/joo/anaconda3/lib/python3.6/site-packages/google/api_core/grpc_helpers.py", line 57, in error_remapped_callable
return callable_(*args, **kwargs)
File "/home/joo/anaconda3/lib/python3.6/site-packages/grpc/_channel.py", line 565, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/home/joo/anaconda3/lib/python3.6/site-packages/grpc/_channel.py", line 467, in _end_unary_response_blocking
raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
status = StatusCode.PERMISSION_DENIED
details = "my-service-account#my-project.iam.gserviceaccount.com does not have storage.objects.get access to mybucket/audio.mp3."
debug_error_string = "{"created":"#1565253582.126380437","description":"Error received from peer ipv4:74.125.131.95:443","file":"src/core/lib/surface/call.cc","file_line":1052,"grpc_message":"my-service-account#my-project.iam.gserviceaccount.com does not have storage.objects.get access to mybucket/audio.mp3.","grpc_status":7}"
>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/joo/Документы/LocalRepository/robotze/speech-to-text-googlecloud.py", line 46, in <module>
operation = client.long_running_recognize(config=config, audio=audio)
File "/home/joo/anaconda3/lib/python3.6/site-packages/google/cloud/speech_v1/gapic/speech_client.py", line 341, in long_running_recognize
request, retry=retry, timeout=timeout, metadata=metadata
File "/home/joo/anaconda3/lib/python3.6/site-packages/google/api_core/gapic_v1/method.py", line 143, in __call__
return wrapped_func(*args, **kwargs)
File "/home/joo/anaconda3/lib/python3.6/site-packages/google/api_core/retry.py", line 273, in retry_wrapped_func
on_error=on_error,
File "/home/joo/anaconda3/lib/python3.6/site-packages/google/api_core/retry.py", line 182, in retry_target
return target()
File "/home/joo/anaconda3/lib/python3.6/site-packages/google/api_core/timeout.py", line 214, in func_with_timeout
return func(*args, **kwargs)
File "/home/joo/anaconda3/lib/python3.6/site-packages/google/api_core/grpc_helpers.py", line 59, in error_remapped_callable
six.raise_from(exceptions.from_grpc_error(exc), exc)
File "<string>", line 3, in raise_from
google.api_core.exceptions.PermissionDenied: 403 my-service-account#my-project.iam.gserviceaccount.com does not have storage.objects.get access to mybucket/audio.mp3.
Process finished with exit code 1
What i tried: gcloud auth application-default login - login in browser works, but still 403 error
From what i can see on your logs, you are able to authenticate your service account inside your code (you are currently authenticating with: starting-account-*******-239919.iam.gserviceaccount.com), however, that service account doesn't have "storage.objects.get" permission over the object "zaudio/audio.mp3".
So you can either:
A.- Give the proper permissions to that service account (may be the role "storage.objectViewer" inside that bucket would be enough, but you could also set it with the role "storage.admin" so it can have more control over that bucket and others).
B.- Authenticate using other service account that have the proper permissions.
I resolved the following issue:
“google.api_core.exceptions.PermissionDenied: 403 my-service-account#my-project.iam.gserviceaccount.com does not have storage.objects.get access to mybucket/audio.mp3.”
To resolve this issue: go to your bucket, click that three dots, choose “Edit permissions”, Entity set “User”, Name set your email (in this case, my-service-account#my-project.iam.gserviceaccount.com), Access: “Reader”. Save and try again. This should resolve this issue. Regardless whether you have created the bucket, whatnot, you have to do this step to explicitly set permission. Hopefully this is useful.
Related
I would like to extract the reporting data of our mobile apps from Google Play Console.
According to this documentation, we have to use SignedJwtAssertionCredentials to authenticate. After some errors and research it seems that the code snippet in the doc is kind of outdated (since SignedJWTAssertionCredentials is not available anymore). Instead we tried the following to automatically download the reports:
from oauth2client.service_account import ServiceAccountCredentials
from apiclient.discovery import build
scopes = ['https://www.googleapis.com/auth/analytics.readonly']
key_file_location = 'files/access_token/mykeyfile.json'
credentials = ServiceAccountCredentials.from_json_keyfile_name(key_file_location, scopes)
cloud_storage_bucket = 'pubsite_prod_rev_123456789'
report_to_download = 'installs/installs_com.someapp.etc.etc_2021*'
storage = build('storage', 'v1', credentials=credentials)
print( storage.objects().get(bucket = cloud_storage_bucket,object = report_to_download).execute())
Now the problem is, that we receive the following error message:
Traceback (most recent call last):
File "C:\Users\dev\dev\z_10_ext_google_play_store.py", line 30, in <module>
print( storage.objects().get(bucket = cloud_storage_bucket,object = report_to_download).execute())
File "D:\Software\Python\lib\site-packages\googleapiclient\_helpers.py", line 130, in positional_wrapper
return wrapped(*args, **kwargs)
File "D:\Software\Python\lib\site-packages\googleapiclient\http.py", line 938, in execute
raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://storage.googleapis.com/storage/v1/b/pubsite_prod_rev_123456789/o/installs%2Finstalls_com.someapp.etc.etc_2021%2A?alt=json returned "Insufficient Permission". Details: "[{'message': 'Insufficient Permission', 'domain': 'global', 'reason': 'insufficientPermissions'}]">
Does anyone have an idea? Our service account even has admin rights in play console (which normally would be to much..)
UPDATE
As DaImTo pointed out, I was missing some scopes which lead me to a different line of scope definition:
scopes = ['https://www.googleapis.com/auth/analytics.readonly', 'https://www.googleapis.com/auth/devstorage.read_only'
,'https://www.googleapis.com/auth/cloud-platform.read_only']
But I still receive some errors, now because of the missing access token which is weird because the .json file contains everything necessary:
Traceback (most recent call last):
File "C:\Users\andre\Desktop\DAWSE\z_10_ext_google_play_store.py", line 33, in <module>
print( storage.objects().get(bucket = cloud_storage_bucket,object = report_to_download).execute())
File "D:\Sonstige Programme\Python\lib\site-packages\googleapiclient\_helpers.py", line 130, in positional_wrapper
return wrapped(*args, **kwargs)
File "D:\Sonstige Programme\Python\lib\site-packages\googleapiclient\http.py", line 923, in execute
resp, content = _retry_request(
File "D:\Sonstige Programme\Python\lib\site-packages\googleapiclient\http.py", line 191, in _retry_request
resp, content = http.request(uri, method, *args, **kwargs)
File "D:\Sonstige Programme\Python\lib\site-packages\oauth2client\transport.py", line 159, in new_request
credentials._refresh(orig_request_method)
File "D:\Sonstige Programme\Python\lib\site-packages\oauth2client\client.py", line 749, in _refresh
self._do_refresh_request(http)
File "D:\Sonstige Programme\Python\lib\site-packages\oauth2client\client.py", line 785, in _do_refresh_request
self.access_token = d['access_token']
KeyError: 'access_token'
You appear to be using the objects.get method
The authenticated user must have sufficient permission to use this method. To start with i would ensure your service account has access.
After that i would check your scopes you are currently only loading the scope for google analytics api.
scopes = ['https://www.googleapis.com/auth/analytics.readonly']
This scope will not grant you access to the google cloud storage objects get method. For that you need to use one of the google cloud storage scopes
I just want to learn how to store data in Firestore using Python and Google Cloud Platform, so I'm calling an API to query some example data.
For it, I'm using requests library and Firebase library from google.cloud package.
Here is the code that I'm running at Cloud Shell:
import requests
from google.cloud import firestore
url = "https://api.coindesk.com/v1/bpi/currentprice.json"
r = requests.get(url)
resp: str = r.text
if not (resp=="null" or resp=="[]"):
db = firestore.Client()
doc_ref=db.collection("CoinData").add(r.json())
When the code try to connect to Firebase to add the json of the API response I got this error:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/google/api_core/grpc_helpers.py", line 66, in error_remapped_callable
return callable_(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/grpc/_channel.py", line 946, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/usr/local/lib/python3.7/dist-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.INVALID_ARGUMENT
details = "Invalid resource field value in the request."
debug_error_string = "{"created":"#1634689546.004704998","description":"Error received from peer ipv4:74.125.134.95:443","file":"src/core/lib/surface/call.cc","file_line":1070,"grpc_message":"Invalid resource field value in the request.","grpc_status":3}"
>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/antonyare_93/cloudshell_open/pruebas/data_coin_query.py", line 11, in <module>
doc_ref=db.collection("CoinData").add(r.json())
File "/home/antonyare_93/.local/lib/python3.7/site-packages/google/cloud/firestore_v1/collection.py", line 107, in add
write_result = document_ref.create(document_data, **kwargs)
File "/home/antonyare_93/.local/lib/python3.7/site-packages/google/cloud/firestore_v1/document.py", line 99, in create
write_results = batch.commit(**kwargs)
File "/home/antonyare_93/.local/lib/python3.7/site-packages/google/cloud/firestore_v1/batch.py", line 60, in commit
request=request, metadata=self._client._rpc_metadata, **kwargs,
File "/home/antonyare_93/.local/lib/python3.7/site-packages/google/cloud/firestore_v1/services/firestore/client.py", line 815, in commit
response = rpc(request, retry=retry, timeout=timeout, metadata=metadata,)
File "/usr/local/lib/python3.7/dist-packages/google/api_core/gapic_v1/method.py", line 142, in __call__
return wrapped_func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/google/api_core/retry.py", line 288, in retry_wrapped_func
on_error=on_error,
File "/usr/local/lib/python3.7/dist-packages/google/api_core/retry.py", line 190, in retry_target
return target()
File "/usr/local/lib/python3.7/dist-packages/google/api_core/grpc_helpers.py", line 68, in error_remapped_callable
raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.InvalidArgument: 400 Invalid resource field value in the request.
Anyone knows how I can fix it?
I've just got it.
It was a little mistake, I was missing the name of the project at the firestore client method call, here's the code working:
import requests
from google.cloud import firestore
url = "https://api.coindesk.com/v1/bpi/currentprice.json"
r = requests.get(url)
resp: str = r.text
if not (resp=="null" or resp=="[]"):
db = firestore.Client(project="mytwitterapitest")
doc_ref=db.collection("CoinData").add(r.json())
During a PythonScriptStep in an Azure ML Pipeline, I'm saving a model as joblib pickle dump to a directory in a Blob Container in the Azure Blob Storage which I've created during the setup of the Azure ML Workspace. Afterwards I'm trying to upload this model file to the step run's output directory using
Run.upload_file (name, path_or_stream)
(for the function's documentation, see https://learn.microsoft.com/en-us/python/api/azureml-core/azureml.core.run(class)?view=azure-ml-py#upload-file-name--path-or-stream--datastore-name-none-)
Some time ago when I created the script using the azureml-sdk version 1.18.0, everything worked fine. Now, I've updated the script's functionalities and upgraded the azureml-sdk to version 1.33.0 during the process and the upload function now runs into the following error:
Traceback (most recent call last):
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_file_utils/upload.py", line 64, in upload_blob_from_stream
validate_content=True)
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_restclient/clientbase.py", line 93, in execute_func_with_reset
return ClientBase._execute_func_internal(backoff, retries, module_logger, func, reset_func, *args, **kwargs)
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_restclient/clientbase.py", line 367, in _execute_func_internal
left_retry = cls._handle_retry(back_off, left_retry, total_retry, error, logger, func)
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_restclient/clientbase.py", line 399, in _handle_retry
raise error
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_restclient/clientbase.py", line 358, in _execute_func_internal
response = func(*args, **kwargs)
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_vendor/azure_storage/blob/blockblobservice.py", line 614, in create_blob_from_stream
initialization_vector=iv
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_vendor/azure_storage/blob/_upload_chunking.py", line 98, in _upload_blob_chunks
range_ids = [f.result() for f in futures]
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_vendor/azure_storage/blob/_upload_chunking.py", line 98, in <listcomp>
range_ids = [f.result() for f in futures]
File "/opt/miniconda/lib/python3.7/concurrent/futures/_base.py", line 435, in result
return self.__get_result()
File "/opt/miniconda/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/opt/miniconda/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_vendor/azure_storage/blob/_upload_chunking.py", line 210, in process_chunk
return self._upload_chunk_with_progress(chunk_offset, chunk_bytes)
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_vendor/azure_storage/blob/_upload_chunking.py", line 224, in _upload_chunk_with_progress
range_id = self._upload_chunk(chunk_offset, chunk_data)
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_vendor/azure_storage/blob/_upload_chunking.py", line 269, in _upload_chunk
timeout=self.timeout,
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_vendor/azure_storage/blob/blockblobservice.py", line 1013, in _put_block
self._perform_request(request)
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_vendor/azure_storage/common/storageclient.py", line 432, in _perform_request
raise ex
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_vendor/azure_storage/common/storageclient.py", line 357, in _perform_request
raise ex
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_vendor/azure_storage/common/storageclient.py", line 343, in _perform_request
HTTPError(response.status, response.message, response.headers, response.body))
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_vendor/azure_storage/common/_error.py", line 115, in _http_error_handler
raise ex
azure.common.AzureHttpError: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. ErrorCode: AuthenticationFailed
<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:5d4e1b7e-c01e-0070-0d47-9bf8a0000000
Time:2021-08-27T13:30:02.2685991Z</Message><AuthenticationErrorDetail>Signature did not match. String to sign used was rcw
2021-08-27T13:19:56Z
2021-08-28T13:29:56Z
/blob/mystorage/azureml/ExperimentRun/dcid.98d11a7b-2aac-4bc0-bd64-bb4d72e0e0be/outputs/models/Model.pkl
2019-07-07
b
</AuthenticationErrorDetail></Error>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/mnt/batch/tasks/shared/LS_root/jobs/.../azureml-setup/context_manager_injector.py", line 243, in execute_with_context
runpy.run_path(sys.argv[0], globals(), run_name="__main__")
File "/opt/miniconda/lib/python3.7/runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "/opt/miniconda/lib/python3.7/runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "/opt/miniconda/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "401_AML_Pipeline_Time_Series_Model_Training_Azure_ML_CPU.py", line 318, in <module>
main()
File "401_AML_Pipeline_Time_Series_Model_Training_Azure_ML_CPU.py", line 286, in main
path_or_stream=model_path)
File "/opt/miniconda/lib/python3.7/site-packages/azureml/core/run.py", line 53, in wrapped
return func(self, *args, **kwargs)
File "/opt/miniconda/lib/python3.7/site-packages/azureml/core/run.py", line 1989, in upload_file
datastore_name=datastore_name)
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_restclient/artifacts_client.py", line 114, in upload_artifact
return self.upload_artifact_from_path(artifact, *args, **kwargs)
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_restclient/artifacts_client.py", line 107, in upload_artifact_from_path
return self.upload_artifact_from_stream(stream, *args, **kwargs)
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_restclient/artifacts_client.py", line 99, in upload_artifact_from_stream
content_type=content_type, session=session)
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_restclient/artifacts_client.py", line 88, in upload_stream_to_existing_artifact
timeout=TIMEOUT, backoff=BACKOFF_START, retries=RETRY_LIMIT)
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_file_utils/upload.py", line 71, in upload_blob_from_stream
raise AzureMLException._with_error(azureml_error, inner_exception=e)
azureml._common.exceptions.AzureMLException: AzureMLException:
Message: Encountered authorization error while uploading to blob storage. Please check the storage account attached to your workspace. Make sure that the current user is authorized to access the storage account and that the request is not blocked by a firewall, virtual network, or other security setting.
StorageAccount: mystorage
ContainerName: azureml
StatusCode: 403
InnerException Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. ErrorCode: AuthenticationFailed
<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:5d4e1b7e-c01e-0070-0d47-9bf8a0000000
Time:2021-08-27T13:30:02.2685991Z</Message><AuthenticationErrorDetail>Signature did not match. String to sign used was rcw
2021-08-27T13:19:56Z
2021-08-28T13:29:56Z
/blob/mystorage/azureml/ExperimentRun/dcid.98d11a7b-2aac-4bc0-bd64-bb4d72e0e0be/outputs/models/Model.pkl
2019-07-07
b
</AuthenticationErrorDetail></Error>
ErrorResponse
{
"error": {
"code": "UserError",
"message": "Encountered authorization error while uploading to blob storage. Please check the storage account attached to your workspace. Make sure that the current user is authorized to access the storage account and that the request is not blocked by a firewall, virtual network, or other security setting.\n\tStorageAccount: mystorage\n\tContainerName: azureml\n\tStatusCode: 403",
"inner_error": {
"code": "Auth",
"inner_error": {
"code": "Authorization"
}
}
}
}
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "401_AML_Pipeline_Time_Series_Model_Training_Azure_ML_CPU.py", line 318, in <module>
main()
File "401_AML_Pipeline_Time_Series_Model_Training_Azure_ML_CPU.py", line 286, in main
path_or_stream=model_path)
File "/opt/miniconda/lib/python3.7/site-packages/azureml/core/run.py", line 53, in wrapped
return func(self, *args, **kwargs)
File "/opt/miniconda/lib/python3.7/site-packages/azureml/core/run.py", line 1989, in upload_file
datastore_name=datastore_name)
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_restclient/artifacts_client.py", line 114, in upload_artifact
return self.upload_artifact_from_path(artifact, *args, **kwargs)
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_restclient/artifacts_client.py", line 107, in upload_artifact_from_path
return self.upload_artifact_from_stream(stream, *args, **kwargs)
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_restclient/artifacts_client.py", line 99, in upload_artifact_from_stream
content_type=content_type, session=session)
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_restclient/artifacts_client.py", line 88, in upload_stream_to_existing_artifact
timeout=TIMEOUT, backoff=BACKOFF_START, retries=RETRY_LIMIT)
File "/opt/miniconda/lib/python3.7/site-packages/azureml/_file_utils/upload.py", line 71, in upload_blob_from_stream
raise AzureMLException._with_error(azureml_error, inner_exception=e)
UserScriptException: UserScriptException:
Message: Encountered authorization error while uploading to blob storage. Please check the storage account attached to your workspace. Make sure that the current user is authorized to access the storage account and that the request is not blocked by a firewall, virtual network, or other security setting.
StorageAccount: mystorage
ContainerName: azureml
StatusCode: 403
InnerException AzureMLException:
Message: Encountered authorization error while uploading to blob storage. Please check the storage account attached to your workspace. Make sure that the current user is authorized to access the storage account and that the request is not blocked by a firewall, virtual network, or other security setting.
StorageAccount: mystorage
ContainerName: azureml
StatusCode: 403
InnerException Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. ErrorCode: AuthenticationFailed
<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:5d4e1b7e-c01e-0070-0d47-9bf8a0000000
Time:2021-08-27T13:30:02.2685991Z</Message><AuthenticationErrorDetail>Signature did not match. String to sign used was rcw
2021-08-27T13:19:56Z
2021-08-28T13:29:56Z
/blob/mystorage/azureml/ExperimentRun/dcid.98d11a7b-2aac-4bc0-bd64-bb4d72e0e0be/outputs/models/Model.pkl
2019-07-07
b
</AuthenticationErrorDetail></Error>
ErrorResponse
{
"error": {
"code": "UserError",
"message": "Encountered authorization error while uploading to blob storage. Please check the storage account attached to your workspace. Make sure that the current user is authorized to access the storage account and that the request is not blocked by a firewall, virtual network, or other security setting.\n\tStorageAccount: verovisionstorage\n\tContainerName: azureml\n\tStatusCode: 403",
"inner_error": {
"code": "Auth",
"inner_error": {
"code": "Authorization"
}
}
}
}
ErrorResponse
{
"error": {
"code": "UserError",
"message": "Encountered authorization error while uploading to blob storage. Please check the storage account attached to your workspace. Make sure that the current user is authorized to access the storage account and that the request is not blocked by a firewall, virtual network, or other security setting.\n\tStorageAccount: mystorage\n\tContainerName: azureml\n\tStatusCode: 403"
}
}
As far as I can tell from the code of the azureml.core.Run class and the subsequent function calls, the Run object tries to upload the file to the step run's output directory using SAS-Token-Authentication (which fails). This documentation article is linked in the code (but I don't know if this relates to the issue): https://learn.microsoft.com/en-us/rest/api/storageservices/create-service-sas#service-sas-example
Did anybody encounter this error as well and knows what causes it or how it can be resolved?
Best,
Jonas
We’ve seen the before, it’s annoying. I think the answer is to go to the data stores page of the AML Studio UI and manually enter the storage account key again.
I am following this document: https://cloud.google.com/text-to-speech/docs/quickstart-client-libraries#client-libraries-install-python
I have followed the instructions. I get an error when I run the provided Python code ("Create audio data" section).
First Try:
It appears the problem is with authentication. I authenticated with gcloud init.
C:\webDev\pycharm\TTS-Nov-\Scripts\python.exe C:/webDev/pycharm/TTS-Nov-/TTS-Main.py
C:\webDev\pycharm\TTS-Nov-\lib\site-packages\google\auth\_default.py:66: UserWarning: Your application has authenticated using end user credentials from Google Cloud SDK. We recommend that most server applications use service accounts instead. If your application continues to use end user credentials from Cloud SDK, you might receive a "quota exceeded" or "API not enabled" error. For more information about service accounts, see https://cloud.google.com/docs/authentication/
warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)
Traceback (most recent call last):
File "C:\webDev\pycharm\TTS-Nov-\lib\site-packages\google\api_core\grpc_helpers.py", line 59, in error_remapped_callable
return callable_(*args, **kwargs)
File "C:\webDev\pycharm\TTS-Nov-\lib\site-packages\grpc\_channel.py", line 533, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "C:\webDev\pycharm\TTS-Nov-\lib\site-packages\grpc\_channel.py", line 467, in _end_unary_response_blocking
raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
status = StatusCode.PERMISSION_DENIED
details = "Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the texttospeech.googleapis.com. We recommend that most server applications use service accounts instead. For more information about service accounts and how to use them in your application, see https://cloud.google.com/docs/authentication/."
debug_error_string = "{"created":"#1541901610.961000000","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1017,"grpc_message":"Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the texttospeech.googleapis.com. We recommend that most server applications use service accounts instead. For more information about service accounts and how to use them in your application, see https://cloud.google.com/docs/authentication/.","grpc_status":7}"
>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:/webDev/pycharm/TTS-Nov-/TTS-Main.py", line 26, in <module>
response = client.synthesize_speech(synthesis_input, voice, audio_config)
File "C:\webDev\pycharm\TTS-Nov-\lib\site-packages\google\cloud\texttospeech_v1\gapic\text_to_speech_client.py", line 257, in synthesize_speech
request, retry=retry, timeout=timeout, metadata=metadata)
File "C:\webDev\pycharm\TTS-Nov-\lib\site-packages\google\api_core\gapic_v1\method.py", line 139, in __call__
return wrapped_func(*args, **kwargs)
File "C:\webDev\pycharm\TTS-Nov-\lib\site-packages\google\api_core\retry.py", line 260, in retry_wrapped_func
on_error=on_error,
File "C:\webDev\pycharm\TTS-Nov-\lib\site-packages\google\api_core\retry.py", line 177, in retry_target
return target()
File "C:\webDev\pycharm\TTS-Nov-\lib\site-packages\google\api_core\timeout.py", line 206, in func_with_timeout
return func(*args, **kwargs)
File "C:\webDev\pycharm\TTS-Nov-\lib\site-packages\google\api_core\grpc_helpers.py", line 61, in error_remapped_callable
six.raise_from(exceptions.from_grpc_error(exc), exc)
File "<string>", line 3, in raise_from
google.api_core.exceptions.PermissionDenied: 403 Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the texttospeech.googleapis.com. We recommend that most server applications use service accounts instead. For more information about service accounts and how to use them in your application, see https://cloud.google.com/docs/authentication/.
Second Try Update:
I ran this
gcloud auth activate-service-account test-2#tts-alphax-nov.iam.gserviceaccount.com --key-file="C:\webDev\keys\google-cloud\tts-alphax-nov-b101c56c70b7.json" --project=tts-alphax-nov
This resulted in:
Activated service account credentials for: [test-2#tts-alphax-nov.iam.gserviceaccount.com]
I then ran the python again and this was the error I got.
C:\webDev\pycharm\TTS-Nov-\lib\site-packages\google\auth\_default.py:66: UserWarning: Your application has authenticated using end user credentials from Google Cloud SDK. We recommend that most server applications use service accounts instead. If your application continues to use end user credentials from Cloud SDK, you might receive a "quota exceeded" or "API not enabled" error. For more information about service accounts, see https://cloud.google.com/docs/authentication/
warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)
Traceback (most recent call last):
File "C:\webDev\pycharm\TTS-Nov-\lib\site-packages\google\api_core\grpc_helpers.py", line 59, in error_remapped_callable
return callable_(*args, **kwargs)
File "C:\webDev\pycharm\TTS-Nov-\lib\site-packages\grpc\_channel.py", line 533, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "C:\webDev\pycharm\TTS-Nov-\lib\site-packages\grpc\_channel.py", line 467, in _end_unary_response_blocking
raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
status = StatusCode.PERMISSION_DENIED
details = "Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the texttospeech.googleapis.com. We recommend that most server applications use service accounts instead. For more information about service accounts and how to use them in your application, see https://cloud.google.com/docs/authentication/."
debug_error_string = "{"created":"#1541903472.332000000","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1017,"grpc_message":"Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the texttospeech.googleapis.com. We recommend that most server applications use service accounts instead. For more information about service accounts and how to use them in your application, see https://cloud.google.com/docs/authentication/.","grpc_status":7}"
>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:/webDev/pycharm/TTS-Nov-/TTS-Main.py", line 26, in <module>
response = client.synthesize_speech(synthesis_input, voice, audio_config)
File "C:\webDev\pycharm\TTS-Nov-\lib\site-packages\google\cloud\texttospeech_v1\gapic\text_to_speech_client.py", line 257, in synthesize_speech
request, retry=retry, timeout=timeout, metadata=metadata)
File "C:\webDev\pycharm\TTS-Nov-\lib\site-packages\google\api_core\gapic_v1\method.py", line 139, in __call__
return wrapped_func(*args, **kwargs)
File "C:\webDev\pycharm\TTS-Nov-\lib\site-packages\google\api_core\retry.py", line 260, in retry_wrapped_func
on_error=on_error,
File "C:\webDev\pycharm\TTS-Nov-\lib\site-packages\google\api_core\retry.py", line 177, in retry_target
return target()
File "C:\webDev\pycharm\TTS-Nov-\lib\site-packages\google\api_core\timeout.py", line 206, in func_with_timeout
return func(*args, **kwargs)
File "C:\webDev\pycharm\TTS-Nov-\lib\site-packages\google\api_core\grpc_helpers.py", line 61, in error_remapped_callable
six.raise_from(exceptions.from_grpc_error(exc), exc)
File "<string>", line 3, in raise_from
google.api_core.exceptions.PermissionDenied: 403 Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the texttospeech.googleapis.com. We recommend that most server applications use service accounts instead. For more information about service accounts and how to use them in your application, see https://cloud.google.com/docs/authentication/.
This error message is usually thrown when the application is not being authenticated correctly due to several reasons such as missing files, invalid credential paths, incorrect environment variables assignations, among other causes. Keep in mind that when you set an environment variable value in a session, it is reset every time the session is dropped.
Based on this, I suggest you to validate that the credential file and file path are being correctly assigned (avoiding to include blank characters when setting the GOOGLE_APPLICATION_CREDENTIALS env variable), as well as try to explicitly specify your service account file directly into your code by adding the following credentials code to your solution:
from google.oauth2 import service_account
credentials = service_account.Credentials.from_service_account_file('<serviceAccountPath>.json')
texttospeech.TextToSpeechClient(credentials=credentials)
I am trying to convert audio clips [.flac format] to text using the google cloud speech api.I am using the python client library for making requests and have a bucket on the GCP with a flat directory structure which is used to store the audio files and make asynchronous requests.Below is the python code for making requests
from google.cloud import storage
from google.cloud import speech
from google.cloud.speech import enums
from google.cloud.speech import types
import json
def AudioAnalyze(bucket_name):
storage_client = storage.Client()
speech_client = speech.SpeechClient()
bucket = storage_client.get_bucket(bucket_name)
bucket_contents = bucket.list_blobs()
urls = []
audio_content=[]
for i in bucket_contents:
urls.append("gs://"+i.bucket.name+"/"+i.public_url.split("/")[-1])
for gcs_uri in urls:
audio = types.RecognitionAudio(uri=gcs_uri)
config = types.RecognitionConfig(
encoding=enums.RecognitionConfig.AudioEncoding.FLAC,
sample_rate_hertz=44100,
language_code='en-US')
operation = speech_client.long_running_recognize(config,audio)
print(gcs_uri)
print('Waiting for operation to complete...')
response = operation.result(timeout=450)
speech2text=[]
for result in response.results:
for alternative in result.alternatives:
speech2text.append(alternative.transcript)
ad_content=" ".join(speech2text)
audio_content.append(ad_content)
with open("path/to a/json file/file.json","w") as f:
json.dump(audio_content,f)
if __name__=="__main__":
AudioAnalyze("adsaudiocontent")
I am encountering an error: google.api_core.exceptions.NotFound: 404 Requested entity was not found on a particular file present in the bucket while all the other files are being processed properly.The audio files have gone through a similar processing pipeline.The audio files are available on my public bucket:adsaudiocontent so that you can replicate the error.For Additional details here is the traceback.
Traceback (most recent call last):
File "/home/pythonuser1/.virtualenvs/virtual_env/lib/python3.5/site-packages/google/api_core/grpc_helpers.py", line 54, in error_remapped_callable
return callable_(*args, **kwargs)
File "/home/pythonuser1/.virtualenvs/virtual_env/lib/python3.5/site-packages/grpc/_channel.py", line 487, in __call__
return _end_unary_response_blocking(state, call, False, deadline)
File "/home/pythonuser1/.virtualenvs/virtual_env/lib/python3.5/site-packages/grpc/_channel.py", line 437, in _end_unary_response_blocking
raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with (StatusCode.NOT_FOUND, Requested entity was not found.)>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "audioanalyze.py", line 40, in <module>
AudioAnalyze("adsaudiocontent")
File "audioanalyze.py", line 25, in AudioAnalyze
operation = speech_client.long_running_recognize(config,audio)
File "/home/pythonuser1/.virtualenvs/virtual_env/lib/python3.5/site-packages/google/cloud/speech_v1/gapic/speech_client.py", line 264, in long_running_recognize
request, retry=retry, timeout=timeout, metadata=metadata)
File "/home/pythonuser1/.virtualenvs/virtual_env/lib/python3.5/site-packages/google/api_core/gapic_v1/method.py", line 139, in __call__
return wrapped_func(*args, **kwargs)
File "/home/pythonuser1/.virtualenvs/virtual_env/lib/python3.5/site-packages/google/api_core/retry.py", line 260, in retry_wrapped_func
on_error=on_error,
File "/home/pythonuser1/.virtualenvs/virtual_env/lib/python3.5/site-packages/google/api_core/retry.py", line 177, in retry_target
return target()
File "/home/pythonuser1/.virtualenvs/virtual_env/lib/python3.5/site-packages/google/api_core/timeout.py", line 206, in func_with_timeout
return func(*args, **kwargs)
File "/home/pythonuser1/.virtualenvs/virtual_env/lib/python3.5/site-packages/google/api_core/grpc_helpers.py", line 56, in error_remapped_callable
six.raise_from(exceptions.from_grpc_error(exc), exc)
File "<string>", line 3, in raise_from
google.api_core.exceptions.NotFound: 404 Requested entity was not found.
You are accidentally chopping part of the URL erroneously with the "/" matching, which probably resulted in this error.
The bucket name from the class bucket can be used directly.