Right now when I kick off the dataflow it kicks it off as my UserName. Is there a way to run the GCP Dataflow as a service account? If so, can you provide any samples?
Much Appreciated!
To set up service account based authentication:
In the GCP Console, go to the Create service account key page.
Here
From the Service account list, select New service account.
In the Service account name field, enter a name.
From the Role list, select Project > Owner.
Note: The Role field authorizes your service account to access resources. You can view and change this field later by using the GCP Console. If you are developing a production app, specify more granular permissions than Project > Owner. For more information, see granting roles to service accounts.
Click Create.
A JSON file that contains your key downloads to your computer.
Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the file path of the JSON file that contains your service account key. This variable only applies to your current shell session, so if you open a new session, set the variable again.
You can get more information from here : https://cloud.google.com/dataflow/docs/quickstarts/quickstart-python
Related
Goal:
Using python, I want to create a service account in a project on the Google Cloud Platform and grant that service account one role.
Problem:
The docs explain here how to grant a single role to the service account. However, it seems to be only possible by using the Console or the gcloud tool, not with python. The alternative for python is to update the whole IAM policy of the project to grant the role for the single service account and overwrite it (described here). However, overwriting the whole policy seems quite risky because in case of an error the policy of the whole project could be lost. Therefore I want to avoid that.
Question:
I'm creating a service account using the python code provided here in the docs. Is it possible to grant the role already while creating the service account with this code or in any other way?
Creating a service account, creating a service account key, downloading a service account JSON key file, and granting a role are separate steps. There is no single API to create a service account and grant a role at the same time.
Anytime you update a project's IAM bindings is a risk. Google prevents multiple applications from updating IAM at the same time. It is possible to lock everyone (users and services) out of a project by overwriting the policy with no members.
I recommend that you create a test project and develop and debug your code against that project. Use credentials that have no permissions to your other projects. Otherwise use the CLI or Terraform to minimize your risks.
The API is very easy to use provided that you understand the API, IAM bindings, and JSON data structures.
As mentioned in John’s answer, you should be very careful when manipulating the IAM module, if something goes wrong it could end in services completely inoperable.
Here is a Google’s document which manipulates the IAM resources using the REST API.
The owner role can be granted to a user, serviceAccount, or a group that is part of an organization. For example, group#myownpersonaldomain.com could be added as an owner to a project in the myownpersonaldomain.com organization, but not the examplepetstore.com organization.
Im not sure how I can usemy service account and Dialogflow API key to get an API token using OAuth2 to get an access token for google. Could someone share a code snippet
I want to access this API: https://cloud.google.com/dialogflow/es/docs/reference/rest/v2/projects.agent.entityTypes/list
In order to use the Dialogflow API (in this case, list all the entity types in the specified agent) you have to:
Enable the Dialogflow API for your project
Go to the Create service account key page. From the Service account list, select New service account, enter a name in the Service account name field.
From the Role list, select Project > Owner.
Click Create. A JSON file that contains your key downloads to your computer.
Once you have the JSON file, you need to set an environment variable in a shell session GOOGLE_APPLICATION_CREDENTIALS="/path/to/key.json".
You can access the API via REST, gRPC or a client library. In this repository, there are some samples with the python client library.
I've been using GCP for some time.
I created a new GCP Project to test new functions, and enabled the Runtime API.
However, clicking Deploy pops up this message:
"User does not have the 'iam.serviceAccounts.actAs' permission on webcalc-taskc2#appspot.gserviceaccount.com required to create the function. You can fix this by running gcloud iam service-accounts add-iam-policy-binding webcalc-taskc2#appspot.gserviceaccount.com --member=user: --role=roles/iam.serviceAccountUser"
I entered this into Shell, replacing with my Owner email for GCP. Still no luck ;(
You have to add permissions that can use service account to your account.
According to Google document, A service account is a special type of Google account intended to represent a non-human user that needs to authenticate and be authorized to access data in Google APIs.
Seems that "webcalc-taskc2#appspot.gserviceaccount.com" is your service account runs cloud function.
The pop up message you received said that a user account running cloud function does not have 'iam.serviceAccounts.actAs' permission on that service account.
So you have to entered command in the message with replaced --member=user to --member=user:Your account email.
You can check your cloud function service account at details -> General information tab.
On a server firestore admin is accessed using python:
import firebase_admin
from firebase_admin import credentials, firestore
cred = credentials.Certificate('myfile.json')
myfile.json has been added to a git repo so now I'm wondering if there is a way to revoke it and create a new one?
Following firebase tutorials it's supposed to be a service account, however there is no old service account on google cloud to revoke.
myfile.json was created months ago so there is no way to know how it was created (as when doing anything with firebase/google cloud you have to jump around to multiple guides all over the place, out of the blue knowing all kinds of things to make any progress whatsoever).
Edit:
The reason I didn't find the service account was because google cloud had selected a project that had a similar name as the firebase project.
If you look at the contents of myfile.json, notice three fields: project_id, client_email and private_key_id
These fields tell you the Project ID that the credentials created in, the name of the service account (client_email) and the Key ID.
With that information, you can go to the Google Cloud Console. Log in and select your Project (if you manage more than one Project). Go to IAM -> Service Accounts and find the service account by the client_email. Then disable or delete the service account.
There are more advanced things that you can do such as disable a key and create another one (key rotation), but since the Service Account Key JSON file has been leaked, I would delete the service account and create a new one.
I have developed a Google Cloud Function (GCF) in python, which i want to access from a web service deployed on AWS (written in python). While in the development phase of the GCF, It had Cloud Function Invoker permission set to allUsers. I assume that is why it didn't ask for an Authorization Token when called.
I want to revoke this public access and make it so that i can only call this function from the web service code and it is not accessible public-ally.
Possible Approach :In my research i have found out that this can be done using the following steps:
Removing all the unnecessary members who have permissions to the GCF.
Creating a new service account which has restricted access to only use GCF.
Download the service account key (json) and use it in the AWS web application
Set environment variable GOOGLE_APPLICATION_CREDENTIALS equal to the path of that service account key (json) file.
Questions
How to generate the Access token using the service account, which may then be appended as Authorization Bearer within the HTTP call made to the GCF? Without this token the GCF should throw error.
The docs say not to put the service account key in the source code. Then what is the best way to go about it. They suggest to use KMS which seems like an overkill.
Do not embed secrets related to authentication in source code, such as API keys, OAuth tokens, and service account credentials. You can use an environment variable pointing to credentials outside of the application's source code, such as Cloud Key Management Service.
What are the bare minimum permissions i will require for the service account?
Please feel free to correct me if you think my understanding is wrong and there is a better and preferable way to do it.
UPDATE: The web service on AWS will call the GCF in a server-to-server fashion. There is no need to propagate the client-end (end-user) credentials.
In your description, you don't mention who/what will call your GCF. A user? A Compute? Another GCF? However, this page can help you to find code example
Yes, secret in plain text and pushed on GIT is not yet a secret! Here again, I don't know what performing the call. If it's a compute, functions, cloud run, or any service of GCP, don't use JSON file key, but the component identity. I would say, create a service account and set it to this component. Tell me more on where are you deploying if you want more help!
Related to 2: if you have a service account, what the minimal role: cloudfunctions.Invoker. It's the minimal role to invoke function
gcloud beta functions add-iam-policy-binding RECEIVING_FUNCTION \
--member='serviceAccount:CALLING_FUNCTION_IDENTITY' \
--role='roles/cloudfunctions.invoker'