I have the following permission policy on an IAM Role
statement {
effect = "Allow"
actions = [
"sagemaker:CreateModelPackageGroup",
"sagemaker:ListModelPackageGroups",
]
resources = [
"arn:aws:sagemaker:my_account_region:my_account_id:model-package-group/*",
]
to be able to create a ModelPackageGroup using boto3
import boto3
import logging
logger = logging.getLogger(__name__)
sagemaker_client = boto3.client('sagemaker')
mpg_name = 'my_model_package_name'
matching_mpg = sagemaker_client.list_model_package_groups(NameContains=mpg_name)[
"ModelPackageGroupSummaryList"
]
if matching_mpg:
logger.info(f"Using existing Model Package Group: {mpg_name}")
else:
mpg_input_dict = {
"ModelPackageGroupName": mpg_name,
"ModelPackageGroupDescription": mpg_description,
}
mpg_response = sagemaker_client.create_model_package_group(**mpg_input_dict)
but I got the following error
botocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the ListModelPackageGroups operation: User: arn:aws:sts::my_account_id:assumed-role/my_role_name/botocore-session-some_session_id is not authorized to perform: sagemaker:ListModelPackageGroups because no identity-based policy allows the sagemaker:ListModelPackageGroups action
After doing some research I've found out that the SageMaker Action ListModelPackageGroups requires an All Resources permission
The actions in your policy do not support resource-level permissions and require you to choose All resources
In fact, the Actions defined by Amazon SageMaker Documentation mentions that some SageMaker actions support resource level permission and others must specify all resources
The Resource types column indicates whether each action supports resource-level permissions. If there is no value for this column, you must specify all resources ("*") in the Resource element of your policy statement. If the column includes a resource type, then you can specify an ARN of that type in a statement with that action.
So, changing to the following policy worked out
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Action": [
"sagemaker:ListModelPackageGroups"
],
"Resource": "*"
},
{
"Sid": "",
"Effect": "Allow",
"Action": [
"sagemaker:CreateModelPackageGroup"
],
"Resource": "arn:aws:sagemaker:my_account_region:my_account_id:model-package-group/*"
}
]
}
Related
Problem: get_secret() returns none.
Landscape:
AWS provides a template function which works fine locally when I try it out in the terminal.
What I have done:
I added return json.loads(get_secret_value_response["SecretString"]) so get_secret() returns a value if successful.
I have attached a policy to the secret in secrets manager.
{
"Version" : "2012-10-17",
"Statement" : [ {
"Effect" : "Allow",
"Principal" : {
"AWS" : "arn:aws:iam::__owner_id__:role/aws-elasticbeanstalk-service-role"
},
"Action" : "secretsmanager:GetSecretValue",
"Resource" : "*"
} ]
}
and I have attached a policy to the role aws-elasticbeanstalk-service-role.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "secretsmanager:GetSecretValue",
"Resource": "arn:aws:secretsmanager:eu-west-1:__owner_id__:secret:route/to/scret-123"
}
]
}
aws-elasticbeanstalk-service-role is for EB service itself. You should be using role associated with your EB instance profile. The default role is aws-elasticbeanstalk-ec2-role, but you may have used different role in your setup, so you have to double check that.
I am trying to execute query on Athena using python.
Sample code
client = boto3.client(
'athena',
region_name=region,
aws_access_key_id=AWS_ACCESS_KEY_ID,
aws_secret_access_key=AWS_SECRET_ACCESS_KEY
)
execution = client.start_query_execution(
QueryString=query,
QueryExecutionContext={
'Database': database
},
WorkGroup=workgroup,
ResultConfiguration={
'OutputLocation': S3_OUTPUT_LOCATION
}
)
This is working code, But I got an unusual scenario.
One day it throws an InvalidRequestException error
Error
InvalidRequestException: An error occurred (InvalidRequestException) when calling the StartQueryExecution operation: Unable to verify/create output bucket <BUCKET NAME>
As per the DevOps application have all the permission, It should work.
We try to execute the same query on the AWS Athena console(Query editor). There it is working.
Then we re-run the python script, it is not throwing any error.
But on the next day, the python script start's throwing the same InvalidRequestException error.
Then we execute the same query on the AWS Athena console(Query editor) and re-run the python script, it started working.
We observed this scenario for a few days, Every 24 hours python script throws the error then we execute the query on the Athena console(Query editor) and re-run the python script.
I don't understand why it is happening, is there any permission issue.
Permissions:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"s3:GetObject",
"athena:GetWorkGroup",
"athena:StartQueryExecution",
"athena:ListDatabases",
"athena:StopQueryExecution",
"athena:GetQueryExecution",
"athena:GetQueryResults",
"athena:GetDatabase",
"athena:GetDataCatalog",
"athena:ListQueryExecutions",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::<BUCKET NAME>",
"arn:aws:s3:::<BUCKET NAME>/*",
]
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:ListBucket",
"athena:UpdateWorkGroup",
],
"Resource": [
"arn:aws:s3:::<BUCKET NAME>/*",
"arn:aws:s3:::<BUCKET NAME>",
"arn:aws:athena:*:<BUCKET NAME>/<PATH>",
]
},
{
"Sid": "VisualEditor2",
"Effect": "Allow",
"Action": [
"athena:ListDataCatalogs",
"s3:ListAllMyBuckets"
],
"Resource": "*"
}
]
}
I also faced same error today and found that execution role requires s3:GetBucketLocation permission also, AWS doc: https://aws.amazon.com/premiumsupport/knowledge-center/athena-output-bucket-error/
I was experiencing the same issue - random failures. The issue turned out to be s3:GetBucketLocation policy being configured wrong. It was bundled with the same cluster as other s3 actions where the resource points to the s3 bucket, including path. It does not work this way.
I fixed it as below, works now.
- Effect: Allow
Action:
- s3:GetBucketLocation
Resource:
- arn:aws:s3:::*
- Effect: Allow
Action:
- s3:PutObject
- s3:GetObject
Resource:
- arn:aws:s3:::<BUCKET NAME>/<PATH>/*
See documentation: https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-with-s3-actions.html
I'm creating an API that will ONLY accept requests made from the GitHub Webhook servers by using a Resource Policy with the GitHub IPs. I've successfully done this using the console and manually creating the Resource Policy, but I'm running into a problem when I'm using the CDK.
Here's my code:
delete_trigger_integration = aws_apigateway.LambdaIntegration(
trigger_step_lambda, proxy=False, integration_responses=[])
api_policy_document = aws_iam.PolicyDocument()
api = aws_apigateway.RestApi(
self,
"GithubWebhookApi",
rest_api_name=PROJECT_NAME + "-apigateway-trigger-delete",
default_integration=delete_trigger_integration,
policy=api_policy_document)
delete_execution_resource = api.root.add_resource("execution")
delete_execution_method = delete_execution_resource.add_method(
"POST", delete_trigger_integration)
delete_execution_resource.add_cors_preflight(allow_origins=["*"])
create_repo_lambda.add_environment("API_URL",
delete_execution_resource.url)
api_policy_document.add_statements(
aws_iam.PolicyStatement(
effect=aws_iam.Effect.ALLOW,
principals=[aws_iam.AnyPrincipal()],
actions=["execute-api:Invoke"],
resources=[api.arn_for_execute_api()]))
api_policy_document.add_statements(
aws_iam.PolicyStatement(
effect=aws_iam.Effect.DENY,
actions=["execute-api:Invoke"],
conditions={
"NotIpAddress": {
"aws:SourceIp": [
"192.30.252.0/22", "185.199.108.0/22",
"140.82.112.0/20"
]
}
},
principals=[aws_iam.AnyPrincipal()],
resources=[api.arn_for_execute_api()]))
I feel like I'm very close to having a solution but I can't figure out what it is. The problem with the code above is that I get the ValidationError: Circular dependency between resources error when trying to deploy it - which I can understand, the resource policy is addressing the very resource it's inside. But I can't find a way around it in the CDK.
For other resources it's really easy to ad IAM policies after the creation using aws_iam.add_to_role_policy but I can't find the equivalent for the RestApi class in the CDK. It seems you have to add the PolicyDocument when you declare the RestApi.
Also, here's the resource policy that I'm trying to recreate in the CDK (it's those resource ARNs which are causing the problem):
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "arn:aws:execute-api:eu-west-1:000000000000:aaaaaaaaaa/*/*/*"
},
{
"Effect": "Deny",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "arn:aws:execute-api:eu-west-1:000000000000:aaaaaaaaaa/*/*/*",
"Condition": {
"NotIpAddress": {
"aws:SourceIp": [
"192.30.252.0/22",
"185.199.108.0/22",
"140.82.112.0/20"
]
}
}
}
]
}
Does anyone know a solution to my problem? Thanks in advance!
Also, you can ignore the empty integration responses - I'm still working on that.
This is related to how the underlying CloudFormation resource works.
As the Policy must be defined within the AWS::ApiGateway::RestApi resource, it cannot reference itself.
From the documentation:
To set the ARN for the policy, use the !Join intrinsic function with "" as delimiter and values of "execute-api:/" and "*".
Which translates into the following in your CDK code:
resources=[core.Fn.join('', ['execute-api:/', '*'])]
First time boto3 user.
I had a user with ACL S3FullAccess and used the following code to try and upload a file; it uses a pandas DataFrame as the source.
s3_client = boto3.client('s3')
io = StringIO()
df.to_csv(io)
response = s3_client.put_object(
Bucket=self.bucket,
Body=io,
Key=self.filename
)
This lead to this response
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the PutObject operation: Access Denied
So I checked that the secret key and access key were being picked up by boto3 from my ~/.aws/credentials file, and they are, on line 604 of client.py in boto3 - request_signer=self._request_signer
So I researched here on SO, and it seemed a lot of people had to add a Policy document, so I did that as follows:
{
"Version": "2012-10-17",
"Id": "Policyxxx",
"Statement": [
{
"Sid": "Stmtx1",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<12 digit id>:root"
},
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::<my-bucket>/*"
},
{
"Sid": "Stmtx6",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<12 digit id>:root"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::<my-bucket>"
}
]
}
I still get the same error, so I added this to my put_object call since the S3 bucket uses AES-256 encryption, which I thought was server-side only, but running out of ideas, so worth a try.
SSECustomerKey=os.urandom(32),
SSECustomerAlgorithm='AES256',
Next I removed those terms associated with the SSE keys, realising that the AES-256 encryption is server side and should not affect my access.
Then I tried to generate a new pair of Access keys and use those instead, same result.
Does anyone have any idea what I might look at next, what have I missing in the hundreds of pages of AWS documentation?
This was simply a case of when the user was created they were added to a couple of groups. Administrators and EC2MFA. I was unaware of the implications of this, but assume the EC2MFA group prevented API or CLI access. I am assuming the combination of the Policy on the user and S3 side is sufficiently secure.
I'm trying to download some files I already uploaded to S3 with some Python code, but I'm getting headaches trying to use a tight policy.
I can list all the files in the bucket, but when I try do download them with what I see as a correct policy, I get botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
Then, when I was trying to add a different policy that worked for 2 different buckets, I added part of the bucket's name, then the asterisk, and for some reason, the same exact thing worked.
So can someone tell me why this happens?
This for example, is what works like a charm:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1499955913000",
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::THE-BEGINING-OF-THE-NAME*"
}
]
}
But this doesn't:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1499955913000",
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::THE-EXACT-COMPLETE-FULL-NAME"
}
]
}
I can add the python code for the download if it's relevant, but this questions seems long enough, and the code is pretty straightforward
Seems I just needed some rubber duck debugging, the answer was I think counter intuitive, but easy:
It seems the ARN it's not only an identifier for the AWS resource itself, but also its content. So, when giving permissions, you need to give permissions to "the bucket" for listing it, and "the content" to download it
Which leads to a policy like this:
{
"Version": "2012-10-17",
"Statement": [{
"Sid": "Stmt1499955913000",
"Effect": "Allow",
"Action": ["s3:GetObject", "s3:ListBucket"],
"Resource": [
"arn:aws:s3:::THE-EXACT-COMPLETE-FULL-NAME",
"arn:aws:s3:::THE-EXACT-COMPLETE-FULL-NAME/*"
]
}]
}
Which as I said, gives control over the bucket itself, with no asterisks, and whatever goes after the slash bar.