I am trying to upload the image to S3 and then have AWS Rekognition fetch it from S3 for face detection, but Rekognition cannot do that.
Here is my code - uploading and then detecting:
import boto3
s3 = boto3.client('s3')
s3.put_object(
ACL='public-read',
Body=open('/Users/1111/Desktop/kitten800300/kitten.jpeg', 'rb'),
Bucket='mobo2apps',
Key='kitten_img.jpeg'
)
rekognition = boto3.client('rekognition')
response = rekognition.detect_faces(
Image={
'S3Object': {
'Bucket': 'mobo2apps',
'Name': 'kitten_img.jpeg',
}
}
)
this produces an error:
Unable to get object metadata from S3. Check object key, region and/or access permissions.
Why is that?
About the permissions: I am authorized with AWS root access keys, so I have full access to all resources.
Here are the few things that you can do:
Make sure the region of the S3 bucket is the same as Recognition. Otherwise, it won't work. S3 service is global but every bucket is created in a specific region. The same region should be used by AWS clients.
Make sure the access keys of the user or role have the right set of permissions for the resource.
Make sure the file is actually uploaded.
Make sure there is no bucket policy applied that revokes access.
You can enable logging on your S3 bucket to see errors.
Make sure the bucket is not versioned. If versioned, specify the object version.
Make sure the object has the correct set of ACLs defined.
If the object is encrypted, make sure you have permission to use that KMS key to decrypt the object.
You have to wait for a while that the image uploading is done.
The code looks running smoothly, so your jpeg starts to upload and even before the uploading is finished, Rekognition starts to detect the face from the image. Since the uploading is not finished when the code runs, it cannot find the object from your S3. Put a wait time a bit.
Related
I have tested almost every example code I can find on the Internet for Amazon Textract and I cant get it to work. I can upload and download a file to S3 from my Python client so the credentials should be OK. Lots of the errors points to some region failure but I have try every possible combinations.
Here are one of the last test call -
def test_parse_3():
# Document
s3BucketName = "xx-xxxx-xx"
documentName = "xxxx.jpg"
# Amazon Textract client
textract = boto3.client('textract')
# Call Amazon Textract
response = textract.detect_document_text(
Document={
'S3Object': {
'Bucket': s3BucketName,
'Name': documentName
}
})
print(response)
seems to be pretty easy but it generates the error -
botocore.errorfactory.InvalidS3ObjectException: An error occurred (InvalidS3ObjectException) when calling the DetectDocumentText operation: Unable to get object metadata from S3. Check object key, region and/or access permissions.
Any ideas whats wrong and dose someone have a working example (I knew the tabs are not correct in the example code)?
I have also tested a lot of permission settings in AWS. The credentials are in a hidden files created by aws sdk.
I am sure you already know, but the bucket is case sensitive. If you have verified that both the object bucket and name are correct, just make sure to add the appropriate region to your credentials.
I tested just reading from s3 without including the region in the credentials and I was able to list the objects in the bucket with no issues. I am thinking this worked because s3 is supposed to be region agnostic. However, since Textract is region specific, you must define the region in your credentials when using Textract to get the data from the s3 bucket.
I realize this was asked a few months ago, but I am hoping this sheds some light to others that face this issue in the future.
I try to build Face-Rekognition using AWS. So, i want to upload file to S3 Bucket only and it uploads from S3 to Collection auto using Python. Pls help me, thank you!
You can create an Amazon S3 Event that triggers an AWS Lambda function when a new object is uploaded to an Amazon S3 bucket.
The AWS Lambda function can make an index_faces() call to AWS Rekognition to detect faces in the input image and add them to the specified collection.
You should first create the Face Collection where the faces will be added. It is also useful to provide an ExternalImageID with each image so that the face can be associated with an identifier within your application. This ID will then be returned with future face-search results.
I need to upload a csv diretly to AWS S3 with public access.
My current code is:
import boto3
s3 = boto3.resource('s3', aws_access_key_id='xxx', aws_secret_access_key='yyy')
s3.Bucket('test1234542').upload_file('C:/Users/output1.csv', 'output1.csv')
Unfortunately the permission is private and i dont know who to change to code to upload it with public access directly.
At the moment i have to go manually to the bucket, click on the folder "Permissions",click on "Public Access" and then make a tick at "Read object".
Does someone know a python code to add the public access permission?
Best
Michi
When you upload a file in AWS S3, by default its private.
In order to give public access to the file, first change the public access options of the bucket to allow the changes made to its objects.
change public access settings of the bucket
code to upload file with public access :
import boto3
s3 = boto3.resource('s3')
s3.Bucket('bucket_name').upload_file('data.csv', 'data.csv')
file_object = s3.Bucket('bucket_name').Object('data.csv')
print(file_object.Acl().put(ACL = 'public-read'))
Hope It helps !!
You could use this to make the S3 bucket public. Make sure your EC2 instance has the correct instance-profile attached and it has the correct permissions to interact with S3( Possible s3 full access). Use the following link and let me know if it helps!
S3 Bucket Access Public
Provide the ACL argument in your put_object statement and the uploaded object will be public. Refer to this documentation.
client.put_object(
ACL='public-read',
Body='file that is to be uploaded',
Bucket='name of the bucket',
Key='obejct key'
)
So I am trying to load a csv file from s3 bucket. The following is the code
import pandas as pd
import boto3
import io
s3_file_key = 'iris.csv'
bucket = 'data'
s3 = boto3.client('s3')
obj = s3.get_object(Bucket=bucket, Key=s3_file_key)
initial_df = pd.read_csv(io.BytesIO(obj['Body'].read()))
It works fine. iris.csv is only 3kb in size.
Now instead of iris.csv, I try to read 'mydata.csv' which is 6gb in size.
I get the following error :
ClientError: An error occurred (AccessDenied) when calling the GetObject operation: Access Denied
I am unable to comprehend how access can be an issue since I put the data there in the first place. Also I am able to read 'iris.csv' from the same location. Any ideas?
Here are the few things that you can do:
Make sure the region of the S3 bucket is the same as your AWS configure. Otherwise, it won't work. S3 service is global but every bucket is created in a specific region. The same region should be used by AWS clients.
Make sure the access keys for the resource has the right set of permissions.
Make sure the file is actually uploaded.
Make sure there is no bucket policy applied that revokes access.
You can enable logging on your S3 bucket to see errors.
Make sure the bucket is not versioned. If versioned, specify the object version.
Make sure the object has the correct set of ACLs defined.
If the object is encrypted, make sure you have permission to use that KMS key to decrypt the object.
Can we access bucket with bucket endpoint like .s3.amazonaws.com using python sdk. i don't want access bucket with following bucket = conn.get_bucket(bucket_name).
I don't know why you need to access it this way because the s3 endpoint is a fixed part where only thing changes is the name of your bucket (because it's global).
But, in the end, what you are looking for is not possible unfortunately. You need to provide bucket name for accessing the bucket and running operations on it.
Verified by boto3 documentation and here you can check:
S3 Boto documentation