I have the following script:
import boto3
# Get the service resource
sqs = boto3.resource('sqs')
# Get the queue
queue = sqs.get_queue_by_name(QueueName='')
for message in queue.receive_messages():
print(message.body)
# Let the queue know that the message is processed
message.delete()
It returns the following as the message's body:
{"Records":[{"eventVersion":"2.0","eventSource":"aws:s3","awsRegion":"us-west-2","eventTime":"2017-03-03T11:06:25.329Z","eventName":"ObjectCreated:Copy","userIdentity":{"principalId":"AWS:<id>:<lambda_name>"},"requestParameters":{"sourceIPAddress":"54.186.104.49"},"responseElements":{"x-amz-request-id":"8577BEEB91F199BF","x-amz-id-2":"<>="},"s3":{"s3SchemaVersion":"1.0","configurationId":"PutFromSisterBucket","bucket":{"name":"<bucket_name>","ownerIdentity":{"principalId":"<>"},"arn":"arn:aws:s3:::<bucket_nmae>"},"object":{"key":"<object_key>","size":1990,"versionId":"anHi0ukirRiApp4jyoSTz2oVGOejR6tJ","sequencer":"0058B94E3141A83718"}}}]}
How do I get the value of the "key" inside the "object"?
Currently, the entire result is a string. Is there any way I can do without string indexing or regex match?
If the variable message is string you need to load it with json.loads, to parse to dict type in python, so you can use it like a json:
import json
for message in queue.receive_messages():
message_dict = json.loads(message)
record = message_dict["Records"][0]
content = record["s3"]["object"]["key"]
print(content)
Related
When we try to publish the json to Azure Event hub, EventData converting the JSON string. Instead of JSON string I need the JSON only at the consumer end.
response = {"status":"Active", "code":400}
event_batch = await producer.create_batch()
event_batch.add(EventData(json.dumps(response)))
At the consumer end I am getting the event as { body: '{"status":"Active", "code":400}', 'sequence_numbe':1} like this. how can we get only Json at consumer end like this
{ body: {"status":"Active", "code":400}, 'sequence_numbe':1}
Can anyone help?
You could use body_as_json method to achieve your requirement.
Sample Implementation :
jsonbody = event.body_as_json(encoding='UTF-8')
This will directly return us the JSON Object
Alternate Approach :
Get it as string and then convert the same to json object.
jsonbody = json.loads(event.body_as_str(encoding='UTF-8')))
The fetch method gives this error:
imaplib.IMAP4.error: FETCH command error: BAD [b'Could not parse command']
I am not attaching all of my code. I want to get the unseen msg using imap to get the body and save it as text and then download the attachment.
import imaplib, email, os
user= "test9101997"
password="Monday#123"
imap_url="imap.gmail.com"
attach_dir='E:\PROJECT\attachment'
filePath='D:\ATTACH'
con=imaplib.IMAP4_SSL(imap_url)
con.login(user,password)
con.select('INBOX')
#UIDs=con.search(None,'UNSEEN')
#print(UIDs)
(result, messages) = con.search(None, 'UnSeen')
if result == "OK":
for message in messages:
try:
ret, data =con.fetch(message,'(RFC822)')
except:
print ("No new emails to read.")
#self.close_connection()
#exit()
#result, data=con.fetch(i,'(RFC822)')
raw=email.message_from_bytes(data[0][1])
I think you may be confused about the return value of con.search(). If you take a look at the value of messages after that call (assuming that result is OK), it's collection of strings, not a list of message ids. That is, after a call like:
result, messages = con.search(None, 'UnSeen')
The value of messages may look something like:
['1 2 15 20']
So when you try to iterate over it like this:
for message in messages:
The value of message in the first loop iteration will be 1 2 15 20, and that's why you're getting the command error: the request you're making doesn't make any sense. You'll want to do something like this instead:
(result, blocks) = con.search(None, 'UnSeen')
if result == "OK":
for messages in blocks:
for message in messages.split():
ret, data = con.fetch(message, '(RFC822)')
raw = email.message_from_bytes(data[0][1])
There's really no good reason for the imaplib module to return data in this fashion.
I am new to python and I am still learning it.
I am writing a python code to send a message to AWS SQS
Following is the code I have written
from boto3.session import Session
session = Session(aws_access_key_id='**', aws_secret_access_key='**',
region_name='us-west-2')
clientz = session.client('sqs')
queue = clientz.get_queue_url(QueueName='queue_name')
print queue responses = queue.send_message(MessageBody='Test')
print(response.get('MessageId'))
On running this code it returns
{u'QueueUrl': 'https://us-west-2.queue.amazonaws.com/##/queue_name', ' ResponseMetadata': {'HTTPStatusCode': 200, 'RequestId': '##'}}
Traceback (most recent call last):
File "publisher_dropbox.py", line 77, in
responses = queue.send_message(MessageBody='Test')
AttributeError: 'dict' object has no attribute 'send_message'
I am not sure what the 'dict' object is, since I haven't specified that anywhere.
I think you mix up the boto3 client send_mesasge Boto3 client send_message with the boto3.resource.sqs ability.
First, for boto3.client.sqs.send_message, you need to specify QueueUrl. Secondly, the the error message appear because you write incorrect print statement.
# print() function think anything follow by the "queue" are some dictionary attributes
print queue responses = queue.send_message(MessageBody='Test')
In addition, I don't need to use boto3.session unless I need to explicitly define alternate profile or access other than setup inside aws credential files.
import boto3
sqs = boto3.client('sqs')
queue = sqs.get_queue_url(QueueName='queue_name')
# get_queue_url will return a dict e.g.
# {'QueueUrl':'......'}
# You cannot mix dict and string in print. Use the handy string formatter
# will fix the problem
print "Queue info : {}".format(queue)
responses = sqs.send_message(QueueUrl= queue['QueueUrl'], MessageBody='Test')
# send_message() response will return dictionary
print "Message send response : {} ".format(response)
I have email message with an unwanted attachment (a PKCS-7 signature in this particular case). I can detect the signature in the email with this piece of code:
payloads = mail.get_payload()
for index in xrange(len(payloads)):
if payloads[index].get_content_type() == "application/pkcs7-signature":
print("Found PKCS-7 Signature", index)
How would I remove this particular payload from the message? The email.message API seems to only have methods for reading and writing whole payloads: get_payload() and set_payload(). Neither of these allow specifying payload index of what to read or write.
One possible solution:
def remove_signature(mail):
payload = mail.get_payload()
if isinstance(payload, list):
for part in payload:
if part.get_content_type().startswith('application/pkcs7-signature'):
payload.remove(part)
return mail
I am using boto library to read messages from SQS queue. My messages have text like this:
{ Command:XXXXXXXXXXX Key:XXXXXXX Input:XXXXXX} . Boto sends with base64 encoded and also reads it, so that if I read the message body then the text is there.
But how can I read the message like
Command = input['Command']
Key = input_message['Key'].split(',')
so that I can use those values for further processing...
I am quite new to Python also
Ok, you seem to have the input in some kind of a format - is it anything standardised? If not, you would need to parse the contents of your message and get the individual keys.
What I have been doing before in my projects was using JSON to facilitate data exchange between platforms.
If you do not have a luxury to edit your incoming data, you would need to do something like this (very naiive example):
input = "{ Command:XXXXXXXXXXX Key:XXXXXXX Input:XXXXXX }"
data = filter(lambda x: ":" in x, input.split())
message_dict = dict()
for item in data:
key, val = item.split(":")
message_dict[key] = val
Consider using good old fashioned JSON to easily send and receive dictionaries acrost the wire.
This test function verifies that the data format is very clear with JSON:
test_sqs.py
import json
import boto3
from moto import mock_sqs
#mock_sqs
def test_sqs():
sqs = boto3.resource('sqs', 'us-east-1')
queue = sqs.create_queue(QueueName='votes')
queue.send_message(MessageBody=json.dumps(
{'Command': 'drink', 'Key': 'beer', 'Input': 'tasty'}))
messages = queue.receive_messages()
assert len(messages) == 1
assert messages[0].body == (
'{"Input": "tasty", "Command": "drink", "Key": "beer"}')