Develop Reference

Python and mailchimp integration

So i have been stuck on this for a week or so and could really do this some advice.
I have a basic website which allows people to sign up to an automated newsletter that i want to send out. After adding their email on the site their address is automatically added to an audience.
I have a small python script running that then web scrapes a site and then if that returns a certain criteria it will send an automated email out via mailchimp to all the contacts in the mailchimp audience.
What i am having issue with is actually creating and sending out the email via mail chimp.
I have been through https://mailchimp.com/developer/api/marketing/campaigns/add-campaign/ a few times and can't seem to get it working. I am able to create a new campaign succesfully as well as the audience and subject line. I am unable to workout how to actually send the email with the content i want inside it however. It just saves itself as a draft and thats it.
When i try to preview the email there is nothing in it and when i click edit next to the campaign everything is ticked except for the 'content.
I have excluded th web scraping part of my program but below is the test i am running to create and sent out via the mailchimp api
import mailchimp_marketing as MailchimpMarketing
from mailchimp_marketing.api_client import ApiClientError
from mailchimp3 import MailChimp
data = {
"recipients" :
{
"list_id": 'XXXXXXXX'
},
"settings":
{
"subject_line": 'Subject of email',
"from_name": 'from_name',
"reply_to": 'reply_email',
},
"type": "regular"
}
try:
client = MailchimpMarketing.Client()
#print(client)
client.set_config({
"api_key": "XXXXXXXXXXXXXXXX",
"server": "XXXXXXX"
})
#client = MailChimp(mc_api='XXXXXXXXXXXX', mc_user="XXXXXXXXX")
client.campaigns.create(data)
response = client.campaigns.get('campaign_id')
#client.campaigns.send()
print(response)
except ApiClientError as error:
print("Error: {}".format(error.text))
This succesfully creates the campaign except without the content that i want to add and simply saves the email as a draft without send. So i guess my question is how to i edit the email content and then how do i actually initiate the send.
Thanks for any help

I also didn't find a minimal example on the web. Plus the examples in the mailchimp api documentation are severely lacking for python (only curl seems correct). Here's a minimal example:
from mailchimp_marketing import Client, api_client
# find this out at https://mailchimp.com/en/help/about-api-keys/
API_KEY = '…'
# log into your Mailchimp account and look at the URL in your browser.
# You’ll see something like https://us19.admin.mailchimp.com/
# the us19 part is the server prefix.
SERVER_PREFIX = 'us19'
try:
client = Client()
client.set_config({
"api_key": API_KEY,
"server": SERVER_PREFIX
})
# find out list id: https://mailchimp.com/en/help/find-audience-id/
campaign_data = dict(
type='regular',
recipients=dict(list_id='…'),
settings=dict(
subject_line='lorem ipsum',
from_name='John Doe',
reply_to='john#doe.com',
)
)
campaign = client.campaigns.create(campaign_data)
print(campaign)
campaign_id = campaign['id']
content_data = dict(
plain_text='lorem ipsum',
html='<p><strong>lorem</strong><br />ipsum</p>'
)
response = client.campaigns.set_content(campaign_id, content_data)
print(response)
response = client.campaigns.send(campaign_id)
print(response)
except api_client.ApiClientError as e:
print("Error: {}".format(error.text))

Send push notification to many users via python

For my project using Firebase messaging to send push notification. I have users's firebase tokens stored on the database. Using them I sent push to each user. Total time of sending is about 100 seconds for 100 users. Is there way to send push asynchronously(I mean at one time to send many push notifications)
# Code works synchronously
for user in users:
message = messaging.Message(
notification=messaging.Notification(
title="Push title",
body="Push body"
),
token = user['fcmToken']
)
response = messaging.send(message)

Sure, you could use one of the python concurrency libraries. Here's one option:
from concurrent.futures import ThreadPoolExecutor, wait, ALL_COMPLETED
def send_message(user):
message = messaging.Message(
notification=messaging.Notification(
title="Push title",
body="Push body"),
token = user['fcmToken'])
return messaging.send(message)
with ThreadPoolExecutor(max_workers=10) as executor: # may want to try more workers
future_list = []
for u in users:
future_list.append(executor.submit(send_message, u))
wait(future_list, return_when=ALL_COMPLETED)
# note: we must use the returned self to get the test count
print([future.result() for future in future_list])

If you want to send the same message to all tokens, you can use a single API call with a multicast message. The Github repo has this sample of sending a multicast message in Python:
def send_multicast():
# [START send_multicast]
# Create a list containing up to 500 registration tokens.
# These registration tokens come from the client FCM SDKs.
registration_tokens = [
'YOUR_REGISTRATION_TOKEN_1',
# ...
'YOUR_REGISTRATION_TOKEN_N',
]
message = messaging.MulticastMessage(
data={'score': '850', 'time': '2:45'},
tokens=registration_tokens,
)
response = messaging.send_multicast(message)
# See the BatchResponse reference documentation
# for the contents of response.
print('{0} messages were sent successfully'.format(response.success_count))
# [END send_multicast]

Google Contacts API: Temporary internal error, when uploading contact photos in parallel

I need to change the contact photo for a large number of contacts, using the python client for the Google Contacts API 3.0
gdata==2.0.18
The code I'm running is:
client = gdata.contacts.client.ContactsClient(source=MY_APP_NAME)
GDClientAuth(client, MY_AUTH)
def _get_valid_contact(contact_id):
contact = client.GetContact(contact_id)
if contact.GetPhotoLink() is None:
# Generate a proper photo link for this contact
link = gdata.contacts.data.ContactLink()
link.etag = '*'
link.href = generate_photo_url(contact)
link.rel = 'http://schemas.google.com/contacts/2008/rel#photo'
link.type = 'image/*'
contact.link.append(link)
return contact
def upload_photo(contact_id, image_path, image_type, image_size):
contact = _get_valid_contact(contact_id)
try:
client.ChangePhoto(media=image_path,
contact_entry_or_url=contact,
content_type=image_type,
content_length=image_size)
except gdata.client.RequestError as req:
if req.status == 412:
#handle etag mismatches, etc...
pass
Given a list of valid Google contact ids, if I run the upload_photo method sequentially for each of them, everything goes smoothly, and all the contacts get their photo changed:
for contact_id in CONTACT_ID_LIST:
upload_photo(contact_id, '/path/to/image', 'image/png', 1234)
However, if I try to upload the photos in parallel (using at least 4 threads), some of them shall randomly fail with 500, A temporary internal problem has occurred. Try again later as a response to the client.ChangePhoto call. I can retry these photos later, though, and they finally get updated:
from multiprocessing.pool import ThreadPool
pool = ThreadPool(4)
for contact_id in CONTACT_ID_LIST:
pool.apply_async(func=upload_photo,
args=(contact_id,'/path/to/image', 'image/png', 1234))
The more threads I use, the more frequently the error happens.
The only similar issue I could find is http://code.google.com/a/google.com/p/apps-api-issues/issues/detail?id=2507, and it was solved some time ago.
The issue I'm facing now might be different, as it happens randomly, and only when running the updates in parallel. So there are chances there might be a race condition at some point at the Google Contacts API end.

How do I return the Instagram Realtime subscription challenge?

I'm trying to subscribe to a tag. It appears that the callback URL is being called correctly with a hub.challenge and hub.mode, and I figured out how to access the challenge using self.request.get('hub.challenge'). I thought I was just supposed to echo the challenge, but that doesn't appear to work since I receive the following errors in the GAE logs:
InstagramAPIError: (400) APISubscriptionError-Challenge verification failed. Sent "647bf6dbed31465093ee970577ce1b72", received "
647bf6dbed31465093ee970577ce1b72
".
Here is the full handler:
class InstagramHandler(BaseHandler):
def get(self):
def process_tag_update(update):
update = update
mode = self.request.get('hub.mode')
challenge = self.request.get('hub.challenge')
verify_token = self.request.get('hub.verify_token')
if challenge:
template_values = {'challenge':challenge}
path = os.path.join(os.path.dirname(__file__), '../templates/instagram.html')
html = template.render(path, template_values)
self.response.out.write(html)
else:
reactor = subscriptions.SubscriptionsReactor()
reactor.register_callback(subscriptions.SubscriptionType.TAG, process_tag_update)
x_hub_signature = self.request.headers.get('X-Hub-Signature')
raw_response = self.request.data
try:
reactor.process('INSTAGRAM_SECRET', raw_response, x_hub_signature)
except subscriptions.SubscriptionVerifyError:
logging.error('Instagram signature mismatch')

So returning it as a string worked. I should have payed closer attention to the error message, but it took a helpful person on the Python IRC to point out the extra line breaks in the message. Once I put the template files on one line, it seemed to work. I can now confirm that my app is authorized via Instagram's list subscription URL.

Script to move messages from one IMAP server to another

Our office uses 2 IMAP servers for e-mail, one is the incoming server and holds the recent e-mails and the other is an archive server. We mainly use Outlook 2010 and our current process is to periodically drag sent messages from the incoming server to the archive.
Today I was asked into looking into writing a script and that would periodically (probably using crontab) grab all sent messages and move them to archive.
I've looked into some example of SSL or telnet to access the server and poke around. However I don't know the best way to script this or how to move files cross server within the IMAP environment.
What's the best way to accomplish this? I'd prefer to use Python just from comfort level, but if there is already an existing solution in another language, I could deal with it.
Update:
Ok, here's some code. Currently It copies the messages just fine, however, it will duplicate exisiting messages on the archive server.
import imaplib
import sys
#copy from
f_server = 'some.secret.ip.address'
f_username = 'j#example.com'
f_password = 'password'
f_box_name = 'Sent Messages'
#copy to
t_server = 'archive.server.i.p'
t_username = 'username'
t_password = 'password'
t_box_name = 'test'
To = imaplib.IMAP4(t_server)
To.login(t_username, t_password)
print 'Logged into mail server'
From = imaplib.IMAP4(f_server)
From.login(f_username, f_password)
print 'Logged into archive'
From.select(f_box_name) #open box which will have its contents copied
print 'Fetching messages...'
typ, data = From.search(None, 'ALL') #get all messages in the box
msgs = data[0].split()
sys.stdout.write(" ".join(['Copying', str(len(msgs)), 'messages']))
for num in msgs: #iterate over each messages id number
typ, data = From.fetch(num, '(RFC822)')
sys.stdout.write('.')
To.append(t_box_name, None, None, data[0][1]) #add a copy of the message to the archive box specified above
sys.stdout.write('\n')
try:
From.close()
From.logout()
try:
To.close()
To.logout()
Some sources:
Doug Hellman's Blog: imaplib - IMAP4 Client Library
Tyler Lesmann's Blog: Copying IMAP Mailboxes with Python and imaplib
I still need to:
delete/expunge messages on the live server
not copy duplicates (actually this would be fixed by deleting originals after copying, but...)
error trapping
Update 2:
Anyone have any ideas on how to not create duplicates when copying? (excluding the option of deleting originals, for now) I thought about searching text, but realized nested replies could throw that off.

Here's what I ended up using. I don't claim that it's perfect, the way it uses msg_num and not id is a little risky. But this is fairly low volume moves, maybe a couple an hour (on cron).
import imaplib
#copy from
from_server = {'server': '1.1.1.1',
'username': 'j#example.com',
'password': 'pass',
'box_names': ['Sent', 'Sent Messages']}
#copy to
to_server = {'server': '2.2.2.2',
'username': 'archive',
'password': 'password',
'box_name': 'Sent'}
def connect_server(server):
conn = imaplib.IMAP4(server['server'])
conn.login(server['username'], server['password'])
print 'Logged into mail server # %s' % server['server']
return conn
def disconnect_server(server_conn):
out = server_conn.logout()
if __name__ == '__main__':
From = connect_server(from_server)
To = connect_server(to_server)
for box in from_server['box_names']:
box_select = From.select(box, readonly = False) #open box which will have its contents copied
print 'Fetching messages from \'%s\'...' % box
resp, items = From.search(None, 'ALL') #get all messages in the box
msg_nums = items[0].split()
print '%s messages to archive' % len(msg_nums)
for msg_num in msg_nums:
resp, data = From.fetch(msg_num, "(FLAGS INTERNALDATE BODY.PEEK[])") # get email
message = data[0][1]
flags = imaplib.ParseFlags(data[0][0]) # get flags
flag_str = " ".join(flags)
date = imaplib.Time2Internaldate(imaplib.Internaldate2tuple(data[0][0])) #get date
copy_result = To.append(to_server['box_name'], flag_str, date, message) # copy to archive
if copy_result[0] == 'OK':
del_msg = From.store(msg_num, '+FLAGS', '\\Deleted') # mark for deletion
ex = From.expunge() # delete marked
print 'expunge status: %s' % ex[0]
if not ex[1][0]: # result can be ['OK', [None]] if no messages need to be deleted
print 'expunge count: 0'
else:
print 'expunge count: %s' % len(ex[1])
disconnect_server(From)
disconnect_server(To)

I'm not sure what volume of messages you're dealing with, but you could extract the Message-ID from each one and use that to find out if it's a duplicate. Either generate a list of IDs already on the target server each time you prepare to move messages, or add them to a simple database as they are archived.
You could narrow things down by an additional message property like Date if the lookups are too expensive, then drop the older lists when you no longer need them.

Presumably too late to be helpful to the OP, but hopefully useful for anyone following along after now.
This looks like a generic requirement. You probably shouldn't be custom coding anything.
You would probably be better off using an MTA configured to send copies of everything to an archive as well as sending stuff to your IMAP server. If this is hard for you to set up, consider using a third party service, who would manage your archives, and forward mail on to your existing mail server.
If you really do want to do this by copying from IMAP, I'd suggest looking at offlineimap.
If you really do want to do it yourself, the way to track the messages you've already seen is by using the Message-ID header.