Develop Reference

How to specify Twilio Whatsapp template from the python client?

I would like to use Twilio Whatsapp messages with custom templates. I'm trying to send messages using a Python client. We have different types of messages and we would like to manage them from Twilio backend. We created several templates, but I have no idea how to specify a template in the code.
In the official documentation there is no mention of this. I looked into the source code of the client, here is a code from class MessageList, that I use for message creation:
def create(self, to, status_callback=values.unset, application_sid=values.unset,
max_price=values.unset, provide_feedback=values.unset,
attempt=values.unset, validity_period=values.unset,
force_delivery=values.unset, content_retention=values.unset,
address_retention=values.unset, smart_encoded=values.unset,
persistent_action=values.unset, from_=values.unset,
messaging_service_sid=values.unset, body=values.unset,
media_url=values.unset):
There is nothing similar to the template name in function parameters.
Is it possible at all to specify the template programmatically?

Twilio developer evangelist here.
When you want to send a message using a template, all you have to do is ensure that the body of the message you are sending matches the template. For example, if your templates is:
Your login code for {{1}} is {{2}}.
Then sending the message:
Your login code for Twilio is 12345.
matches that template and will send successfully. So you don't specify the template programmatically, just with your message content.
You can see more about sending template messages with the Twilio API for WhatsApp here.

from twilio.rest import Client
client = Client() # this is the Twilio sandbox testing number
from_whatsapp_number = 'whatsapp:+14155238886'
to_whatsapp_number = 'whatsapp:+15005550006'
client.messages.create(body='Ahoy,world!', from_=from_whatsapp_number, to=to_whatsapp_number)

Implementing Google Directory API users watch with Python

I'm having some trouble understanding and implementing the Google Directory API's users watch function and push notification system (https://developers.google.com/admin-sdk/reports/v1/guides/push#creating-notification-channels) in my Python GAE app. What I'm trying to achieve is that any user (admin) who uses my app would be able to watch user changes within his own domain.
I've verified the domain I want to use for notifications and implemented the watch request as follows:
directoryauthdecorator = OAuth2Decorator(
approval_prompt='force',
client_id='my_client_id',
client_secret='my_client_secret',
callback_path='/oauth2callback',
scope=['https://www.googleapis.com/auth/admin.directory.user'])
class PushNotifications(webapp.RequestHandler):
#directoryauthdecorator.oauth_required
def get(self):
auth_http = directoryauthdecorator.http()
service = build("admin", "directory_v1", http=auth_http)
uu_id=str(uuid.uuid4())
param={}
param['customer']='my_customer'
param['event']='add'
param['body']={'type':'web_hook','id':uu_id,'address':'https://my-domain.com/pushNotifications'}
watchUsers = service.users().watch(**param).execute()
application = webapp.WSGIApplication(
[
('/pushNotifications',PushNotifications),
(directoryauthdecorator.callback_path, directoryauthdecorator.callback_handler())],
debug=True)
Now, the receiving part is what I don't understand. When I add a user on my domain and check the app's request logs I see some activity, but there's no usable data. How should I approach this part?
Any help would be appreciated. Thanks.

The problem
It seems like there's been some confusion in implementing the handler. Your handler actually sets up the notifications channel by sending a POST request to the Reports API endpoint. As the docs say:
To set up a notification channel for messages about changes to a particular resource, send a POST request to the watch method for the resource.
source
You should only need to send this request one time to set up the channel, and the "address" parameter should be the URL on your app that will receive the notifications.
Also, it's not clear what is happening with the following code:
param={}
param['customer']='my_customer'
param['event']='add'
Are you just breaking the code in order to post it here? Or is it actually written that way in the file? You should actually preserve, as much as possible, the code that your app is running so that we can reason about it.
The solution
It seems from the docs you linked - in the "Receiving Notifications" section, that you should have code inside the "address" specified to receive notifications that will inspect the POST request body and headers on the notification push request, and then do something with that data (like store it in BigQuery or send an email to the admin, etc.)

Managed to figure it out. In the App Engine logs I noticed that each time I make a change, which is being 'watched', on my domain I get a POST request from Google's API, but with a 302 code. I discovered that this was due to the fact I had login: required configured in my app.yaml for the script, which was handling the requests and the POST request was being redirected to the login page, instead of the processing script.

django send_mail() function taking several minutes

I'm trying to send emails in a function within my views.py file. I've set up the email in my settings file in the same manner as here.
Python Django Gmail SMTP setup
Email sending does work but it takes several minutes to occur which my users have been complaining about. I am receiving a gethostbyaddress error in my var/log/mail.log file which I'll post here. I used to get nginx timeout errors but put "proxy_read_timeout 150;" into my /etc/nginx/sites-enabled/django file.
http://www.nginxtips.com/upstream-timed-out-110-connection-timed-out-while-reading-response-header-from-upstream/
This solved the timeout errors when interacting with the website but the emails still take several minutes to load. I'm using a digitalocean django droplet and this slow speed has occured on all my droplets.
Here's my view function
#login_required
def AnnouncementPostView(request, leaguepk):
league = League.objects.get(pk=leaguepk)
lblog = league.blog
if request.method == 'POST':
form = AnnouncementPostForm(request.POST)
if form.is_valid():
posttext = request.POST['text']
newAnnouncement = Announcement(text=posttext, poster=request.user)
newAnnouncement.save()
lblog.announce.add(newAnnouncement)
titleText = "%s Announcement" % (league.name)
send_mail(titleText, posttext, settings.EMAIL_HOST_USER, ['mytestemail#gmail.com'], fail_silently=False)
return HttpResponseRedirect(reverse('league-view', args=[league.pk]))
else:
form = AnnouncementPostForm()
return render(request, 'simposting/announcementpost.html', {'form': form, 'league': league})
This has worked, the announcement is posted to the desired page and is even emailed, it's just a time problem, people have come to expect nearly instant emailing processes which makes the 2-3 minutes unacceptable, especially when signing up also causes the 2-3 minute wait.
One issue may be the fact that while trying to solve this issue with the DigitalOcean support team I changed my droplet name and the hostname to be the domain that I set up.
My current hostname and droplet name is mydomain.com. I have it setup that way in my /etc/hostname file. My /etc/hosts file looks like this
127.0.0.1 localhost.localdomain localhost mydomain.com
127.0.1.1 mydomain.com
My var/log/mail.log file responds with this whenever I try to send mail
Oct 6 16:13:24 "oldDropletName" sm-mta[13660]: gethostbyaddr(10.xxx.xx.x) failed: 1
Oct 6 16:13:24 "oldDropletName" sm-mta[13662]: starting daemon (8.14.4): SMTP+queueing#00:10:00
I hope this is enough information to help, it's been troubling for several weeks and usually I can either solve my problems by looking up stuff here or working with the support team but it's got us stumped. Thank you for taking the time to help!

Sending an email is a network bound task and you don't know how long it will take to finish exactly like in your case. Although there might be a latency in your network but it's better to do such task in an async fashion so your main thread is free.
I am using the following code in one my projects.
utils.py
import threading
from django.core.mail import EmailMessage
class EmailThread(threading.Thread):
def __init__(self, subject, html_content, recipient_list, sender):
self.subject = subject
self.recipient_list = recipient_list
self.html_content = html_content
self.sender = sender
threading.Thread.__init__(self)
def run(self):
msg = EmailMessage(self.subject, self.html_content, self.sender, self.recipient_list)
msg.content_subtype = 'html'
msg.send()
def send_html_mail(subject, html_content, recipient_list, sender):
EmailThread(subject, html_content, recipient_list, sender).start()
just call send_html_mail from your view.

I am not particularly familiar with sendmail (I use postfix) but I would suspect this is almost certainly related to something with sendmail and probably not Django. The second log entry has "SMTP+queueing#00:10:00". and this link would indicate that sendmail takes a flag on startup to determine how often to process the mail queue. You may want to look around your init or wherever your startup scripts are and see how sendmail is configured. Also, if you are using Gmail you really can't control any delays on their end, so along with determining the configuration of your mail server, you'll need to check logs for when actions are actually occurring such as the mail being queued/sent. Is the time that line shows in your log from when the view was executed? If so, it is in the hands of sendmail.

Making the stack levels in Django HTML email reports collapsable

Django has an awesome debug page that shows up whenever there's an exception in the code. That page shows all levels in the stack compactly, and you can expand any level you're interested in. It shows only in debug mode.
Django also has an awesome feature for sending email reports when errors are encountered on the production server by actual users. These reports have a stacktrace with much less information than the styled debug page.
There's an awesome optional setting 'include_html': True that makes the email include all the information of the debug page, which is really useful. The problem with this setting is that the HTML comes apparently unstyled, so all those levels of the stack are expanded to show all the data they contain.
This results in such a long email that GMail usually can't even display it without sending you to a dedicated view. But the real problem is that it's too big to navigate in and find the stack level you want.
What I want: I want Django to send that detailed stacktrace, but I want the levels of the stack to be collapsable just like in the debug page. How can I do that?
(And no, I don't want to use Sentry.)

We use a middleware to process any exceptions that happen in production. The basic idea is save the debug html to a location on the file system that can be served via a password protected view and then send a link to the generated view to whoever needs it.
Here is a basic implementation:
from django.conf import settings
from django.core.mail import send_mail
from django.views import debug
import traceback
import hashlib
import sys
class ExceptionMiddleware(object):
def process_exception(self, request, exception):
if isinstance(exception, Http404):
return
traceback = traceback.format_exc()
traceback_hash = hashlib.md5(traceback).hexdigest()
traceback_name = '%s.html' % traceback_hash
traceback_path = os.path.join(settings.EXCEPTIONS_DIR, traceback_name)
reporter = debug.ExceptionReporter(request, *sys.exc_info())
with open(traceback_path, 'w') as f:
f.write(reporter.get_traceback_html().encode("utf-8"))
send_mail(
'Error at %s' % request.path,
request.build_absolute_uri(reverse('exception', args=(traceback_name, ))),
FROM_EMAIL,
TO_EMAIL,
)
And the view
from django.conf import settings
from django.http import HttpResponse
def exception(request, traceback_name):
traceback_path = os.path.join(settings.EXCEPTIONS_DIR, traceback_name)
with open(traceback_path, 'r') as f:
response = HttpResponse(f.read())
return response
The full middleware is tailored to our needs but the basics are there. You should probably password protect the view somehow. Unless you return a response Django's own error handling will kick in and still send you an email but I'll leave that to you.

The tracebacks in the debug page are stackable because they are done in javascript. I doubt you can have what you want since HTML emails should not accept javascript.

As J. C. Leitão pointed out, the django error debug page has javascript and css(most css doesn't work in email). But all these css and js code are inline. The debug page is a single html file that has no external resources.
In my company, we include the html as a attachment in the report email. When we feel the plain text traceback is not clear enough, we download the html page and open it. The user experience is not as good as Sentry, but much better than the plain-text only version.

What is the first step to getting in e-mail into my python / flask app code?

I am researching what it would take to make a web app that would interact with e-mails directly. Like you would send to something#myapp.com and the app would tear it apart and determine who it's from, if they are in the DB, what is the subject line, etc.
I am working with/most familiar with python and flask.
Could anyone get me started in the right direction of how to get an e-mail to interface with my flask app code?

There are several approach you can take:
write some code which uses IMAP or POP to retrieve emails and process them. Either run this from a crontab (or something similar) or add it to your flask app and trigger it in there, either through a crontab that requests a magic URL or setting up a custom timer thread.
configure your MTA to deliver email for something#myapp.com by feeding it to a program you write (for example in Exim you could use a pipe transport) . In that program you can either process it directly, or do something like POSTing it to your flask app.

I've done something along these lines recently, with a simple bookmarking web-app. I have the usual bookmarklet way of bookmarking something to it, but I also wanted to be able to e-mail links to it from apps like Reeder on my iPhone and whatever. You can see what I ended up with on GitHub: subMarks.
I use Google Apps for your Domain for my email, so I created a special address for my app to look at - I really didn't want to try building/configuring my own e-mail server.
the mail_daemon.py file from above is run as a cron job every 5 minutes. It connects to the email server using the poplib Python package, processes the emails that are there and then disconnects (one part I feel compelled to point out is that I check that the emails are from me before they are processed :) )
My Flask app then provides the front end to the bookmarks, displaying them from the database.
I decided not to put the email handling code into the actual flask app, because it can be rather slow and would only run when the page was visited, but you could do this if you wanted.
Here's some barebones code to get things going:
import poplib
from email import parser
from email.header import decode_header
import os
import sys
pop_conn = poplib.POP3_SSL('pop.example.com')
pop_conn.user('my-app#example.com')
pop_conn.pass_('password')
#Get messages from server:
messages = [pop_conn.retr(i) for i in range(1, len(pop_conn.list()[1]) + 1)]
# Concat message pieces:
messages = ["\n".join(mssg[1]) for mssg in messages]
#Parse message into an email object:
messages = [parser.Parser().parsestr(mssg) for mssg in messages]
for message in messages:
# check message is from a safe recipient
if 'me#example.com' in message['from']:
# Get the message body text
if message['Content-Type'][:4] == 'text':
text = message.get_payload() #plain text messages only have one payload
else:
text = message.get_payload()[0].get_payload() #HTML messages have more payloads
# decode the subject (odd symbols cause it to be encoded sometimes)
subject = decode_header(message['subject'])[0]
if subject[1]:
bookmark_title = subject[0].decode(subject[1]).encode('ascii', 'ignore') # icky
else:
bookmark_title = subject[0]
# in my system, you can use google's address+tag#gmail.com feature to specifiy
# where something goes, a useful feature.
project = message['to'].split('#')[0].split('+')
### Do something here with the message ###
pop_conn.quit()