Bulk Email -- Requirements

Overview
Course Student Requirements
Mail to Course Students -- Authoring
Mail to Course Students -- Sending
Mail from Course Students -- Receiving
Marketing Mails Requirements
Mail to Marketing Lists -- Authoring [P1]
Mail to Marketing Lists -- Sending [P1]
Mail from Marketing Lists -- Receiving [P1]
Operation Requirements
Non-Requirements
Open Issues

Overview
The high-level feature is enabling professors (and their staff) to send rich email to the students
in their class. Weve found that frequent mails is one of the best ways to keep students
engaged. Other platforms support this, and so profs have come to expect that they can email
everyone.
Class2Go supported this feature, so in many ways this first cut is porting over the feature that
we had implemented there. See screenshot from Class2Go showing that interface, for
reference.

This document covers two distinct use cases. The first one, and most important, is for the
course staff to mail the students directly. Thats the P0. Its also valuable to send a marketing
email to an arbitrary list of students. Thats a P1 feature. It will eventually share much of the
same plumbing, but has differences in authoring, sending, and mail format, so those
requirements are called out separately in this doc. Marketing emails will come in a subsequent
release.
All requirements are P0 unless noted otherwise.

Course Student Requirements

Mail to Course Students -- Authoring
1. As an instructor I want to compose an email to send to the students in my class.
I want to use a rich-text editor so it looks good in basic HTML.
2. [P1] As an instructor I want to be able to save a draft of the message without
sending.
3. As an instructor I want the option to copy/paste in HTML for my mail message so
I can compose it somewhere else.
4. As an instructor I want the email I author to be able to contain unicode
characters.
5. [P1] As an instructor I want course email to be customizable by a small number
of static fields. We will treat the mail as a template (mako or django template) and will
render the template to personalize the mail.
$student_name -- student full name, John Smith
$student_email -- johnsmith@gmail.com
$student_id -- johnsmith
$student_token -- 234820ab492b41, the token we use for
qualtrics
$course_institution -- Berkeley (or in Stanfords use, Education)
$course_number -- EDU101N
$course_name -- Math Learning
6. [P1] As an instructor I should be able to specify the from name and from
address manually. The site should verify that the from address is from the approveddomains list (see technical requirements below) and give a helpful error message if the
from address isnt in that list (mail has to be sent from one of these domains:
class.stanford.edu, etc...).
7. [P1] As an instructor I would like to have a default from address for the course,
either a mailing list (eg. cs144-staff@class.stanford.edu) or a no-reply address (eg.
do-not-reply@class.stanford.edu).
8. [P1] As an instructor, if I specify unicode characters in my name or email address
then I want that converted to ascii (how?)

Mail to Course Students -- Sending

9. As an instructor I want to be able initiate sending my message to a selection of

recipients
myself
course staff
all students + staff
10. As an instructor, when initiating a send, I should be warned that this could take
some time for the mail to go out, depending on other jobs in the system and the size of
the class. You should only get this warning when sending to all students (ie. not myself
or course staff)

Mail from Course Students -- Receiving

11. As a student with a modern mail client I want to get my mail in a pretty HTML
format.
12. As a student with a bare-bones mail client I want to get mail that has been
flattened into plain HTML so I can still read it. How to test? What rudimentary mail
clients understand MIME multipart, but not HTML?
13. As a student I want to optout of receiving emails from a course by going to my
dashboard and clicking on the Email Settings link for the course, and then unchecking
the receive email option that appears in the modal display, and click Save Settings.
14. As a student I want opt-outs to be per course. Opting out of one course email
doesnt imply opting out of another

Marketing Mails Requirements

Mail to Marketing Lists -- Authoring [P1]
15. [P1] As course operations, if I author a mail and the human-readable tag I choose
isnt unique, I want to get a validation error but I dont want to lose my work.

Mail to Marketing Lists -- Sending [P1]

16. [P1] As course operations I want to author a mail that will go to a mail list. I
should be able to identify it via a useful human readable tag.
17. [P1] As course operations, if I try to upload to a mailing list and the list name I
provide isnt unique then I want to get a nice error message saying so, and I dont want
the original mail list disturbed.
18. [P1] As course operations I want a backend command to upload a CSV file that
will create a new mail list. The incoming mail format should be name,address.
19. [P1] As course operations I want the mail list I upload to support unicode
characters in both the name field and the email address (see RFC-4952)
20. [P1] As course operations I want to create very large email lists, upwards of one

million entries. Creating a list that large should take minutes, not hours.
21. [P1] As course operations I want a UI to initiate sending a mail to a mail list.
(needs more detail)
22. [P1] As a student, if I opt out of one marketing email, I want that to be respected
for subsequent marketing emails. Not course opt-outs though, those are per-course and
non-global.

Mail from Marketing Lists -- Receiving [P1]

23. As a student I want to have a mail link on the bottom of the page that will opt me
out of mail for this course. Clicking on the link at the bottom of the mail should take me
to an unsubscribe page.
If I havent opted out previously, then that page that tells me that I
am about to unsubscribe, and gives me an opportunity to confirm or proceed with
unsubscribe. Giving this option will reduce errant optouts.
If I have opted out previously, then I can undo my optout here.
That page cannot require authentication.
This page can only be accessible by a URL that cant be easily
guessed, since it is unauthenticated.
24. As course operations I want some way to undo an errant optout. Sometimes
people unsubscribe by mistake and then they send us mail asking us to re-instate them.
Can just be a documented method, doesnt have to be fancy.

Operation Requirements
25. There should be a configurable list (per-site) with the list of approved domains
that you can send mail from. Verifying a domain is sufficiently heavyweight (dkim
records, etc.) that this will be done manually. This list is for verification.
26. [P1] As OpenEdX technical operations Id like to be notified if we get failures from
SES due to hitting thresholds, either instantaneous rate limits or daily limits.
27. As OpenEdX technical operations, celery workers cant be clogged up by mail
send failures.
The retry logic has to handle the case where we have other jobs
queued up, but mail cannot be processed (connectivity issues to SES, hitting
thresholds). So failed jobs need to be requeued instead of just blocking the
worker in a retry loop.
An alternate way to satisfy this requirement is to isolate the mail
queue so that only mail failures are handled on that queue. That way, if there is a
blocked queue, it will affect all mails, but only mails. This is OK since we only
need to retry global mail failures (cant send **any** mails) and not per-email
failures. For example, if an individual mail is invalid, that should just fail, not retry
28. [P1] If we treat the mail as a template then we need to take care that theres
nothing the author can do with the templating langugage that would cause security or
availability problems.

Non-Requirements
Difference Between This and Forum Notifications
The forum team is building a notification system that uses mailgun as its MTA. This seems like
duplicated efforts. But as I discussed with Jim it seems like these are two distinct sets of use
cases:
We need to handle scale -- tens of thousands of recipients will be very common,
up to a million for the marketing mail case.
We are sensitive to cost -- Amazons SES is cheaper than the negotiated mailgun
rate Ive heard. At big scale this matters! Amazon retail SES rate is $0.10 per 1000
mails. Our monthly spend last month was $100, mostly driven by the bulk mail to our
800,000 address list, but there were also a normal bunch of course emails.
The open source community doesnt want to go negotiate another vendor
relationship. Theyre already buying something from Amazon. This is a big deal for
adoption.
Forum Notifications can focus on isolation, for example by storing the mail to be
sent in the payload of the celery message itself. At scale we dont want that many big
messages, so we need to store the message body itself in the database.

Specifically Off the Table

These are things that we dont plan on doing for the initial version or even with the P1s. These
would be considered follow-on projects.
1. Status of mail sends. That would be nice later, but difficult to implement without
more complicated queue monitoring
2. Handling bounces and complaints. Amazon SES reports those back to us via
separate SNS channels, but we dont gather those notices up and reconcile them those
back to the sender.
3. We do **not** need to keep a record of what messages were sent to whom aside
from normal logging. Audit trail.
4. Injection of beacons to track (and report on) delivery and open rates.
Commercial mailers do this, we wont.

Open Issues
1. International Content -- Exactly how to handle Unicode in all fields: from name,
from address, to address, subject, and mime multipart body.

Implementation
Student UI