Learning About User Expectations — The Hard Way

So, you ever thought you had something pretty well figured out, only to have it turn around and bite you in the butt with very sharp teeth?

That happened to me last Friday, when I sent out the approximately one billionth email action alert I’ve had the privilege to distribute to an activist list, in this case for NET. What went wrong? Well, historically we’ve presented our activists (about 17,000 — a relatively small list) with a pre-written message in our grassroots action system, so that all they have to do is push a button to attach their name to it and send it to the relevant decision-maker(s). Of course, we’ve encouraged our folks to re-write the message in their own words, but almost nobody does (fewer than 10% of people do, according to vendors I’ve spoken with before).

So, on Friday we were rushing to get an alert out and at the same time experimenting with asking people to write their own message based on some talking points we provided, which is supposed to be more effective than sending in identical mass emails. Our mistake? Well, we made two, the first a technological one: in the blank on the web page in which people were supposed to compose their message, I left the words “[insert your message here]” to help keep them from getting confused.

Big mistake — our grassroots module won’t let someone send a completely empty message, but it will let them send a note with only a small bit of text, for instance maybe “[insert your message here].” Oh, my — you can imagine the results. Within minutes, we’d received roughly 200 emails whose only text besides a greeting and a signature was, of course, “[insert your message here].” Nice! And we’d have never known that anything was going wrong if we hadn’t happened to cc: a staff member with copies of each note, so that she could collect them and present them in person at a meeting next week (the messages were aimed at a regulatory agency, and we were emailing them in right away but also planning to bring the copies all at once in a big stack).

The technological mistake is obvious — I didn’t know enough about how the grassroots advocacy system worked and made a goof. Our second mistake is more subtle — we changed the behavior we expected from our activists without giving them enough warning. Turns out, activist groups like ours have trained people to send advocacy messages without a second thought (I doubt many of them even read what they’re signing). So, when our folks got our alert last week, many of them clicked the “send” button without even scrolling down to see what the message said, just as they always do.

One thing did go right, though: we reacted quickly enough to salvage the alert. Within a couple of minutes of realizing what was wrong, I jumped into the landing page and stripped out that evil little snippet of text, keeping people from just clicking and running. We also sent out an immediate message to our list explaining what had gone wrong and asking people to be patient with us. Since then, I’ve answered a dozen or so personal messages from list members who wrote in to complain, explaining what happened and why we were asking them to do something different than our normal practice. In the end, several hundred people DID take the time to compose a unique message and we were able to mollify the annoyed list members who wrote us.

What’ll we do next time? If we’re going to ask people to compose a unique message, we’re going to make it clear at the beginning of our email and the top of the action page what’s up. We’re also make sure that activists know that WE know that it’s more trouble for them to compose text, that we appreciate their work and that we understand if they don’t have the time to write. Loyalty is fragile, and we want to do everything we can to preserve it. Keeping reader expectations in mind is a good start.

cpd

Written by
Colin Delany
View all articles
2 comments
  • Turns out, activist groups like ours have trained people to send advocacy messages without a second thought (I doubt many of them even read what they’re signing). So, when our folks got our alert last week, many of them clicked the “sendâ€? button without even scrolling down to see what the message said, just as they always do.

    This is one of the biggest concerns that I’ve heard from people working on the hill who receive our messages – that when so many messages come in saying the same thing, they are unsure people know, and really support what they are signing.

    Without meaning to, this does give a little insite into the number of people this is true of. It must also be said, however, that this doesn’t mean that these people didn’t read the message sent to them and beleive 100% in the action, and in taking action.

    And of course the big take away is, as the author says, we have to be very careful when we are changing the experience for our readers.

    It’s a bummer that it happened, but I do appreciate you sharing it. We definately all need the reminder!

  • We also sent out an immediate message to our list explaining what had gone wrong and asking people to be patient with us.

    I’m somewhat surprised that sent another email to your list – why did you decide to do this? Did you have to direct them to another petition page?

    No, it was simply courtesy — as soon as we got rid of that little piece of default text, people were going to start getting an error message when they hit the button as they were used to doing. This way, they at least had some warning that things were different. It definitely seems to have helped — we got several thank-yous and, it was also a good starting point for people who DID write in to complain. I’d definitely recommend in the future that if you screw something up like that, let the list know! They’re adults and will usually understand the problem and appreciate the communication. — cpd