Guide
Here’s how to create your own API Usability Evaluation:
1. Plan
Sell this concept to your team - You’re going to be spending some time getting this together, so let’s make it count. Do people understand why User Experience is important? Are they willing to make changes based on the feedback you get? Get people on board, and get them excited about the evaluation.
Find some reviewers - You want to get 2-4 people who are close to your customers as possible. But in the end, anyone who isn’t on your team and understands APIs can offer useful feedback. Some tips to finding reviewers: Actively reach out to your developer community by posting an invitation on your developer hub or emailing known constituency groups that you’ve worked with in the past on your programs requirements; visit local meetups such as the DC API meetup to find active and interested citizen developers; or contact the GSA API Usability team who will always be happy to help you recruit.
Get your logistics covered - Spend a lot of time confirming people’s involvement. Use this email template to invite everybody. Try and get as many stakeholders from each API as you can. Create a calendar invite and bring in people as they agree to take part. Send follow ups to everyone two weeks out, one week out, and the day before.
Prep your evaluators - Tell your reviewers to spend some time on the site first, even just five minutes. Tell them their goal is to do a live review of the strengths and weaknesses of the API. Tell them there will be a time crunch, so try to get into specifics as quickly as possible. Here’s a list of API Best Practices for them to refer to, if needed.
Get a room - It needs to hold your team, a reviewer and any observers you want to bring in. Remember, this can be a very powerful demonstration about why customer feedback is so important. The customer data you receive can solve disputes you’re having.
Create a webinar (optional) - If people can’t make it, let them watch virtually. But don’t rely on it - the customer data is really important, and being there in person to hear it is invaluable.
2. Hold the review
Set up the room - Print out the handout and the attached Usability Problem Sheet. Give people pens and paper. Set up a computer for the reviewer with the API open and ready to be used.
Assign roles - You’ll need a few people: * Facilitator - Lead the meeting, explain the rules and keep the meeting moving along * Logistics - Let people into the building, help with tech issues * Notetaker - Take screenshots of your API before the test, and take notes throughout
Lay the ground rules - Tell the API teams and the evaluators that you are doing a usability review of the API. The goal is to educate people about the importance of user experience, to show the entire API team about ways to improve it, and to leave with action items about issues to fix. Tell all the observers to take notes. Thank the reviewer and let them begin.
Let the reviewer talk - Let each reviewer speak without interruption for 30-45 minutes, or however long they can. Encourage them to talk out loud as they look around the API. Have them focus on the API landing page, documentation, and the individual API pages and code.
Write down the problems - Look for problems that seriously prevent people from understanding or using your API. Also pay attention to problems discovered by more than one reviewer - there’s a good chance that’s a big issue.
Q+A When the evaluator is done, and if you have time, ask thme follow up questions, what the main problems were, examples of good APIS, etc.
Debrief - Once the evaluators are done, talk about the problems you heard about. Agree on the big problems. Pick some long hanging fruit and agree to fix them quickly.
3. After the review
Send out a survey - Email everyone involved and thank them, and send out this survey to collect feedback for future evaluations.
Fix problems - Do it!
Schedule another evaluation - You’ll discover more reviews every time, and dtermine whether fixes you’ve made address the problem
Document sucesses - See if your metrics improve as a result of your test. Use the screenshots you took to create a Before and After. Spread the word about the important of this kind of evaluation, and invite more people to the next one!