Fork me on GitHub

API Usability Testing

Your agency logo

Templates

Archives

Template Archives

Email Invitation to Agencies to Participate

We’re been putting together a pilot project that combines usability testing and APIs and are scheduling a session on March 4. For this, we are looking for three agencies who have multiple APIs and who would like to get feedback on them and their developer resources from three outside experts. The event would be from 9-12, with time at the end for the three agencies to compare notes and ideas with each other, too. Each agency could send one person or bring their whole team. I think your operation would find this substantial and enriching. Any interest in this?

Email Invitation to an Outside Developer to Participate

Hey,

We’re prototyping a form of API feedback later this month and I wanted to see if you were free to join in.

Here’s what it’d be like - We take three agencies who have at least a few APIs and over the course of a morning, ask three outside developers to give them feedback. We’d schedule it such that we’d only need an hour of your time and the attention would be entirely on what you had to say. Thoughout your session, we’d have lined up for you the developer hubs of each of the three agencies and would ask you to go through each, spending 10-12 minutes per agency, giving general or specific feedback, sharing what you would change, what you find helpful, and what you find inconvenient or confusing. The agency’s API producers would be present but would be there solely to hear your thoughts. If there was time or if it’d be helpful, we’d have prompts on hand to ask you to walk through a workflow and respond specifically to it, though you could also just preach truth to the agencies the entire time instead. :) We have a number of agencies who are eager to engage and hear your thoughts - so many that we have a full waiting list.

A few notes - * I’m coordinating this event and would ensure that it is set up to be an efficient use of your time.
* After each of the outside experts have given their feedback, we will have a followup discussion where agencies share with each other what they learned and what they would suggest for each other. During this, I’ll be requesting each agency to choose 2 short- and 2 medium-term changes that they plan to implement as a result of this feedback so you can be sure that your feedback is going to good use.
* You could do this in person here at GSA or over Google Hangout from your office.

Our next session is from 9-12 on [insert date], but we’d only need you for one hour in that time. This should be a great opportunity to rub off on some of the agencies who are best in a position to do justice to your feedback. Would you be game?

Email to Agency Participants - One Week Before

Hi folks,

We’re really looking forward the upcoming feedback sessions this Thursday. We’re going to be in Room [Room Number] at GSA HQ* and will be going from 9am-12pm, so please plan to arrive before 9 so we can start on time. At security, you can skip the metal detectors and show them your government ID to get in. If you have any trouble, have them call my cell (below).

The schedule is below and we’ll look to hold tightly to it. We’ve got a great lineup of developers, though, and we think that you’ll find this really useful.

Most of all, we want to give you a healthy swath of fresh outside feedback and developer insight.

Each of these three developers are familiar with the government space and have built mashups consuming APIs before. Each developer will come in for a session of 45 minutes and in that time, will go through each of the agencies for about 10 minutes, then engage in Q&A for the last part. You will receive feedback at three levels - the developer hub, individual API documentation, and the API endpoint itself. We’ll be encouraging them to speak aloud as they navigate your developer resources so that you can have plenty of raw feedback.

Each reviewer is advanced enough not to need further direction and I’d ask you to mostly just observe so that we can best replicate what outside developers experience. If they ask a question, you can answer it but try to be more brief. I’ll lovingly encourage reviewers and agencies both to stay on track. :)

If you desire feedback on a particular API or developer resource, let me know and I’ll ask them to prioritize that but otherwise, they’ll be encouraged to give feedback on whatever is catching their attention.

I’m really looking forward to this and want to thank you again for being an early adopter of this initiative. I’m hoping that the feedback each agency is getting will be of use to you and your team. One of our goals for this session is to identify two (or more) easily obtainable improvements to your pages/APIs, and we’ll follow up with you a couple weeks after the session to check up on the progress. We’re definitely glad that you’ll be representing.

Please reach out any time with questions and feel free to call me anytime at [Your Number].

Thanks!

[Your Name]

Developer prompts

Pull up agency developer hub and look at it from the perspective of a potential developer. Begin at the developer hub level, then - as time permits, drill down to one or more individual API pages, and, as you see fit, possibly the actual endpoint and call returns themselves.

Email to Outside Experts - One Week Before

Hey folks!

Thank you again; we’re really looking forward to having your insight on [day of the week]. A bit of quick background: One thing to note is that this is very much a prototype event. We’re trying something new here, so as we go through this, please feel free to suggest improvements to the process as well. The goals are to maximize your opportunities to give feedback and suggestions to these three agencies.

We’re going to be in Room [Room Number] at GSA HQ [Link address]. * The schedule is below and we’ll look to hold tightly to it.

We have three teams who will be in the room, each reasonably mature so please feel free to give them solid and straightforward feedback. The agencies: [List agencies, linking to the developer hub each time]. Please plan to arrive (or dial in) 10 minutes before your session is scheduled, so you can go through security. Call as you arrive and we can meet you on the other side of security and escort you up.

Your session is 45 minutes, allowing 10-15 minutes of feedback for each agency. There are three levels of potential feedback, along the lines of the prompts below, I’m including links for the areas of focus but you should feel free to give any and all feedback that you see fit. In fact, we’d like you to keep up whatever stream of consciousness you’re willing to, in order to offer more raw perspective and to share an outsider’s perspective. With such short timeframes, I’ve told the agencies that we only have time for the you to give their input, so to observe only. If you have a question, they’re to keep their answer brief, but feel free to ask whatever you need to in order to give them the best feedback you can.

I’m really looking forward to this and want to thank you again for being an early adopter. An outline of potential feedback is below as well as links for [day of the week]. We’ll have a laptop with chrome for you to use to drive, so you only need to bring yourself. If you prefer to use your own laptop and have a VGA adaptor, that’s cool, too.

Please reach out any time with questions and feel free to call me over the weekend at [your number].

Thanks.

[Your Name]

p.s. Please also note that all of this feedback is off of the record and solely intended for constructive feedback, so you should speak your mind and not hold any punches. :)

–

Developer prompt.

Pull up agency developer hub and look at it from the perspective of a potential developer. Begin at the developer hub level, then - as time permits, drill down to one or more individual API pages, and, as you see fit, possibly the actual endpoint and call returns themselves.

– Links

[Include Links to the Developer portal and/or API pages]

Scheduler for Outside Experts - One Week Before

Thanks for being part of the API UX test! If you can arrive maybe 5-10 minutes early to give time for security, that’d be great. You can reach me at [cell number] when you get here and I’ll serve as your escort to the testing room. [Your names here], and the agency representatives look forward to seeing you!

Day Before Reminder

Hello all:

We are excited to see everyone tomorrow and have a great API UX session at [Location]. If you have any security or location problems, please call me at [phone number]. If you can, please plan to arrive five or ten minutes before your scheduled start time (9 a.m. for agencies, varied for developers) so we can make sure we’re efficient and respectful of everyone’s time.

As a quick reminder, the schedule looks like:

I think that’s it! See you all tomorrow.

Best,

[Names]

A Collaboration of 18F and DigitalGov UX Program.