CSCW: Protocol for Sakai Widget

Artifact: CSCW

This artifact was a protocol for testing for CSCW taught by Jim Laffey, Ph.D.

Purpose of Protocol

The primary purpose of the test is to observe how users interact with the Sakai widget prototype.

Testing Location

Participant's office.

Testing Procedure

A laptop is running with the Sakai widget prototype on the desktop. The test coordinator has a notepad for taking notes during the one-on-one interview.

Script

Thank you for agreeing to test the Sakai widget today. The Sakai widget is a new online course awareness tool that we are testing at the University of Missouri. First I will ask you some general questions about yourself. Then I will show you the widget. I will observe you using the widget and then ask you some questions about your experience. Your feedback will help us to improve the widget so it is a valuable tool.

Background Information Collection

  1. What is your name?
  2. Are you an undergrad or grad student?
  3. What is your degree program?
  4. What type of computer do you normally use, PC or Mac?
  5. How would you describe yourself as a computer user: beginner, intermediate or expert?
  6. Do you currently use any widgets? Which ones?
  7. Have you ever taken any online courses before this semester? If so, which ones? Which course management system did you use (Blackboard, Sakai, etc.)?
  8. How many hours a week do you use a computer?

Widget Introduction

Spend a moment reviewing the widget and then we will ask you to try a task on the prototype for evaluation. Imagine today is a normal day that you are working on the computer and doing your usual activities. Take a moment to check the widget for updates. Use the areas on the widget that you would find most relevant for your current course.

Interview Questions

Usefulness Questions

  1. What was the most useful part of the widget for you?
  2. What was the most useless part of the widget for you?
  3. What was the most important part of the widget for you?
  4. How often do you think you would check the widget during a normal day?
  5. How would the widget be more convenient for you than logging into Sakai using a browser?
  6. If you could change the widget any way that you wanted, how would you change it?
  7. Is there anything you would like to add about the usefulness of the widget that has not been discussed?

Usability Questions

  1. What was easy for you to find on the widget?
  2. What was difficult for you to find on the widget?
  3. Were the links easy to identify?
  4. Did the text labels make sense? If not, which areas were unclear?
  5. Was the most important content placed high on the widget, or did you have to search for it?
  6. Was the text content easy to read and/or scan?
  7. Were the widget's colors in the graphics, text and background visually pleasing?
  8. Were the font types and sizes attractive?
  9. With a focus on the course portions of the widget (discussion board and our workspace), what information did you expect to see and then find on the widget?
  10. With a focus on the chat portions of the widget, what information did you expect or hope to see but did not find on the widget?
  11. With a focus on the social networking portions of the widget, what information did you expect or hope to see but did not find on the widget?
  12. Is there anything you would like to add about using the widget that has not been discussed?
back to top