Printer-friendly version

Project Guide

Overview

The focus of this course is a semester-long project in user interface design, implementation and evaluation project. You will do the project in groups of 4 - 5 students. The project will be completed in several stages with weekly milestones. Be sure to keep up with the work, and frequently review work scheduled for upcoming weeks. Each project group will present and demonstrate its project at a special event held during the last week of class. Visitors will be invited to see the presentations.

Forming Project Groups

Project groups should consist of four or five students. Groups of five are particularly encouraged. User interface design requires an exchange of ideas, and many of the evaluation mechanisms require several people. We will include activities during the first few class sessions to help people meet project partners and select projects. It is OK to have specialties within the group: for example, 1 - 2 people might take the lead on implementing the prototype, whereas 1 - 2 others might take more responsibility for written reports. However, all members must participate in all facets of the project (except that non-programmers may focus on items other than programming), and must be prepared to discuss any aspect of an assignment at a weekly TA meeting.

Groups must be formed by September 17^th and must have defined a project. Students without groups on that date will be assigned to groups.

Special Note to Non-Programmers: Non-CSci students without a programming background are admitted into this course because they bring a different perspective to human interface design (e.g., from psychology, rhetoric, education, architecture, etc.). Non-programmers can and should be full partners on their teams. This can involve taking added responsibilities in usability testing during the time that the programmers spend extra effort on implementation.

Choosing a Project

The goal of the project is to learn how to design, evaluate, and implement user interfaces. It is the process and the user interface, not the underlying application itself, which are being evaluated. For this reason, an excellent project might tie together (or wrap around) existing applications with poor interfaces. Some good projects also select a particularly ineffective part of an existing application and redesign and implement just that part. (This is a common approach for dealing with existing web systems.) Writing a complete, detailed application for the purpose of this project is a bad idea.

The application domain or problem shall be chosen by the group, but with the following key requirements:

1. The problem must be primarily an interface problem. This is not the place to focus on computation or database solutions for which the interface is well-known or obvious.
2. The problem must be one for which your group can find at least two real users who are willing to work with your group. At the minimum, these users must agree to spend time with you at the start of the project to educate you about their activities and to spend time with you later in the semester for user testing. In many cases, users will be more flexible and provide valuable intermediate feedback on your designs. Users need not be (and indeed ought not to be) special experts; the only qualification is that they actually do the task your project will help support and that they not be members of your group.
3. The problem must be one that is suitable for the task-centered design approach we will be following in the course. That rules out most applications where users interact without specific goals or tasks to accomplish. For example, a purely decorative screen-saver would not fit. Game-related applications should be reviewed carefully. The course staff will help you determine whether a project is likely to work well, and we can often help you identify an aspect of a domain that interests you that would be suitable.

You may implement the project in the tool/environment of your choice. Successful implementations usually involve either high-level prototyping tools (e.g., Visual Basic, web development tools) or toolkits with which group members are already familiar (e.g., Java Swing). You should be careful not to let "tool time" dominate your efforts at the expense of usability engineering.

Grading

The project is the focus of this class and represents the majority of your grade in the class. The grade for the project is divided into many pieces, all of which will count toward the total of 65%. A small part of the grade (about 10% of the project total) will be allocated to individual grades for specific components (heuristic evaluation, individual notes and logs). The project overall will then receive a single grade for the entire group. If every member of the group contributed fully to the project, that grade will be used for the remaining 90% of each student's project component of the course grade. If members of the group did not contribute fully, the grade may be divided unequally. Except in highly unusual cases, individual project grades will not be more than one letter grade below or two letter grades above the group project grade. Our judgment of contribution comes from observation, from the quality of participation in group meetings with the TA, from knowledge of the project during presentations, and from the group self-assessment.

The project grade is based on two aspects:

• 1/2 is based on the week-by-week deliverables, i.e., the quality and effort you put into following the process
• 1/2 is based on the final result of the project, including the interface design, implementation, and your Open House presentation and poster

TIMELINESS IS IMPERATIVE! The project becomes more time-consuming as weeks go by, and projects that get behind will have great difficulty catching up. Each group may have the project items due extended one week without penalty one time only on the condition that the following week's tasks are completed on time. Other lateness will be penalized at a rate of 5% of the total project grade for each week during which the project is not on-schedule. No extensions of any type can be given on the project presentation or final project submission.

Each member of the project group is expected to be at each weekly meeting. A single absence will be excused, but further absences will be used as evidence of poor contribution to the group effort and will reduce the individual's grade.

Potential Problems

As is the case in real-world teams, CSci 5115 teams may encounter problems including team members who drop from the course, non-contributing team members, and disputes among team members. Your TA and professor are willing to help moderate discussions among team members, but please remember that some inequality of effort and ability is natural. Projects should not be so time-consuming as to require 100% effort from every team member. A good team should have one person more than is strictly necessary for completing the project.

Meeting With Users

Throughout the course of this project, you will be meeting with users who have agreed to help you evaluate and test your prototype. These meetings occur in Week 4, Week 11, and Week 12 of the course. It is required that each group member meet with at least one user during the user visits (Week 4), and one user during the user testing (Weeks 11-12).

Weekly TA Meetings

Each project will be assigned a specific weekly meeting time. Each group will have a 30 minutes each week in which to review status, discuss that week's deliverable and review plans for the following week. Each member of the project group is expected to be at each weekly meeting. A single absence will be excused, but further absences will be used as evidence of poor contribution to the group effort and will reduce the individual's grade.

Unless otherwise specified, your group should e-mail the weekly deliverable to the TA no later than midnight the day before your TA meeting. (Noted exceptions: Paper Prototype (Weeks 6 & 8) and Initial Coded Prototype (Week 12)). This will allow the TA to review your assignment and provide feedback at your meeting. To help us keep track of project progress, each TA will maintain a folder with copies of submitted assignments. Accordingly, please keep a copy of everything you submit, since the TA will not return their copy.

Some of the class discussion will revolve around your projects, and you will have several opportunities to evaluate and change your designs. Please bring your project materials to class each week. Also, please do not hesitate to ask questions both about the assignments and about how the class material applies to particular projects. The rest of this document lists the specific tasks associated with the project on a week by week basis.

Weekly Deliverables

Weeks 1 and 2: Getting Started

During the first two weeks of class we will start forming groups. By September 17^th, you will need to have groups and a tentative project idea, so it is important to start quickly. You might start by thinking about projects of greatest interest to you. Some groups have been successful in the past by choosing people first and identifying the project second. Experience has shown that it is valuable to discuss ambitions and expectations with prospective group members. A group that combines students shooting for "top 'A's" with ones seeking "easy 'C's" often creates tension and resentment on both sides. You also should be clear about your working style -- e.g., will you be available for group meetings on nights and weekends? Read over the Forming Project Groups and Choosing a Project sections of this document for more information.

Week 3: Groups & Ideas completed; Schedule Weekly TA meeting

By the end of class on September 17^th, you must have formed a group of 4-5 members, and selected a project that the group will pursue.

At this time you should also collect the following information from each group member:

• Name, Student ID and e-mail address
• Weekly schedule. This will be used to determine when your group will be available for your weekly TA meetings

One member of your group should e-mail this information, along with a brief description of your selected project, to the TA. See the Teaching Staff Page for TA contact information. The TA will contact you via e-mail to schedule your weekly meeting.

Week 4: Project Proposals & User Visit Plans

The purpose of this project proposal is to summarize the problem domain, the existing interface or infrastructure, related interfaces of interest, and your plans on where to focus as a group. This is a critical step in the process, since it allows the TA and professor to ensure that the project is appropriate and to suggest changes if parts of the project are not appropriate.

Included in the proposal should be the following things:

• A description of the project problem domain. This description should answer the questions:
• What types of users will use this interface?
• What types of things will users try to accomplish with this interface?
• What context of use is relevant to the project? Will users be coordinating this interface with other tools?
• The names of at least two users who meet the following criteria:
• They must be people who actually do the tasks that this interface is trying to address.
• They must not be members of the project team.
• They must agree to spend time with you for user/task analysis and then later for user testing.
• A description of the existing solutions to the problem, with some discussion of each to explain why it is inadequate or why you can do better. Note: The fact that an existing solution is free, or works on a different platform, is not sufficient, since that would not require any innovation in the user interface, merely re-implementation or porting.
• A brief (1 or 2 paragraph) description of where your group wants to focus, and your particular strengths (as a team) in that area. It is not always necessary to do a complete design or application -- for some projects it will be most valuable to focus on a particularly difficult part of an existing system or process. If you have design ideas, a sketch can help here. When you're looking for an advantage, familiarity with the domain, with existing applications, etc., can all help you. Feel free to include a list of "out of scope" items, if you want to be clear in managing our (and your) expectations.
• A very brief (1 paragraph) description of the tools and environment you expect to use for the project. Is this a project that can be completed in HTML? Will you be developing a Visual Basic application?

At this same meeting, you also need written plans for user visits, with specific goals, approaches, people, dates, and times. You should have a very general "script" which indicates who will ask or observe what in what sequence. You should also decide what types of notes you will take (and whether you need to prepare note-taking forms). You should, of course, be opportunistic during the actual visits, but you should at least imagine and document a possible scenario for this visit. This document can be informal.

It is not necessary for each group member to attend each user visit. However, each group member must participate in at least one visit. A minimum of 2 members must be present at each meeting. One member should be the 'facilitator' while the other(s) should play the role of "note taker(s)". Also, recognize that the number and duration of user visits may vary greatly by project.Remember that you need to build a relationship with these users, in part to help ensure their cooperation later in user testing.

The grade on the project proposal will be based on the evidence that you understand the project, the existing solutions, and the project environment. Length is not a good indicator of quality; excessive detail often reveals insufficient understanding and thought. Poor proposals, and proposals for projects that are rejected as inappropriate, can be revised and re-submitted before your next meeting.

The user visit plan will be graded based on the amount of thought you have given to the visit in general and the particular goals and methods appropriate for your project.

Recommended reading to put you in the right mindset: Beyer, H. R. and Holtzblatt, K. 1995. Apprenticing with the customer. Commun.ACM 38, 5 (May. 1995), 45-52. DOI=http://doi.acm.org/10.1145/203356.203365

Week 5: User visit reports; Task Analysis

This week you submit a report on the user visits you carried out. Turn in one report for the entire group, combining your separate findings into one document. Included in that report is a task analysis for the project. The report should include the following:

• A summary of the actual process you carried out (or summaries, if it differed for the different users).
• A user description that discusses the relevant user characteristics you are considering in your design. For example, what is the expected range of computer and Internet experience? What is the use environment? Special user concerns or motivations?
• A general description of what the users do today to fulfil the needs that your project is directed towards.
• Detailed descriptions of at least five tasks (as defined in Task-Centered User Interface Design) that users would accomplish using the results of your project. Each task description should include an identification of who the users are (i.e. which user generated the task), what they are doing, and why they are doing it. At least three of the tasks should be general tasks that users accomplish today, although perhaps not as easily or well as they will with your design. Remember that the task descriptions should not be tied to a specific interface, and should be detailed enough for your users to understand them and for you to use in analysis.
• For two of the tasks, you should include a detailed (technology/interface specific) scenario explaining how users accomplish that task today. The current scenarios may involve computer software, paper, other devices, or any combination.
• A summary of what you learned from the user visits that is not captured in the above.
• A brief summary of how effective your process was, and any lessons learned about how to carry out the user visits differently in the future.
• A brief summary of any information you were unable to gather, along with a description of either plans for how to gather it or plans for continuing without it.

During this week's group meeting with your TA, you should be prepared to discuss what you learned from the analysis, including any changes in your conception of the project. This discussion will be a major part of the meeting and will be graded.

Weeks 6 & 8: Paper Prototype

These weeks mark a major checkpoint in the project. Your group will produce and submit your first prototype - a paper prototype. Further, to aid your design process, you will present several initial design concepts to the class staff and your fellow students, then produce an updated design based on their feedback.

ON PAPER PROTOTYPES. A paper prototype should illustrate the basic components of the interface with sufficient detail to walk through the interface looking at navigation (labels and icons) and consistency. Ideal prototypes will be paper sketches or print-outs from a drawing program; you may build an executable prototype, but if you do you must print screen shots from it to proceed on paper with the initial analysis.

There are several reasons for building a paper prototype first. Most important, it gives you an easy way to get significant feedback on your design ideas before spending the effort to implement them: remember that it's easy and cheap to make changes early in the design process, hard and expensive to do so later. Also, building your prototype now allows it to evolve as you start the process of revision, a key to creating an effective design.

FOR WEEK 6. Following the adage "to get a good idea, get lots of ideas", your group will prepare two independent paper prototypes for this week. This means that you should split the group into two subgroups, each of which will work completely independently to develop a paper prototype. Both groups will base their designs on the work you've done to this point as represented by your user and task analysis and personas.

In week 6, each subgroup will give a presentation of their design to the class. Each subgroup should show the design itself and explain the ideas behind it. After reminding the audience of the main tasks identified in Week 5, subgroups must pick one task and carry out a walk-through of how their interface will support the task. Both subgroups must present the same task in order to help the audience compare and contrast possibly different designs for the same task. You should explain why you think your design is likely to be successful, referring to principles from the readings and to the results of your user and task analyses. You will receive feedback from the course staff and your fellow students on your design ideas.

FOR WEEK 8. Based on the feedback each subgroup received, the group as a whole should come together and produce a single unified prototype. You might merge the best ideas from the two initial prototypes, or use one and abandon the other... it's up to you. However, for this week's deliverables you must include not just your updated prototype, but -- at least as important -- an explanation for how you arrived at it. First, you should summarize the feedback you received on your initial prototypes. Second, you should give a rationale for your updated design, referring to the two initial prototypes and to the feedback you received. You should explain what features you kept from each initial prototype (and why), what you changed (and why), and any difficulties you had in merging design concepts from the two initial prototypes. Bring a copy of your new merged prototype as well as copies of your older prototypes for the TA. This will enable you to explain how you went from your initial versions to the final prototype based on the feedback and merging process.

Note that you get two weeks to merge the prototypes. This reflects the importance of this, and emphasizes that you should be thoughtful in how you do the merger and in the rationale you produce to explain the merger process.

Week 7: Scenarios for User Tasks

Based on your prototype, you must write up detailed scenario descriptions for three of your project tasks (including two of the tasks for which you have current scenarios from week 5's task analysis). The scenarios may be expressed in terms of standard interface widgets and concepts (e.g., "press the button marked "GO", "pull down the menu labeled "FILE", "follow the 'Schedule' link," or "go back"). The scenarios should be expressed in terms of your user interface. A guideline for evaluating a scenario is that a person familiar with the type of software (e.g., web, windows) but not familiar with the task should be able to follow it. One way to describe scenarios is as steps similar to those in a user manual. For example, Step 1: Open the travel planning site in a browser (IE7 and Firefox 2.x supported) with at least 1024x768 resolution, by typing in www.travel2day.com; Step 2: Enter username and password. etc.

Next, write up two persona descriptions. In addition to the assigned reading, you might also check out the articles at http://www.uie.com/articles/personas/ (both the main article and the "see also" ones). Your persona descriptions should be anchored in your user analysis and chosen to be useful for your ongoing design.

By week 7, you should also be preparing for your executable prototype. While you still have plenty of time for development, you need to decide upon and become comfortable with the development tools you plan to use.

Week 9: Cognitive Walk-through Evaluation Report

Prior to this week"s TA meeting, your group should conduct a cognitive walk-through evaluation on your interface prototype, stepping through each step of three scenarios. At each step, you should ask the four questions from section 4.1.3 of TCUID along with the question "Is there another control that the user might select instead of the correct one?" (You may find it useful to assign individual group members to certain questions for the duration of a scenario.) Keep a log (separate from your report) of each walk-through. You should then write up a list of interface problems discovered during the walk-through, with a brief note about how you discovered them (e.g., direct result of a particular step in the walk-through, follow-on discussion based on an issue raised in the walk-through, etc.).

Finally, make a list of your ideas for improving the interface (both to resolve the identified problems or for other improvements) . These need not correspond individually with problems--some broad re-designs can address several problems. You may make any improvements you like before the next evaluation, but are not required to do so.

By now, you should starting work on your executable prototype.While you have three weeks before it is due, and you will likely make design changes before that time, you should be building the necessary infrastructure for the application.

Week 10: Individual Heuristic Evaluations; Heuristic Evaluation Report

Each group member should individually complete a Nielsen-style heuristic evaluation (see section 4.3 of TCUID. This corresponds to Neilsen's Heuritics Ver 1.0 circa 1990 from the lecture notes on 'Heuristic Evaluation') using at least the first nine heuristics (help and documentation are optional at this stage). You should each maintain a log of the problems you find, along with a brief note about how you found it. A copy of this log will be turned in to the TA for grading. The assignment will be graded individually based on the thoroughness of your individual evaluation.

After each member of your group has completed the individual evaluation, your group should meet to combine your individual heuristic evaluation lists into a single list of usability problems. The list should be organized in a manner that makes it easier to address related usability issues together. Once you have a combined list, annotate it with possible solutions and improvements. Note: The group should set a deadline for individual evaluations to be complete -- if a group member has not completed the evaluation in time for the group meeting, the rest of the group should proceed; that member will receive a substantially reduced score.

Week 11: Prototype Plans

Now that you've finished your non-user evaluations, you need a group plan for the executable prototype, including what changes you plan to incorporate and what the design will be (along, of course, with who is coding what and how). The executable prototype should incorporate improvements from the walk-through and heuristic evaluations. Your plan will be evaluated based both on how well you've incorporated lessons from the evaluation and on how effectively the prototype will facilitate user testing.

Week 12: Coded Prototype, Plans for User Testing

Your first running prototype is due this week. You must demonstrate it to the TA in your weekly meeting. Be prepared to discuss issues that arose in implementing it. In particular, be prepared to step through the usability problems identified in the walk-through and heuristic evaluations. For each problem, you will need to explain whether and how the problem was addressed and why.

In addition, your group should write a plan for your user testing. The test model should include asking users to accomplish tasks while thinking aloud, but may include other techniques as well. The test plan should, at a minimum, include the following:

• Identify the users who will be involved with this testing, the time and place for user testing, and the role that will be played by each group member in each of the user tests.
• A description of the testing logistics. How will the prototype be made available to the user? Where and when will the test take place?
• What are the user tasks that will be tested? How will they be described to the users?
• User testing goals. Beyond simply searching for usability problems, what specific design decisions or assumptions about users will you test?
• Instructions for the user. Include both orientation instructions and specific instructions for the test. These may be read aloud during the test, but they should be written out for the test plan. The instructions are best captured in a user testing script that will be printed and used the group members. The script will serve as a checklist/outline for the facilitator to carry out the testing in a systematic fashion. Typical items in a script include greeting and introduction, describing the role of the users in the test process and setting of expectations, paper work (consent form if required), prototype orientation, task description and task execution, etc.
• If the user testing is to be followed by a structured interview, what questions will be asked?
• What data will be collected? What form should be used to collect it? (e.g., video, audio, note taking, etc.)
• How will the test be carried out? How many group members at each test (each group member need only participate in one test; if you are testing several users at once, you may want to split up roles)? What roles will be used in each test (e.g., orienting the user, logging user actions on a tracking form, post-test interview)?
• Anticipated problems and how you will address them. What will you do if the user gets stuck? What if the user selects a non-operative feature?

Week 13: User Evaluations & Report

During the preceding two weeks you will have conducted your user tests and written reports on the testing. Your group should submit the following as a result of user testing:

For each user visit you complete, you should submit:

• Raw notes from user testing
• Any other information discovered during the user test (e.g., information about user, tasks, etc.)
• A brief (1 page) report on how the test progressed, where the test deviated from the plans, and any changes that should be made in the plans based on the experience. This report should conclude with a one paragraph summary of what was learned from this user test.

In addition, your group should compile a combined list of usability problems discovered during your user visits. Along with each of these problems, include a list of possible solutions.

Week 14: Prioritized Change List; Plans for Presentation

Now that your final testing is completed, your group has four remaining goals:

• Finishing the implementation of the interface (most groups will work on this incrementally during the semester).
• Improving the design in response to evaluation.
• Presenting the project.
• Completing the project report.

At the week 14 meeting, you provide intermediate results for the last two of these goals.

• A list of your design changes. There are three phases of changes. Clearly identify the phase for each change.
• Phase 1: Changes made from the initial paper-prototype through the coded prototype. What changes were made before you met with your users?
• Phase 2: Changes made from the user evaluations through the end of the semester. What have you changed since meeting with your users, and what do you plan on changing before the end of the semester? You should prioritize the list and label each item with an importance and a difficulty rating. It is sufficient to have three levels of each: (importance: critical, medium, low; difficulty: hard, medium, easy). Please note which of the changes you have already made and those you plan to make before the presentation.
• Phase 3: Future changes. Assume this course were a year-long course, and the work you've done this semester is only the first step in a longer development process. What would your project look like at the next step? What direction are you heading? What features would you add if you had time, resources, etc.? Provide an idea of what your vision for this project is.
• An outline of the presentation you plan for the open house. This outline should include:
• A list of one to three key messages that you'd like people to take away from your presentation.
• A sketch of the poster design for your presentation.
• An outline of how you will demonstrate the interface to passers-by in a quick way (figure that you have about 2 minutes to capture their attention), and what you'll do for people who are more interested.

Week 15: Presentation; Final Implementation, Code, and Documentation; Final Report; Retrospective

NO MEETING WITH TA THIS WEEK

The class open house will be held during the class week of class, the Class Schedule will be updated when the time and location are set. You will have at least 1/2 hour before the open house to set up your station. Each group should set up a working demonstration of its interface and a poster highlighting the design and the key interface decisions.

The presentation provides an opportunity for you to present your project to a wide audience including other students, course staff, outside experts in HCI and UI design, and your users (if you choose to invite them). The course staff will use the presentation as part of the project evaluation. We will schedule a tour through the demonstrations, making sure we see each one during the session. You will have approximately 10 minutes for your graded presentation with the course staff.

Your poster and presentation should each contain the same type of information. The poster should be self-contained, and not depend upon your presentation to make sense. Information you should include:

• Background information on your project. Similar to a "Motivations" section of a research paper, this describes why this project was chosen, what the existing solution(s) were (if any), and why it interested you. What were you trying to improve? Who are your users?
• What are the main tasks your system supports?
• What's interesting about your project?
• What did you learn from your user visits? How did your design change after you met with your users?
• What would you do next if you were going to continue working on this project?
• The poster should also include screenshots (approximately 3 to 5) of the main areas of the interface.

The following items need to be e-mailed to the TA before Monday, Dec 8 at 4PM.

• A brief description of the final implementation, including what works (and what doesn't), how to use the program, and any plans the group has for continuing the project.
• Any materials (e.g., copies of poster elements) from your presentation.
• A group self-evaluation that addresses three questions:
• How good was the project? What grade does it deserve and why? (about 1 page)
• How well did the group work together? Describe qualitatively how you divided work among the group members. How often did you meet? Any particular lessons, problems, highlights from the experience? (about 1 page)
• A static HTML page describing your project. This will be used as part of a portfolio to show future students examples of work completed in this class. This page should include:
• The name of your project.
• A brief description (3 paragraphs) of your project. What is the purpose of your interface? What tasks does it support? Who are the intended users?
• Screenshots or pictures showing the main components of your interface. Please limit the number of photos you include to 5.
• Benefits and drawbacks of choosing this type of project (1 paragraph). What advise would you give students who are considering a project similar to yours? Would you recommend that other students attempt similar projects?
• The source code from your project.

In addition to the group's assessment, each group member must individually e-mail the TA the following items before Monday, Dec 8 at 4PM.

• A set of individual "lessons learned" essays (about 1 page) indicating what you learned about UI design, implementation, and evaluation from the project and course.
• Your own assessment of each member's contribution, including discussion on both how each person contributed and the total amount of contribution. Please indicate the quantitative contribution as a percentage, with the total from all group members totalling 100%. Remember that the course is not a course about coding; it is possible that one or two group members did all the coding but all contributed equally. It is also possible that one or two members did equal work in all other aspects and did all the coding, in which case they contributed more than the others in the group did.

At your open-house presentation, you will show the TA and professor your project in detail, and instruct them on how to run it for later evaluation.

After the open-house presentation, the TA and professor will meet to grade your final project and assign final grades to individuals in the course.