- The presentation is based on the attached data (DATA SET FOR STUDENT PROJECT, and completed SUS forms)
- The PowerPointreport format is attached (Student Project Report Format).
- The grading rubric also is attached (student project grading).
DATA SET FOR STUDENT PROJECT POWERPOINT The following are the data that you will need to prepare your individual project PowerPoints. The data are representative of what you would have collected. You are doing a limited evaluation of the Windows 10 operating system (running on a Dell Latitude 3450 and Apple iOS X operating system running on MacBook Pro (Model MacBookPro12,1). Assume that the laptops are sufficiently similar in hardware and processor speed to make a valid comparison. You have collected data from four participants, all of whom are GMU undergraduates. (Four participants are sufficient for the project.) You conducted your experiment in the Fenwick Library Reading Area, also known as the MIX. Five tasks were evaluated in this limited study. As discussed in class, you used a “heads down” protocol because you wanted to measure time on task. The five tasks are: · Uninstall an application · Change the wallpaper · Add a Bluetooth device (a headphone) · Change the time & date · Zip a file Note: You do not need to provide any more detail than just the names of the tasks in your PowerPoint. Thus, no details, specifically, the task analysis steps are provided here. The sequence of events followed for each of the participants you dealt with follow. 1. Participant asked to participate using the introduction I provided. “Hi! My name is Jack Jones and this is Hank Smith. We’re doing a study to determine how easy it is to use two university library web sites for our class, SYST 469. We’d like you to help us out by doing a few tasks that you would do with operating systems. We only need about 5-10 minutes of your time, and all responses will be anonymous. Also, you can’t do anything wrong as we’re assessing the products and not you. Please have a seat and we’ll start.” 2. Participant given the informed consent form. All participants looked at the form and agreed by nodding their heads. As required by anonymity, you did not ask for any participant names. Also, all participants left the form on the table. SYST469 – Human-Computer Interaction Class Project Informed Consent Form RESEARCH PROCEDURES This research is being conducted to support a student class project relating to the ease of use (usability) of different products. If you agree to participate, you will be asked to perform several tasks using a commercial product or products, and give your opinion about product or products usability while being observed by the students doing the project. RISKS There are no foreseeable risks for participating in this research. BENEFITS There are no benefits to you as a participant other than to help students complete a course requirement for SYST469. CONFIDENTIALITY The data in this study will be confidential. Names and other identifiers will not be placed on questionnaires or other research data. PARTICIPATION Your participation is voluntary, and you may withdraw from the study at any time and for any reason. If you decide not to participate, or if you withdraw from the study, there is no penalty. CONTACT This research is being conducted the following students: Jack Jones & Hank Smith. The faculty advisor is Jack Laveson of the System Engineering and Operations Research Department at George Mason University. He may be reached at 703-577-7398 for questions, or to report a problem. You may contact the George Mason University Office of Research Subject Protections at 703-993-4121 if you have questions or comments regarding your rights as a participant in the research. This research has been reviewed according to George Mason University procedures governing your participation in this research. CONSENT I have read this form and agree to participate in this study. 3. Participant given specific task instructions. “You will perform a series of tasks using the two operating systems. Do these tasks as you would do them at home following the steps on the sheets that we will give you. Remember that we are evaluating the operating systems, not you. Your first task is to uninstall an application listed on this paper.” You then handed the participant the task analysis steps for uninstalling an application for one of the operating systems. This was followed by the other tasks for the same operating system. Note that the first and third participant performed the tasks first for Windows 10 and then iOS X; the second and forth participant performed the tasks first for iOS X and then Windows 10.” 4. Participant performed tasks, during which you recorded time (from start of task to end of task). No errors were noted so errors will NOT be part of the results and conclusions. Report that no errors occurred. 5. Participant completed SUS after performing all tasks for the first operating system just dealt with. “For each of the following statements, mark one box that best describes your reactions to the tasks that you just performed with the operating system.” 6. Participant repeated the tasks for the other operating system followed by the completion of another SUS. “For each of the following statements, mark one box that best describes your reactions to the tasks that you just performed with the second operating system.” 7. At end of tasks for the second operating system, participant answered the opinion question, “Which operating system was easier to use, and why?” 8. Participant was thanked for participation. “Thank you for your time and effort. We really appreciate your help.” Here are the time on tasks for the participants. Note that because the time was recorded in tenths of seconds, only report averages in tenths of seconds. PARTICIPANT TASK Win 10 Time (sec.) iOS X Time (sec.) #1 Uninstall 29.4 36.2 Wallpaper 20.0 12.0 Bluetooth 23.7 17.7 Time & Date 35.8 23.1 Zip Folder 19.1 10.4 #2 Uninstall 67.3 60.0 Wallpaper 51.0 15.7 Bluetooth 30.8 42.8 Time & Date 32.1 41.2 Zip Folder 51.3 29.5 #3 Uninstall 37.5 14.5 Wallpaper 25.7 5.9 Bluetooth 21.4 9.3 Time & Date 29.3 18.7 Zip Folder 20.9 7.3 #4 Uninstall 45.9 24.1 Wallpaper 20.3 12.2 Bluetooth 18.5 20.8 Time & Date 23.3 25.5 Zip Folder 16.1 16.0 Each participant completed two SUS forms, one for Win 10 and the other for iOS X. The SUS forms are in the pdf file, completed SUS forms. You will need to score each of the eight forms. I suggest that you use the scoring technique found in the Lecture 9 PowerPoints, slide 30 (single version of PowerPoint lecture). Lastly, the participant responses to the question, “Which operating system was easier to use, and why?” are as follows. Participant #1 – iOS X because icons easier to understand. Participant #2 – iOS X without any comments. Participant #3 – iOS X because of better layout. Participant #4 – Win 10 although iOS X also worked well. PowerPoint Presentation Student Project Report Format PowerPoint report format (1/7) Executive summary – briefly, what did you do and what are your results? What did you do: a limited usability field study of two products/apps specifying what these are Who were the users/participants (no names, but OK to state students) What tasks did you evaluate? List short titles, e.g., Enable “Rhythmic backlight alert” Which product/app has better usability This format follows that of a research report 2 Red arrows indicate problem areas with prior student project submissions PowerPoint report format (2/7) Introduction – What are you doing, and why? This is a usability test as a class project for SYST469 State definition of usability, and purpose of field study Usability is the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use. [ISO 9241-11: Usability (1998)] The project is an evaluation of two existing interactive products based on data obtained from participants during a field study. State the products/apps to be evaluated including specific models, operating system versions, etc. in sufficient detail that someone could buy exactly the same product/app 3 PowerPoint report format (3/7) Method – describes the details of what you did What tasks did you evaluate? List titles and, if needed, brief descriptions – e.g., Enable “Rhythmic backlight alert” – makes screen flash on incoming call Where did you do the testing? Who were the users? Remember, no names or other personally identifying information as users are anonymous. How were the tasks presented? What was the presentation order? Were task procedures given to the users / participants A heads down protocol was used 4 PowerPoint report format (4/7) Method – continued Specifically, what did the user do? User asked to participate User advised of informed consent requirements – no need to include informed consent form User give specific task instructions User performed task, during which you recorded time and possibly errors User completed SUS after performing all tasks for a product Remember there will be two SUS scores/user, one for each product Do not include SUS questionnaire as this is well known At end of all tasks, user completed opinion question(s) User thanked for participation 5 6 Hint: - The numbers above the boxes were added after the user completed the SUS, but before scoring to make the job easier. - It is OK for you to do this as well. - Also remember to multiply the sum (example = 29) by 2.5 to get the SUS score. PowerPoint report format (5/7) Computed data – the information you collected about users performing the tasks Provide computed data (not collected / raw data) in table format Raw data will be submitted separately Graph the computed data analysis in a way that you consider useful, however: SUS has a range of 0 to 100 (so format axis manually from 0 – 100 to give the viewer the correct range of values) TOT has a range of 0 to maximum time (so format the axis manually from 0 – maximum time) Examples of computed data tables & graphs on next 3 slides 7 Raw / collected is NOT to be submitted; this was applicable only if you were collecting the raw data that you are not doing this semester SecondsDownload & Install AppTake Photo & Set As WallpaperCreate Group & Add 2 AppsAdd A ContactChange Ringtone Android17.731.415.637.411.2 iOS13.727.112.231.914.4 Mean Time/Task Seconds Format Vertical Axis so range is 0 to maximum time Mean Time/Task TOT by Task and Product 8 AndroidDownload &