From our first wireframe of BuddyUp, a heuristic evaluation was conducted based on Jakob Nielsen’s 10 general principles for interaction design. The user was asked to provide a score for each of the heuristics on a scale of 1-4 with 1 being very poor and 4 very good and provide an explanation of what could be improved. Out of the 10 heuristics, only “consistency and standards” and “flexibility and efficiency of use” received scores of 2. It was noted that the consistency between the buttons and their functions did not match convention, and the expected outcome of their actions was not executed. The user had noted that it felt like they were “digging for options” in the sidebar menu to find what they were looking for.
However, the cognitive walkthrough produced results that were accurate to what was anticipated. Both participants picked the scenario where a man wanted to buddy up with someone who was relatively new to the gym, and both successfully listed the complete tasks to achieve that goal. These tasks aligned with the experiences we wanted the users to have.
The background question attempts to see if participants have used similar apps in the past and how they’ve met people in their life. This allows us to get an idea of their past experiences and judge the possible advantages more experienced participants may have regarding learnability.
For Task 1, participants are asked to create an account on the app. They are given basic instructions that involve selecting a gym within a few minutes of their home and indicating interest in weight loss and strength improvement. Participants are also told that they want to express their interest in movies, indicating that the corresponding category should be selected when choosing from hobbies. Once complete, participants are asked to rate the ease by which the process was completed.
Task 1 primarily seeks to gauge the effectiveness of the sign-up process, examining if users are able to select options that reflect their needs and interests. If they are able to complete the process while selecting the proper information, it is likely that the sign-up process is effective. As an account is required to use the app, an effective sign-up is key to the overall effectiveness.
During Task 2 participants are told to pretend they are a new gym-goer who is looking for a more experienced person to help them get started. They are also told that they are looking for a gym near CSU, Chico. This information should be used to filter potential matches by examining profile information of other users. Once complete, participants are asked to rate the ease by which the process was completed.
Task 2 gives insight into the learnability and effectiveness of the matching process, which is one of the app’s core features. On the learnability side of things, the task lets us observe if users can figure out how to get information on other users and successfully match with others. On the effectiveness side, it allows us to see if users can use the given information to successfully find a match with all or most of their requirements.
Task 3 asks participants to make changes to their profile, giving them instructions to change their profile pictures, gym location, experience level, and activities. Once complete, participants are asked to rate the ease by which the process was completed.
This task focuses on gauging learnability, error tolerance, and effectiveness. If the app is learnable, then users should have little problem finding the page and options to edit their profile. If editing can be done successfully then it provides users a way to fix any mistakes that they may have made during sign up. Being able to update gym, goals, and experience is also important to reflecting users’ locations and improvements, which is important the app effectiveness.
The wrap up questions aim to get any additional feedback on what participants liked, disliked, or would have liked to see. These questions help us assess what users appreciated, if there are any issues that need to be fixed, and possible future endeavors.
The study employed a think-aloud approach. Participants were given tasks to complete with a click-through protype and told to explain how they approached these taks. This gave insight into why they chose to take the actions they did and gave us feedback about any instances of confusion or content. From this, we not only see if they are able to accomplish tasks, but also if the conceptual and mental models are in sync.
In this case, the study was a summative evaluation, as this is the final prototype that will be produced.
When asked about their previous experience meeting friends or romantic partners, participants often cited school and work as vectors of introduction. A few had experience meeting others over communication apps and websites, such as Skype or Discord, however there was little in the way of feature suggestions or complaints that would be applicable to BuddyUp.
From our study, we see a high degree of overlap between participants in terms of actions taken. Most followed the same, or extremely similar paths when accomplishing tasks. Almost all tasks were considered easy by participants, with only one participant giving a task a difficulty rating below 4 (where 1 is very difficult and 5 is very easy). All tasks were completed successfully by each participant.
The lack of failure and perceived ease of use were further confirmed by participant comments. Many remarked that tasks were “straightforward” or “simple”. This is a good sign for the learnability and effectiveness of BuddyUp.
However, there were some common issues observed during the study. Most users missed or nearly missed an important feature that allows one to view a more in-depth user profile by tapping the profile card on the home page. If this feature is not used, a good chunk of the other user’s information is not viewable, which may hurt effectiveness and efficiency when looking for a gym buddy. Along with this, two out of five participants described the sign-up process as too long and remarked that they may have given up due to this. Such an event would be devastating to the effectiveness of the app as they would be unable to use it at all. The same two participants also attempted to sign up using a 3rd party, namely Google, but were not fully successful due to this flow not being completed on the prototype. There were some smaller complaints regarding the readability of profile information on the home page. Some reported that the small size of the information made it difficult to read. Another commented that the gym location was not prominent enough despite its importance to matching.
Additional feature suggestions included a greater variety of activities and a way to filter the home page based on certain criteria.
To fix the issue of users missing the detailed profile view we could either add a mention of this feature to the tutorial or add a more obvious way to expand the profile information. For instance, a drop-down button labeled “view more” that expands the profile to display all the information, or simply a button labeled “view profile” on the bottom of the profile card.
To address the complaints of a long sign-up process, we could condense the number of pages, putting more information on a single page. This might make the process feel shorter but may still be considered too long. Another solution would be to put a “skip” or “select later” button on any non-essential pages.
Google, Facebook, and Apple sign-ups and logins can be addressed by adding additional pages to reflect the options more thoroughly. The login page is also lacking buttons for these options, so they can be added.
Increasing profile card and font size should alleviate complaints regarding readability. Attention to size and positioning hierarchy can be employed to make important information, such as gym location, more prominent.
Finally, additional features can be worked on once base functionality is completed. Filters may necessitate back-end storage of user selections so that filtering is possible.