Boj Mobile App

SI 622 - Needs Assessment and Usability

Over the course of the semester, I worked with Boj, a Detroit-based startup that matches employers with job and internship seekers that match their culture and values. Boj is currently embarking on a major redevelopment of their product in order to succeed in the saturated job-matching industry.

During the semester, I was able to apply various user experience research using a variety of methodologies, including a heuristic evaluation, needs assessment, comparative evaluation, concept test, usability test, and surveys.

Interview Research

Since Boj has been primarily focused on college students, I reached out to current students who have used job-matching service before. However, since Boj is expanding its reach, I also reached out to recent graduates and individuals looking for a full-time job. I helped conduct 4 interviews with 2 students looking for an internship as well as 2 working professionals. The goal of the interview was to better understand the needs, criteria, and pain points of users when actively seeking a job or internship.

Comparative Evaluation

In addition to conducting research through interviews, I analyzed 13 job-seeking services, with a focus on each app’s sign up, profile/resume creation process, and job application tracking. Our team also generated tables and visual reports to provide better insights into these results.

Survey

Next, a survey was deployed to ensure the interview data was fully representative. In addition, I wanted to gather data about the important factors users look for when they search for a job or internship, including the following questions:

• What features do users find useful or enjoyable across different types of job-service applications?
• What sort of devices do users use when looking for a job or internship?
• Are users okay with having interactive games to complete their profile or match with companies?

Heuristic Evaluation

Next, a heuristic evaluation was conducted for Boj's mobile iOS app. Using Nielsen’s usability heuristic evaluation to conduct the comprehensive usability evaluation, I evaluated the app using a ranking system that prioritizes the major issues based on their severity. Afterwards, I conducted an interpretation session to identify and consolidate the the most pressing issues and key findings, and to brainstorm solutions.

After examining Nielsen’s 10 key heuristic criteria, Insight Research has determined that the functionality of the app could be improved by addressing several minor issues that presented themselves during the analysis. These issues, in order of severity, are:

• Users are not presented with error messages when completing their profile
• The only documentation for help, or “tips” is difficult to navigate and understand
• On some sections, users would have to restart their inputs if they made a mistake or would want to change something in the future

Concept Test

Since I was not able to gather any meaningful data about Boj's idea of implementing gamification on our surveys, I decided to conduct a concept test to ask users about how they feel about different gamification features, including examples like skills assessments, badges, neuroscience-based games, and personality questions.

Our findings are as follows:

• Participants are less likely to spend time playing a games (1-5 minutes) but are willing to spend more time (15-20 minutes) with assessments and personality quizzes
• Many participants expressed concern over biases and discriminatory results of implementing gamification in a job-matching service

Usability Test

Since Boj currently faces high fall-off rates during the sign-up process, I wanted to investigate why this might be happening so they can improve on these issues during the redevelopment process. I helped conduct a usability test with five participants of different education majors who have searched for a job or internship within the past year. Four tasks were centered around profile creation and exploration. Upon completion, participant's feedback were recorded. The findings are as follows:

• Profile creation can be difficult and overwhelming for users. Required fields aren’t clearly identified and users don’t like having to fill out excessive information. This might explain the high falloff rates Boj has experienced in the past.
• Users were confused and frustrated with the lack of profile customization and options. College-specific categories aren’t fully representative of the college experience and lack fields such as leadership roles and clubs. Profiles also lack fields that cater to professionals.
• Users found editing their profile was confusing.Users want their prior selections to be remembered within the Boj App instead of having to re-enter existing fields every time they make a new change.

Summary & Reflection

Overall, Boj was given the following suggestions on how to tackle various aspects of their mobile app and service:

• Boj should improve their profile creation by clearly identifying required fields and removing any non-essential questions that might hinder a user from signing up. If additional information is needed from users, look at other methods of collecting it (notifications, e-mails, alerts, etc)
• Since so many users use a traditional computer to apply for jobs, Boj should focus on creating a web-based version of their service instead of relying solely on a mobile app
• If gamification is implemented, look at implementing assessments that last no longer than 15-20 minutes. Be transparent about how assessments will be utilized by employers.