Cloud product evaluation for productivity improvement
* Note: Product is under NDA, specific details are not included below
Team
My Role:
UX Researcher
Collaborators:
Product Owner
Lead Software Engineer
Software Engineer
Timeline
Duration: 5 Weeks
Study Presented: 01/06/2023
Methodologies
User Interviewing
Contextual Inquiry Demonstration
For a full case study explanation, including study process, please schedule a case study review with me either through the contracting agency that is representing me for the role you are hiring for or via my email.
Problem Space
Anecdotal evidence and user feedback indicated areas of improvement for a feature within an internal cloud platform for Optum Software Engineers (SEs). User experience for the feature was understood to be so poor that productivity of SEs suffered. Unnecessary time was spent using the feature and 3rd party platforms were used to complete tasks that the feature should have allowed users to do. Using 3rd party platforms to complete tasks created security issues as protected company information was put at risk.
A feature roadmap had been put together by the Product Owner (PO). The PO wanted to understand user pain points for the feature to validate their roadmap and include any improvements not identified anecdotally by users.
Sample roadmap- does not reflect product discussed
Study Impact
When addressing the study objective of understanding how the feature could better function to meet Software Engineer needs, user feedback was very clear. Participants expected the feature to function and look just like a well-known product used by the general population. My recommendations centered around emulating key features of this popular product to match the mental model of Optum Software Engineer users.
As part of my share out, I presented a comparative analysis of the Optum feature that called out conventional elements that should be improved or included, citing conventional elements in the product used by the general population that had been cited by study participants as an example of the best possible user experience.
During the discussion after I presented my insights and recommendations, the Lead Software Engineer and Software Engineer assigned to the feature addressed some of my recommendations. One recommendation to move the location of an element of the feature to a conventional place on the page was addressed by saying that they had designed it that way based on their conception of how users would prefer to use it.
The back and forth discussion we had was instrumental in providing final recommendations to improve the feature. The PO altered the feature roadmap to include additional improvements and reprioritization of existing items based on study findings.
Research Process
The feature of focus for the study was comparable to another widely used product by the general population, so there was a user expectation that it would function conventionally. With this comparable product in mind, I wanted to understand the mental model users had for the internal Optum feature, and how the feature could be improved to satisfy that mental model.
The best way to understand the mental model users had for this feature was to use the interview methodology paired with a contextual inquiry component by asking participants to show me how they used the feature by demonstrating past behavior. I collaborated with the PO to come up with research questions, as well as define the user segments that were important to understand feature use. The PO wanted to gather feedback from users in 3 groups based on how many times they used the feature within the past 90 days from the start of the study.
High Volume Users: 90+ uses
Intermediate Users: 6-89 uses
Low Volume Users: 1-5 uses
I wrote a moderator guide that asked interview questions related to job title and responsibilities as they related to use of the feature, current feature use behavior, how the feature solves user problems, and how the feature does not solve user problems.
After asking the interview questions, I had participants share their screen and demonstrate how they used the feature by showing me past behavior, asking follow up questions when appropriate based on their behavior as I observed them using the feature.
During sessions, many times without being asked, participants reported pain points and compared the Optum feature with the well-known product used by the general population. This was a clear indication how the feature could be improved and allowed me to present recommended changes that would meet the needs of SEs for productivity and the security of company information to the PO and other team members.
What I Learned
This was my first study at Optum, and I really enjoyed working with the PO who was responsible for the feature of the cloud product that I ran the study for. In my previous UXR role, I had been trained to use Confluence to construct my study plan. At Optum, I was given a study plan example in a Word document that I templated for all of my Optum studies.
I learned:
I preferred putting together my study plan in a Word document
Sharing it was easier as team members didn’t need to create an Atlassian account to access a Confluence link
Readability was improved compared to the non-linear layout of the Confluence study plan I had been taught to construct