This project was completed as part of a class assignment in collaboration with Lenovo. We were asked by Lenovo to assess signal detection theory as a possible tool for analyzing how information is displayed on a website. Using a public Lenovo laptop selection page, we coded user responses based on signal detection theory and showed that the method could be successfully applied to UX work to measure changes to a website. More details of the project are included below.
Investigating the website
In this project, we used the version of the UK Lenovo laptop selection website that was live in September to November of 2016 (pictured above). We first familiarized ourselves with the website by conducting a heuristic analysis and a cognitive walkthrough.
The heuristic analysis identified possible areas to focus on when designing changes to the website.
The cognitive walkthrough allowed us to focus on each aspect of the website. The laptop pages were organized in a decision tree, which allowed users to narrow the range of displayed laptops based on their needs. The walkthrough helped identify the process behind each decision.
First study with users
Signal detection theory measures whether an individual detects a signal in the presence of noise. The signal can either be present or absent, and the individual can either decide the signal is there or not. This gives a total of four responses (pictured on right). We conducted an initial study to test how we could use signal detection theory in a UX setting.
Using UserTesting.com, we asked users to go through the process of selecting a laptop using the live website. We also asked users to identify which features they looked for in a laptop. We applied signal detection theory to their responses by coding whether the features they stated were present on pages they visited, and whether they noted those features were present or not.
This initial study gave us valuable insight into users’ thoughts during the process of selecting a laptop. However, our attempt to use signal detection theory was not as successful as we hoped. We reevaluated our study design and chose a different approach for our next study.
Designing Changes to the Website
In our final assessment of signal detection theory as a tool, we wanted to use it to measure changes to a website. To do this, we needed to propose changes and develop a prototype to test those changes with users.
Based on our heuristic analysis, cognitive walkthrough, and feedback from users in our initial study, we decided to address consistency on the laptop search results page. This page allows users to compare multiple laptops quickly, but not all laptops show the same information. We developed guidelines to standardize the information displayed for each laptop, and designed a prototype (example pictured on right) based on those guidelines.
Measuring Changes Using Signal Detection Theory
In this study, I created a prototype website with our proposed changes in Axure. We asked users to state whether they would consider purchasing each laptop on the comparison page, then again after they viewed each laptop’s detail page.
When we compared the responses from participants who viewed the original page to participants who viewed our prototype page, we found that there was an increase in the percentage of hit responses. This means that more participants stated they would consider buying a laptop on the comparison page, and then confirmed they would consider buying that same laptop after viewing the specific detail page for that laptop.
We considered this study a success because we were not only able to measure how changing information on a website affects user responses using signal detection theory, but we were also able to increase the percentage of hit responses.
Mak Pryor 