High Display Value, Low Consumption: Analyzing Hot Topics Discrepancy
How to act strategically despite limited time and budget?
Overview
This is an evaluative project I led at Helo@ByteDance. From the data analysis, it has been observed that there is a high display value for hot topics on popular feeds, but the actual consumption or engagement is poor. This project aims to investigate the reasons behind this data discrepancy.
Some information is removed or modified due to confidentiality.
-
Role: UX Research
-
Research Method: User Interview
-
Time: 5 days
Context
Helo is a social media app targeting the Southeast Asian and Brazilian markets. From the data analysis, it has been observed that there is a high display value for hot topics on popular feeds, but the actual consumption or engagement is poor. This project aims to investigate the reasons behind this data discrepancy.
Research Goals:
-
Explore the user motivations and expectations when interacting with hot topics on popular feeds.
-
Identify potential design improvements to enhance user engagement with hot topics on popular feed.
-
Evaluate the effectiveness of current algorithms or mechanisms used for selecting and displaying hot topics.
As depicted in the image, the highlighted section represents a featured content on the app's homepage, visible to nearly every user upon app launch (display rate of 100%). However, the click-through rate stands at only around 3%.
Homepage
Methods
As a UX researcher, I frequently need to strike a balance between the rigor and speed. In situations of extremely tight time constraints, I have employed the following approaches to swiftly achieve "good enough" results.
How did I manage this task in 5 days?
Completing the entire process of interview design, user recruitment, conducting interviews, summarizing analysis, and finalizing the report within five days was challenging. However, I remained determined to ensure research rigor to the best of my ability. As a result, I implemented two approaches:
-
Collaborate with colleague: Fortunately, another researcher colleague was also conducting qualitative research and interviewing users. After careful consideration of user profiles, we determined that the participants from her study could be utilized for my interviews as well, serving as my interview sample. This significantly reduced the time required for user recruitment.
-
Decrease the project's scope: I made necessary adjustments to the project's scope. Initially, the product manager wanted UX research to be conducted simultaneously for an additional page. After evaluating priorities, we decided to postpone the lower-priority project, enabling us to focus on completing the current project within the extremely tight timeframe.
Recruit. How?
I have decided to recruit two groups of participants based on their behavior, with three participants in each group:
Group 1:
The hot topics are displayed but NOT clicked.
They read trending content every time they open the app.
Group 2:
The hot topics are displayed and clicked.
They read trending content every time they open the app.
Iteration
After obtaining user data, I organized a meeting involving the product manager, UI team, and algorithm team. Following discussions, we derived the following issues and their corresponding solutions.
Key Finding#1: Weak recognition of personalized recommendations
Users are not aware that these topics are carefully selected and recommended by the system based on their preferences.
Solution - change in the algorithm
The core issue exposed by the first key insight is that the algorithm lacks sufficient data, resulting in inadequate recommendation accuracy. After discussions, we ultimately realized that instead of displaying recommendations with low accuracy, it would be more effective at this stage to directly showcase the most popular content on the platform as a whole. This led to a change in the algorithm for recommending content.
Key Finding#2: Incomplete display of topic names and misunderstanding
Through interviews, we discovered significant differences in the perception of whether the showcased trending topics were clickable buttons between the two participant groups. Additionally, we found that a majority of Indonesian users were not accustomed to horizontally scrolling through app pages.
Solution - UI adjustments
As a solution, the UI team adjusted the horizontal scrolling interaction across all app pages.
For additional findings and learnings, please contact zhouquincey@gmail.com.
Learnings
-
In a time-constrained research task, it is essential to confirm the priority of research questions with the project stakeholders, establish the hierarchy of importance, and appropriately streamline the questions.