I recently examined user-generated data from a mobile app that delivered content to a niche audience. Users rated each piece with a five-star system, which the design team treated as an objective measurement. The problem: user behaviour is inherently subjective.
Objective Metrics for Subjective Behaviour
Different users assign varying value to identical content. In the world of subjectivity, ceteris paribus does not exist. Countless variables influence whether a user rates content 5/5 versus 4/5 — their mood, prior expectations, what they read before, the device they are using.
Two Classic Examples
Bounce Rate. A 100% bounce rate appears catastrophic until you learn the site is single-page, designed specifically for video viewing. Every user who watches the video and leaves registers as a bounce. Context transforms the interpretation entirely.
Pages per Session. Eight pages visited might indicate strong interest — or it might indicate that a user searched unsuccessfully through eight pages before giving up and leaving the site. The number is identical. The meaning is opposite.
The Core Principle
Understanding the context under which data was collected is as important as going through excel sheets or presentations. Data analysis requires both metrics and contextual knowledge. One without the other leads to confident wrong conclusions.
When analysis reveals missing variables, the cycle is: gather data, identify gaps, and repeat. Start with context. Work forwards to measurement.