Recently, I was analyzing some user generated data in a mobile app. The app was sending content on specific categories to a niche audience, and at the end of each content piece, there was a simple 5 star rating feedback for users to rate the piece.
The assumption that the design team who thought of this was that the feedback data was an objective metric.
Objective metric for Subjective behavior
Unfortunately, the behavior of users and how they understood the content piece is a very subjective topic. By subjective, I mean to say that for two different users, the value they would associate to the usefulness of the same piece varies.
We could always say ceterus paribus, but I would say – “Let’s not fool ourselves here”.
In the world of subjectivity, ceterus paribus doesn’t exist
There could be so many factors that are associated to my giving a 5/5 to a piece v/s 4/5 to the same piece, that in the end, I’d be forced to say it depends, and then list out of a whole new set of variables.
Slicing the Data with new variables
This is a problem. Since, my existing data set does not have these new variable. So, from analyzing – now I am back to collecting data. To be frank, there’s no end to this cycle … collect data, realize that you might want more data and rinse, repeat.
Where do we divine the new rules and new variables? We start from the context.
Ergo, the simple and freeing approach of the answer to the questions we were looking for in the data, sometimes lies partially in the data points, and partially in the context.
Let me illustrate this
Let’s take a fairly popular metric – Bounce rate.
Now, if I were to say that my website’s bounce rate is 100%, what would you say?
Sucks, right??
Now, if I were to tell you that my website is a single page website where I want my users to watch a product launch video. That bounce rate suddenly pales and aren’t you itching to ask me about the number of users who played the video up to a certain point?
If you have been working with Google Analytics, then some of you might even suggest that adding a non-interaction event in GA when the play button is hit.
One more example
Let’s take one more metric. Pages/Session to measure how much content the user is consuming on a site.
Let’s see this in a different spiel. A user is on your site, searching for content and is not able to find what he wants, and keeps visiting different pages. After going through 8-9 pages, he finally gives up and leaves the site. That 8.5 as pages/session now doesn’t seem that sexy now does it?
Understand the context
Therefore staring at a pure data puke may not help. Understanding the context under which that data was collected is as important as going through excel sheets or powerpoint presentations.