When you release a new feature, you'll know when it's super successful: Your users are happily using it and your metrics are moving in the right direction. However, when something isn't working, it's pretty hard to figure out what's causing that, and what you should do next. It can take a lot of time building, tracking, and analyzing your analytics to figure this out, and this often means that it takes a lot more time to iterate. Which is not how product development should work.
I've seen this many times: a team works on something they have high confidence in, and after it's released they don't see any results. They spend so much time to figure out what's causing it, that by the time they've come up with a next iteration, most of them are already working on the "next big thing". I believe good software is created when teams iterate fast and often, and this is the opposite of that.
Introducing Discover-Use-Love
To make things simple, and effective, I like to use a simple framework for this, called Discover-Use-Love. It tries to answer three basic questions with analytics and has clear follow-up actions. This also keeps the amount of analytics you need to add when building something new at a minimum.
I've seen multiple versions of this in previous companies, but I don't know who invented it and I can't find any documentation of this on the Internet, which is why I'm writing it here.
While you're building the first version of a feature, you include analytics to answer the following three things:
1. Can users discover this feature?
For this, you track whether users have clicked or tapped on the feature. If users can't discover a feature, they will definitely not be able to use or love it. So if you see that rarely anyone is discovering it, it might mean that the feature is good, but not located in the right place in the interface or flow. Your next iteration should be about the location of this feature.
This happens more than you'd might think. I've often tried to make features better while not looking at how users were discovering them, which wasn't impacting anything. Because a user doesn't care how good your feature is if they can't find it.
2. Do users use this feature?
If people can discover a feature, the next question you want to answer is whether they've used it. You can track this by tracking whether people reach the final step of the feature. If they don't reach that, it means that they couldn't figure out the way the feature works, or they expected something different. Your next iteration should be about the way the feature works.
I recently added a sound option to a product I'm working on. A lot of users had requested it, so I assumed after building it, it would be an instant hit. When I released it, I saw that many people clicked it, but very few people actually turned on sound afterwards. The reason? It didn't have enough options to achieve my users goals. I had to get back to the drawing table and design it in a different way.
3. Do people love this feature?
So, people are finding and using the thing, great. Is there anything left to desire? There is. You want to figure out whether people's behavior actually changes after using a feature: Do they retain longer? Are they more likely to pay? Are they creating more projects? Are they using your product more often?
If none of their behavior changes after using a feature it probably means that they don't care about it too much. The metric you use here is the one you value most in your company or your team. This is the metric that you intended to move when building the new functionality in the first place. For example, at VEED, we want to increase the number of videos that people create. So even if a feature is being used, we want to see if these users are actually creating more videos after they've used it.
This is trickiest one in my opinion, if people don't love the feature, you can do multiple things:
Improve the feature so much that it will change users' behavior. This is very hard, but not impossible. You should only do this for features that you believe are the differentiator of your product, else it's probably not worth it: because you could spend your time on building something else...
Leave it as-is, and work on something else. You should do this in most cases. If I see something is working, but not really changing the game, it probably never will and it's not worth iterating much on. It's better to spend your time on features that are moving your most important metrics.
When to use this framework?
Whenever you release something new! That being said, this framework doesn't answer all questions - and that's on purpose. It's a simplification of reality to make quick decisions and to keep things easy. There are scenarios in which it won't serve you, so always handle it with care.