Data Analytics experiments
Data analytics is an interesting area! Not just a passing fad but this will help you take the right actions based on the numbers. If you are numbers person then this article is for you. Here are some of the things I learnt in my journey with customers and their needs for analytics. Prior to sharing analytics with customers, it would be great if you can share it internally within your company and get some feedback. Learn more about internal product analytics
Which personas of the analytics data?
Managers might want to measure the performance of their team. Team members might want to keep track of their own performance. These can help you measure outcomes (business or product). You might want to expose some analytics to managers whereas some to the team.
What metrics matter to them the most?
Depending on the industry, goals and maturity, our customers may want analytics around different metrics. Let’s say you want to improve efficiency and say serve more number of customers, then you ’d want to know the number of customers they serve now. You need to interview them and understand their goals. That way, you ’d understand what matters to them.
Low-tech BETA experiments
Hypothetically, you can measure multiple permutations and combinations of all data points. But that is not really what we want for our customers. This is where our low-tech pilot BETA experiments come in. You might have certain assumptions about what they want and they might have theirs. It is important to verify our assumptions. We sent monthly analytics to our customers and then interviewed them for the same. We had a baseline idea of what the customer would want. We used Tableau to generate reports and applied changes based on their feedback.
It is important to understand the frequency at which your customer wants the analytics. Do they want it real-time or a delay of few days is fine? This will help you decide whether to use a real-time database or BI platform like Snowflake, wherein there is lag in the data update. It was a very a scrappy way of doing the experiment but TOTALLY worth it because we learnt a LOT. We didn’t even have a full engineer to work on this but that’s where the collaboration piece comes in. We used the engineer ONLY when needed.
We also realized clean data isn’t the norm. In other words, we (both customers and us) realized that user errors were there and as a result, sometimes we had errors in analytics. So, when you think of the workflow, the data entry should have validations in order to have the correct analytics.
What was everyone’s reactions to the reports?
These low-tech pilots helped us understand what was useful and what wasn’t. The customers too started understanding what was good for their business. Infact, they reshuffled their teams. Sometimes, analytics can be perceived to be a competition between team members. This is not the intent at least for the companies we experimented with. There were 2 types of customers - 1. Some of them had 0 idea of numbers. 2. Some of them used inefficient methods like manually tracking it in an excel sheet. In either case, they wanted to make the MOST of their resources. So when we ran this pilot with them they were happy with what they saw. They realized the importance of having clean data as well.
Final Thoughts
Data analytics is the basis of AI/ML. As product managers, when we build the whole workflow, we need to think of the data that our users will be entering. Validations are important so that we have QUALITY data. If you really want to know the correct analytics around structured data especially (That are usually user-entered) then build that into your workflow and application. Also, if your ultimate goal is to use AI/ML to add predictions then this is the key.