Product Analytics via PostHog
Decision
We track the success of features that we work on via PostHog analytics. For new features, we track initial usage via Slack notifications.
Problems
It is hard to understand the bulk usage of our features, especially by those who are not available for customer research calls. This leads to prioritization driven by management intuition and availability of customers for research calls.
We also don't fully know yet whether the features we build really improve value to our customers.
Context
As an enterprise B2B company, we can afford to talk to most of our customers directly, and successfully do so for user research. We have Google Analytics for marketing analytics.
Options
The following analytics tools were considered:
- June
- Amplitude
- MixPanel
- Google Analytics
- PostHog
- No analytics
Reasoning
PostHog is a bit complicated to get into, but very powerful as it does not include only analytics, but also screen recordings and feature flags which might potentially replace LaunchDarkly for us in the future. It has grouping functionality which is important for analyzing behavior across a full account. Data is stored on a server in Germany, which increases GDPR compliance. It is also reasonably cheap.
Other than Google Analytics, it is focused on Product Analytics which allows better measurement of feature interactions compared with acquisition funnels.
We also looked into June, which was not as GDPR-ready as we had hoped.
Not having any analytics solution would not solve the problem of better understanding for customers we don't talk to.
Consequences
How do we implement this change?
The PR adding this ADR also contains an implementation in the client-dashboard-2. Implementation in other systems happens ad-hoc when needed by the team working there.
Who will implement the change?
The technical change is already part of this PR. The process change of tracking success of newly created features will be first implemented by the Create team.
How do we teach this change?
This PR includes a Learning Journey. There will also be a workshop on using PostHog on a Learning Friday.
What could go wrong?
If we need to change analytics providers, we might lose existing data in the system.
What do we do if something goes wrong?
Changing analytics provider is quick (maybe a day or two of work). Losing analytics data is a bit painful, but since we currently don't use analytics at all, it is still better than not collecting anything in the first place.
What is still unclear?
How will we incorporate other systems, e.g. in the backend, for tracking? The tooling supports this, but we need to think about which event to track where.