LTV FAQ
Data requirements
What data is required for AlgoLift to predict LTV?
We need user-level attribution data and revenue data to model LTV. Providing engagement data is highly suggested in order to supply more signals that might be indicative of user value.
What value does AlgoLift offer if there is limited data history in the case of a soft launch or unestablished app?
AlgoLift provides the most accurate LTV projections when we have 1 year+ of data.
- This enables our models to observe temporal changes in revenue and user behavior and to fully understand how cohorts mature.
- We can provide projections with less historical data, but this restricts the projection windows based on the amount of data available to train our models.
- This table gives an overview of how AlgoLift releases new projection windows to clients based on the historical data available for modeling.
LTV Modeling Methodology
Can you share more about your general modeling framework and how you incorporate machine learning?
Our models combine machine learning techniques, which do the heavy lifting in terms of finding complex patterns and relationships between user demographic information, engagement and purchase behavior, and value. Secondary models then further predict LTVs into the future without a reliance on outdated cohort LTV curves or similar methods.
How does the size of inputs affect the efficacy of your models (e.g. install count, number of customers)?
Predictions do typically become more accurate with higher input volume. In order to reach our typical level of accuracy we need approximately 20,000 installs and 1-2% D7 payer conversion per week.
How do your models handle users who are active, but who only have one purchase?
We understand some existing LTV models use a RFM (recency, frequency, monetary) approach. We do not use RFM-based models and so we do not require a minimum number of purchases to operate. We will forecast LTV for every user regardless of their purchase behavior to date, this also includes users that have not yet made a purchase (“non-payers”).
How do your models account for changes to the product, competition, marketing strategy, etc?
This is dependent on the effect of the change. Updates whose effects are quickly reflected in early user behavior are better picked up by our models than those whose effects are recognized over a longer period of time.
For example, a change that decreases retention after D30, but has little effect on earlier user behavior will only be picked up by our models after D30 and only if the new user behavior persists. Furthermore, AlgoLift is able to ingest app version at the user-level as an input as well as any in-app engagement.
Do you need to be notified of these types of upcoming product changes?
No, we do not need to be notified. Our models are developed to adapt to changes in user behavior regardless of whether we're notified of the change.
It’s important to understand that while our models adapt to changes in user behavior it does still take time for the models to learn any realized impact of a given change. The biggest improvements to projections will come after the models see updated conversion and retention behavior; this may vary based on app category, revenue type, etc.
Projection Accuracy
How do you ensure the quality of your projections?
We use a threefold approach to ensuring we are delivering the best projections possible:
Periodic back-testing of our models across all of our client portfolio. Back-testing refers to making projections using data up to a certain date in the past, then comparing them to actuals.
We maintain secondary models that provide cohort-level LTV projections. While not as flexible or useful as user-level models, these still tend to be quite accurate and serve as effective QA tools to which we can compare our aggregated user-level projections and find any discrepancies. This mitigates the risk of more granular user-level modeling.
We maintain a suite of anomaly detection tools that prevent questionable projections, or projections based on erroneous input data, from being delivered.
How long does a cohort need to mature for the LTV projections to be actionable?
When using our LTV projections for Intelligent Automation we leverage these projections immediately rather than wait for a cohort to mature. We take the approach that it’s better to use projections earlier and be directionally accurate than wait for a cohort to mature and lose the benefit of the early feedback on recent cohort behavior. However - for the most accurate predictions we’d recommend waiting 7 days for a cohort to mature. For high level long-term ROAS reporting or any other less time sensitive use-cases, wait 14 to 30 days.
At what cohort age(s) are projections updated?
Each user receives an updated projection daily.
Can you provide prediction intervals for your LTV projections?
We do not provide prediction intervals. The main reason for this is that they would not properly aggregate up to a cohort level that way a single expected value of LTV does, and thus are of little practical value.
Internally, when estimating LTV for a specific dimension (e.g. sub publisher) we do incorporate estimates of the range of uncertainty in the pLTV of future acquisitions.
Can you provide ongoing reporting of historical projection accuracy?
The preferred way to view historical errors varies by client. We deliver the daily user-level projections to clients through Amazon S3 to enable them to measure accuracy with their preferred approach.
Churn Predictions
What types of churn can you predict?
As long as we receive the event types that define retention, then we can predict various types of churn including:
- Payer churn: Probability of no further purchases in the next X days
- Subscriber retention: Probability of someone renewing a subscription at the next opportunity
- Explicit churn: Probability the user will complete a known account closing or cancellation event
- Custom: login retention, app retention, etc