Whether you are a new or long-standing LifeStreet partner, each and every campaign launch requires a training phase to get to a stage where we are hitting your ROAS goals. But for new partners who are unfamiliar with LifeStreet’s process, the first question they often ask is, “What do you mean by training?,” followed by, “How long will it take?”
The training phase is an essential part of setting up your programmatic campaign, and over the years we’ve built a process to ensure campaigns are set up to put marketers on track to maximize their goals. This blog was written to explain LifeStreet’s training phase, why it’s necessary, and what to expect when launching a new campaign.
- What is the campaign training phase, and why is it needed?
- What type of data is collected?
- How are models trained?
- What types of models are used to make a bidding decision?
- Does the training period vary depending on whether you are launching with a custom model?
- When do you start making campaign optimizations?
- What is the optimal test budget?
- If we are already working with some DSPs, should we expect a performance decrease because of cannibalization?
- How does the bidding mechanism work?
- When is the testing phase is over and what happens next?
1. What is the campaign training phase and why is it needed?
When we refer to “campaign training phase,” we are referring to the model building process which includes:
- Collecting enough data for model training
- Model training
- Model evaluation
Our machine learning models are what power our real-time bidding decisions and their ability to find the right user at the right price relies on their predictive accuracy. Models become more accurate with more data and more accurate predictions drive more effective bidding. But it takes time for a model to learn patterns from data, which is why we refer to the first few weeks of a campaign launch as the “training phase.”
2. What type of data is collected?
There are several standard inputs we use for our training data, which include:
- Device-specific features: make and model
- Geographic features: continent, country, ZIP / DMA codes
- Time of day features: hour of day, day of week
- App features: store IDs, categorization, other app metadata
- User-specific attributes (when device ID present): downloads, clicks, purchases and playable interactions
We always use contextual signals–with or without IDFA present–but when we don’t have access to the IDFA, we rely on contextual signals more for our model training.
3. How are models trained?
By using historical data of existing users, our ML models learn patterns about the “right” user in order to make predictions about future users. What makes it the “right” user? That depends on the advertiser’s goals (IAP, retention, engagement, LTV, etc.). But ultimately, advertisers want to maximize ROAS and that’s what our models are trained to do.
Model training is when our ML models learn the rules (or patterns) for how to identify and value a user and can then produce accurate bidding decisions on things like:
- If we show the ad to the user, how likely are they to install?
- Which should we show?
- If the user installs the app, how likely will they be to make an IAP?
- What is their predicted LTV?
Every time a bid request comes in, we run many of these models to make our final decision on whether or not to bid and how much to bid.
4. What types of models are used to make a bidding decision?
We typically use two main types of models. The first type is our ITI (impression to install rate) model which predicts the likelihood a user will make an install if we were to win the ad impression and show them an ad.
The second type is our VAB (Value Added Bidding) model. These are customized models used to predict advertiser-specific KPIs and help us adjust our bids to win users based on their value to the advertiser. The VAB models typically output things like “probability that a given user will become a payer.”
All of our campaigns are launched using the install rate model. Whether we launch a customized model or not depends on how much unattributed data we are collecting from your tracking provider and how much historic payer data you share.
5. Does the training period vary depending on whether you are launching with a custom model?
The training period isn’t a set number of days or weeks, rather it’s the time needed to ensure we have collected enough unique events (e.g. installs) to make accurate predictions about the types of users who will install your app.
If we are launching with a custom model using your IAP data, in addition to install data, we would need to gather sufficient payer data that is predictive enough to hit your KPIs.
There are a few ways to make the training period go faster. Turning on all revenue data via your MMP will speed up the process of collecting payer data. Also, any additional data for custom models will always accelerate the learning curve. Simply put, the more quality data we have to learn from, the faster our algorithms will be able to identify the most valuable users and make meaningful optimizations.
We’re happy to answer questions and get you better acquainted with LifeStreet. Enter your name and email
and one of our customer success managers
will be in touch.
6. When do you start making campaign optimizations?
We start making optimizations right away. During any training phase for an app, we focus on increasing conversion rates to lower eCPIs and as a way to quickly gather enough installs to make meaningful optimization decisions.
Early optimizations are about setting up a level playing field and solid foundation for the campaign to grow. It’s making sure we are spending on top exchanges, and that there is no one app that we are disproportionately spending on.
When we have enough event completions (e.g. installs), or other events that are directional to conversions (e.g. reaching a certain game level), we can start to make more aggressive optimizations.
7. What is the optimal test budget?
The optimal test budget depends on your average eCPIs and how many conversions we can collect per day.
8. If we are already working with some DSPs, should we expect a performance decrease because of cannibalization?
No, you should not expect a performance decrease. We work directly with all the major ad networks and main exchanges and currently see over 25 billion bid requests per day from our supply partners served across thousands of apps. Therefore any overlap is very minimal.
9. How does the bidding mechanism work?
Our bidding decisions govern which users we win and how much we pay for them and those decisions are made using different ML models that work together.
Instead of flat or tiered CPIs, depending on the predicted advertiser value, we choose our bids dynamically for each impression by using an Advertiser Value Score to generate each bid request.
The Advertiser Value Score will dictate whether to bid more or less based on the prediction that an event will occur.
10. When is the testing phase over and what happens next?
There are a couple of phases that need to be completed before we can consider the learning phase to be over.
Hit target CPI: Typically CPIs are higher than their target for the first ~3 days while the model is learning. Once we get enough unique installs, the models learn quickly and CPIs drop to the target rate.
Hit ROAS goal: After CPIs are stable, we want to see consistent payers week over week or look for when we start to hit advertiser’s ROAS goals consistently. We try to tell advertisers it’s about 4 weeks to get to that point, but it can differ depending on the title. With all that said, sometimes we don’t ever reach their ROAS targets.
After ROAS goals are hit, the next step we typically recommend is increasing scale. Depending on the advertiser’s objective, some may not be looking to scale and just want to maintain a specific spend level. But for those who are, we let the campaign stabilize at each new spend level before scaling more.
The training phase will see more performance fluctuations. But this is normal and is because the ML algorithms are training the campaign across different pockets of inventory. With time, the models will collect enough data and optimize towards getting as many conversions as possible within budget and our ability to find users who convert will improve. Programmatic advertising is a financial investment that requires the right partners, data, and patience.