Case Study | Chip Rewards in POP! Slots
What happens when you faucet your monetizing currency as a Reward in your Loyalty Store?
We've got the answer.
Abstract
We recently collaborated with the POP! Slots Economy team to test rewards that grant POP! Slots chips. Our goal was to show that we could (a) increase engagement with the rewards store (and experience at least 5% consumption rate, which is the corporate OKR), and (b) make LPs relevant to more players, (c) all without negatively impacting near- and mid-term revenue.
Method
-
We divided ~45K POP players (the test group) into 13 different sub-groups based on their POP game level.
-
Each of these groups saw ONE new reward in the rewards store – a chip reward sized appropriately for their game level.
-
The amount of chips were worth roughly $3 worth of chips for that sub-group.
-
Each player could purchase the reward up to 2x per week.
-
We ran the test for 30 days. During that time, there were just one email and three in-app inbox messages sent to the test group mentioning the reward.
-
Each level group was then compared to an equivalent control group that did NOT see any chip rewards.
Results
-
Success!
-
Blended across all groups, we saw a roughly 11% consumption of these rewards, far higher than any reward we've ever offered (and higher than our goal of 5%)!
-
We observed no cannibalization. In fact, there was a positive impact on monetization. Purchasers of the Chip Reward saw the highest 30-day LTV (cumulative ARPU).
-
We also saw a positive impact on engagement. Purchasers saw increases in sessions, session length, spins, coin-in, and average bet.
-
Purchasing chip rewards had seemingly no impact on the rate of consumption of the rest of the rewards catalog, which is a positive and expected outcome.
Let's continue to be smarter and better together!
MATERIALS & METHODS
At playAWARDS, we have long since desired to include in-app benefits as rewards in our rewards store. Around 50% of players in our games have stated in surveys that they find our real-world rewards a relevant motivator to play, and players representing around 25% of our revenue have stated they play our games ONLY for rewards.
Our Motivations
While those are encouraging numbers, it still leaves 50% of players off the table, rendering Loyalty Points for those players irrelevant. When members of loyalty programs find value in their points, and actually engage and consume them, their lifetime value goes up by as much as 50%.
So if we create a purpose for Loyalty Points for the half of our DAU that doesn't engage with real-world rewards, then we double the effectiveness of the loyalty program and double the value for the apps. The best candidate for rewards that are relevant to 100% of our audience is in-game benefits. We conducted this study to ensure we could roll out in-game benefits without cannibalizing revenue. In other words, increase long-term value without risking short-term value.
In this test, we offered players the opportunity to purchase Chip rewards using their Loyalty Points in the Reward Store. playAWARDS collaborated very closely with the POP! Slots Economy team in structuring, conducting, and analyzing the test. The goal was to answer a simple question...
Would chip rewards have a positive, neutral, or negative impact on monetization and engagement?
About the Test
The test group (45,895 players) and control group (45,464 players) were divided between 13 different sub-groups based on their player level.
Players must have met the following criteria:
-
Active in POP in trailing 30 days
-
Not hosted
-
Have access to rewards (i.e., not in hold out group)
-
Mix of tenure and authentication type
-
10% of active audience
In the next section, we'll cover some of the key results. Some of the absolute values of sensitive data will be obfuscated.
Test Groups
RESULTS
About an even percentage of test and control group players were active and viewed the reward store during the test period. 11% of the test group players purchased the Chip rewards overall. This was more heavily skewed toward the higher level groups. In the higher level groups, it was as high as 26%. The same proportion of active players purchased rewards during the test period, but the Chips drove more players to purchase rewards.
11% of the test group players purchased the Chip rewards.
Chip Rewards Purchased
12.52% higher than before the test
2.5% higher than before the test
14.34% higher than before the test
10.26% higher than before the test
ARPDAU among the test group was 12.5% higher during the test period vs. before, compared to just 11.4% among the control group. Number of sessions were 2.5% higher vs. the 2.18% among the control group. Coin-in went up 14.3% among the test group, where the control group only went up 5.2%. Average bet was 10.3% higher for the test group vs. just 7.6% for the control.
The test group had higher monetization and higher engagement during the test period.
11.42% higher than before the test
2.18% higher than before the test
5.23% higher than before the test
7.61% higher than before the test
Test Group
(Had Chip Rewards)
Control Group
(No Chip Rewards)
ARPDAU
# of Sessions
Coin-In
Average Bet
This also holds true when you isolate the analysis to only those who actually purchased one of the Chip rewards. When comparing Chip reward purchasers’ monetization during the test period against the period before the test, no cannibalization was seen. The purchasers’ average daily ARPDAU was higher than ARPDAU before the test period. This is also true among all other meaningful KPIs, including number of sessions, session length, spins, coin-in, and average bet.
It's worth noting that non-reward purchasers ARPDAU is lower as they are a "self-selected group." In other words, players who are in that group must generally be less engaged to not see and purchase the test rewards.
11.4% higher during compared to pre-test period
6.38% higher during compared to pre-test period
15.4% higher during compared to pre-test period
Percent Difference in ARPDAU From Pre-Test Period to During
Control Group
Test Group
Non-Reward Purchasers
Test Group
Reward Purchasers
CONCLUSIONS & NEXT STEPS
It is our conclusion that overall, chip rewards in POP can be confidently rolled out to a broader audience with a similar framework without cannibalizing revenue. As the high conversion rate suggests, the amount of chips can perhaps be even more modest and still drive significant engagement.
Overall
It was our hope to observe NO impact on monetization, so it was a pleasant surprise to instead observe a POSITIVE impact. But how could this be? If we were increasing faucets, why didn’t we see cannibalization?
We generally saw an increase in engagement, while all other metrics remained unaffected. It was concluded that the increased engagement is the cause of the increase in monetization.
Why a Gain?
Different Chip Amounts
In the future, we would like to run additional tests around various chip amounts to further home in on precisely how many chips relative to an individual’s economy is ideal for providing a positive-to-neutral impact.
Do Non-monetizing Currencies Provide the Same Impact?
In the future, we’d like to test a variety of non-monetizing currencies to see if they have the same impact. On the one hand, non-monetizing currencies feels “safer.” On the other, it doesn’t excite the player as much as the primary currency (and thus might not increase engagement), and additionally, it might not be as motivating for the player, thus not drive as much rewards engagement.
Program Maturity
In the future, we’d like to mature this offering into more of a program. That means offering a compliment of currencies, amounts, frequencies, etcetera. This will inevitably require more testing around what the ideal complement of in-app benefit rewards should be offered.
Segmentation
Ideally, we would prefer to also segment based on myVIP Tier because it speaks to the magnitude of the player. This could have significant impact on the ideal chip amount.
Merchandising
During this test, the rewards were very lightly merchandised. There is a lot we could do to provide more exposure and thus drive more engagement, including a presence in the chip store, on the out-of-funds blocker, and on promotional blockers. In theory, since these have been shown to be accretive, then in the long-term we wouldn’t expect cannibalization from this merchandising either, but in fact, the opposite. Additionally, we have controls to limit the amount of consumption from players on a daily, weekly, monthly basis that we could leverage to ensure optimal controls are in place. But it merits further testing.
Next Steps
If you have any questions about this analysis, want to dive deeper, or want to run a test of your own, reach out to your CSM!