Amazon Payments tested acquisition personalization, reports conversion improvement for only one group
Amazon Payments tested personalized content selection using contextual bandits on Amazon SageMaker AI. In a seven-week A/B test, according to the company, final conversion increased relatively by a higher single-digit percentage for one group; for the other, it did not improve.
Amazon Payments deployed personalized content selection in the customer acquisition process using a multi-objective contextual multi-armed bandit on Amazon SageMaker AI. In a seven-week online A/B test, according to the company, one customer group achieved a relative increase in final conversion in the higher single-digit percentage range. For the other group, no improvement was observed compared to the existing solution. The company attributes this difference to the content, not the model.
The solution uses the Linear UCB (LinUCB) method, which when selecting a variant takes into account both the estimated benefit and the uncertainty of that estimate. Personalization is based on behavioral signals, such as payment behavior and transaction composition. The customer identifier is used only to assign the recommendation to the correct visitor and is not an input to the model.
The system combines the results of three LinUCB models for application initiation, submission, and approval, with the company using approximately equal weights for these steps. The goal is to optimize the entire acquisition process: an increase in the number of initiated applications does not necessarily lead to a higher number of approvals. The article also mentions a repository with code for trying out the method on synthetic data. See the source article for details.
Why it matters
For teams responsible for customer acquisition, this is a way to select content based on visitor behavior while tracking multiple steps all the way through application approval. The differing results for the two groups show that the benefit needs to be verified for individual customer groups, and that personalization alone does not guarantee more effective content.
Relevant practical impact
What this means
For a business
Personalization can increase the final conversion of the acquisition process, but the documented benefit varied between customer groups. Company evaluation must therefore track completion of the entire process as well as the suitability of the offered content for a specific group.
Sales and marketingCheck the original
Event sources
only one source so far · 1 publisher, 0 independent. We count feeds from the same owner only once.