In recent years, e-commerce has grown exponentially. We as a society have embraced this new way of shopping due to its ease and immediacy. BBVA is no exception; in fact, 79% of our sales were digital last year.1
In this context, where information, supply, and choice are multiplying, it makes sense to adopt an approach based on personalizing commercial offers with AI. Products and services tailored to each customer’s needs, along with when and how they need them, will be more successful than general offers that are not personalized. (View Highlight)
For decades, commercial campaigns were designed based on expert criteria and limited datasets from which target audiences were identified. This process involved creating and launching unscalable, static campaigns.
A couple of years ago, if we wanted to launch a campaign to offer an account for minors, we would have used the bank’s available data to find insights that would help us identify the target audience for that product. For example, we could have looked for people aged 20 to 40 with direct debit bills from schools, daycare centers, or extracurricular activities. We would have used this information to configure the target audience for the commercial offer and create the advertising message. (View Highlight)
This approach has limitations as it fails to consider individual factors. To address this, BBVA has developed a commercial product recommendation system that provides real-time offers. These messages are tailored using data from various sources and machine learning models. (View Highlight)
The default methods to discover the best message for a specific audience are A/B testing and market research. Today, technology enables us to move from a “one size fits all” approach to customizing the message for each customer, channel, and moment. Furthermore, we can continually experiment and use the results to refine our offers (or their absence) and provide the best service to all customers. (View Highlight)
We collect data from various sources and carefully process it using feature engineering. This technique helps us convert the data into useful features, such as the average and maximum amount spent by the customer in the supermarket in the last month. This enables our artificial intelligence models to better comprehend our customers’ financial situation, their activity in our mobile app, and their interests. Over time, they can even discover new characteristics and patterns based on data analysis. (View Highlight)
For these models to function across different countries, we utilize standardized global data sources. This means that all data used have the same format in all the bank’s operating regions.This standardized information, along with records of products that customers have previously obtained from BBVA, is utilized by an AI model to learn and identify customer characteristics indicating interest or need for specific products. (View Highlight)
In a second step, we choose features based on their “mutual information,” representing the relationship between each feature (for example, the total expenditure on bars and restaurants) and the target variable (e.g., opening a “Metas” saving account). We then eliminate features with a low coefficient of mutual information, as they are less relevant for training the model. (View Highlight)
Finally, in this first phase of data preparation, it is important to balance the dataset so that the model makes more accurate and balanced predictions for all types of products.
Our recommendation system offers a range of commercial products, including credit cards and savings accounts. Sales volume varies significantly for each product, which is evident in the historical data used to train the model. This variation can create an “unbalanced dataset,” causing the model to favor and predict the more prevalent product types. For instance, if there are far more debit card sign-ups than savings accounts, the model may prioritize predicting debit card sign-ups and overlook savings accounts. (View Highlight)
In order to address this issue, we utilize a library known as CleanLab which assists us in identifying and eliminating noisy observations from the most represented classes, thereby achieving a more balanced dataset. (View Highlight)
After preparing the data, the next step is to train a multiclass classification model to predict the types of products that a customer might need. In this instance, we utilize an algorithm based on decision trees’ models called LightGBM, which is highly efficient for this task. (View Highlight)
In the training process, we use the Halving Search method to optimize the hyperparameters. Hyperparameters are internal settings of the model that can be adjusted to enhance its performance (training speed) and behavior (model accuracy). Examples of hyperparameters include the learning rate and the number of iterations. (View Highlight)
We use several metrics to evaluate model performance, both online and offline. We measure accuracy, which tells us how many of the model’s predictions are correct. Recall measures how many true cases the model identifies. To get a balanced view of model performance, we use F1-Score, which combines accuracy and recall. These metrics are also calculated in their weighted versions, meaning we consider the relative importance of different product classes. (View Highlight)
One of the primary goals of the evaluation phase is for all teams involved in the development, including technical and business teams, to comprehend which data variables influence the recommendation of specific products and how they do so, both individually and globally. To achieve this, we employ explainability techniques:
• Feature Importance. This tool helps us identify the most important variables for the model. In essence, it reveals which data carries more weight when the model is learning to make predictions. For instance, if we’re recommending a corporate credit card, the feature importance can indicate that the payroll balance or social security expenses are highly influential variables.
• Shapley values. The SHAP values are useful for understanding how each variable impacts a model’s predictions for each product. They provide detailed explanations of how each characteristic influences the model’s decision. For instance, they can demonstrate how a customer’s monthly income affects the likelihood of receiving a credit card recommendation. (View Highlight)
