In 2016, a retail bank sold several products (mortgage account, savings account, and pension account) to its customers. It kept a record of all historical data, and this data is available for analysis and reuse. Following a merger in 2017, the bank has new customers and wants to start some marketing campaigns.
The budget for the campaigns is limited. The bank wants to contact a customer and propose only one product.
The marketing department needs to decide:
From the historical data, we can train a machine learning product-based classifier on customer profile (age, income, account level, ...) to predict whether a customer would subscribe to a mortgage, savings, or pension account.
Table of contents:
This notebook requires a mathematical background.
If you're new to optimization, following the online free Decision Optimization tutorials (here and here) might help you get a better understanding of Mathematical Optimization.
This notebook is part of Prescriptive Analytics for Python
It requires either an installation of CPLEX Optimizers or it can be run on IBM Watson Studio Cloud (Sign up for a free IBM Cloud account and you can start using Watson Studio Cloud right away).
The purpose of this Notebook is not to provide a perfect machine learning model nor a perfect optimization model. The purpose is to show how easy it is to mix machine learning and CPLEX data transformations by doing a forecast, then getting fast and reliable decisions on this new data.
This notebook takes some time to run because multiple optimization models are solved and compared in the part dedicated to what-if analysis. The time it takes depends on your subscription type, which determines what optimization service configuration is used.
Prescriptive analytics (decision optimization) technology recommends actions that are based on desired outcomes. It takes into account specific scenarios, resources, and knowledge of past and current events. With this insight, your organization can make better decisions and have greater control of business outcomes.
Prescriptive analytics is the next step on the path to insight-based actions. It creates value through synergy with predictive analytics, which analyzes data to predict future outcomes.
Prescriptive analytics takes that prediction to the next level by suggesting the optimal way to handle that future situation. Organizations gain a strong competitive advantage by acting quickly in dynamic conditions and making superior decisions in uncertain environments.
With prescriptive analytics, you can:
In [1]:
import pandas as pd
import matplotlib.pyplot as plt
from matplotlib.colors import ListedColormap
%matplotlib inline
In [2]:
known_behaviors = pd.read_csv("https://raw.githubusercontent.com/vberaudi/utwt/master/known_behaviors2.csv")
known_behaviors.head()
Out[2]:
In [3]:
a = known_behaviors[known_behaviors.Mortgage == 1]
b = known_behaviors[known_behaviors.Pension == 1]
c = known_behaviors[known_behaviors.Savings == 1]
print("Number of clients: %d" %len(known_behaviors))
print("Number of clients predicted to buy mortgage accounts: %d" %len(a))
print("Number of clients predicted to buy pension accounts: %d" %len(b))
print("Number of clients predicted to buy savings accounts: %d" %len(c))
In [4]:
known_behaviors["nb_products"] = known_behaviors.Mortgage + known_behaviors.Pension + known_behaviors.Savings
In [5]:
abc = known_behaviors[known_behaviors.nb_products > 1]
print("We have %d clients who bought several products" %len(abc))
abc = known_behaviors[known_behaviors.nb_products == 3]
print("We have %d clients who bought all the products" %len(abc))
In [6]:
products = ["Savings", "Mortgage", "Pension"]
It's possible to use pandas plotting capabilities, but it would require a new version of it. This Notebook relies on matplotlib as it is present everywhere.
In [7]:
def plot_cloud_points(df):
figure = plt.figure(figsize=(20, 5))
my_cm = ListedColormap(['#bb0000', '#00FF00'])
axes = {p : ('age', 'income') if p != "Mortgage"else ('members_in_household', 'loan_accounts') for p in products}
for product in products:
ax = plt.subplot(1, len(products), products.index(product)+1)
ax.set_title(product)
axe = axes[product]
plt.xlabel(axe[0])
plt.ylabel(axe[1])
ax.scatter(df[axe[0]], df[axe[1]], c=df[product], cmap=my_cm, alpha=0.5)
In the following visualization, you can see the behavior of the 2016 customers for the three products. The green color indicates that a customer bought a product; red indicates a customer did not buy a product. The depth of the color indicates the number of purchases or non-purchases.
In [8]:
plot_cloud_points(known_behaviors)
We can see that:
In [9]:
known_behaviors.columns
Out[9]:
Let's use the following columns as machine-learning features:
In [10]:
cols = ['age', 'income', 'members_in_household', 'loan_accounts']
In [11]:
X = known_behaviors[cols]
ys = [known_behaviors[p] for p in products]
In [12]:
X.head()
Out[12]:
We use a standard basic support gradient boosting algorithm to predict whether a customer might by product A, B, or C.
In [13]:
from sklearn import svm
from sklearn import ensemble
In [14]:
classifiers = []
for i,p in enumerate(products):
clf = ensemble.GradientBoostingClassifier()
clf.fit(X, ys[i])
classifiers.append(clf)
In [15]:
unknown_behaviors = pd.read_csv("https://raw.githubusercontent.com/vberaudi/utwt/master/unknown_behaviors.csv")
In [16]:
for c in unknown_behaviors.columns:
assert c in known_behaviors.columns
In [17]:
to_predict = unknown_behaviors[cols]
In [18]:
print("Number of new customers: %d" %len(unknown_behaviors))
In [19]:
import warnings
warnings.filterwarnings('ignore')
In [20]:
predicted = [classifiers[i].predict(to_predict) for i in range(len(products))]
for i,p in enumerate(products):
to_predict[p] = predicted[i]
to_predict["id"] = unknown_behaviors["customer_id"]
In [21]:
offers = to_predict
offers.head()
Out[21]:
In [22]:
plot_cloud_points(offers)
The predicted data has the same semantic as the base data, with even more clear frontiers:
The training data contains customers who bought more than one product, let's see our prediction
In [23]:
a = offers[offers.Mortgage == 1]
b = offers[offers.Pension == 1]
c = offers[offers.Savings == 1]
print("Number of new customers: %d" %len(offers))
print("Number of customers predicted to buy mortgages: %d" %len(a))
print("Number of customers predicted to buy pensions: %d" %len(b))
print("Number of customers predicted to buy savings: %d" %len(c))
In [24]:
to_predict["nb_products"] = to_predict.Mortgage + to_predict.Pension + to_predict.Savings
abc = to_predict[to_predict.nb_products > 1]
print("We predicted that %d clients would buy more than one product" %len(abc))
abc = to_predict[to_predict.nb_products == 3]
print("We predicted that %d clients would buy all three products" %len(abc))
The goal is to contact the customers to sell them only one product, so we cannot select all of them.
This increases the complexity of the problem: we need to determine the best contact channel, but also need to select which product will be sold to a given customer.
It may be hard to compute this. In order to check, we will use two techniques:
In [25]:
offers.reset_index(inplace=True)
In [26]:
# How much revenue is earned when selling each product
productValue = [200, 300, 400]
value_per_product = {products[i] : productValue[i] for i in range(len(products))}
# Total available budget
availableBudget = 25000
# For each channel, cost of making a marketing action and success factor
channels = pd.DataFrame(data=[("gift", 20.0, 0.20),
("newsletter", 15.0, 0.05),
("seminar", 23.0, 0.30)], columns=["name", "cost", "factor"])
offersR = range(0, len(offers))
productsR = range(0, len(products))
channelsR = range(0, len(channels))
In [27]:
gsol = pd.DataFrame()
gsol['id'] = offers['id']
budget = 0
revenue = 0
for product in products:
gsol[product] = 0
noffers = len(offers)
# ensure the 10% per channel by choosing the most promising per channel
for c in channelsR: #, channel in channels.iterrows():
i = 0;
while (i< ( noffers // 10 ) ):
# find a possible offer in this channel for a customer not yet done
added = False
for o in offersR:
already = False
for product in products:
if gsol.get_value(index=o, col=product) == 1:
already = True
break
if already:
continue
possible = False
possibleProduct = None
for product in products:
if offers.get_value(index=o, col=product) == 1:
possible = True
possibleProduct = product
break
if not possible:
continue
#print "Assigning customer ", offers.get_value(index=o, col="id"), " with product ", product, " and channel ", channel['name']
gsol.set_value(index=o, col=possibleProduct, value=1)
i = i+1
added = True
budget = budget + channels.get_value(index=c, col="cost")
revenue = revenue + channels.get_value(index=c, col="factor")*value_per_product[product]
break
if not added:
print("NOT FEASIBLE")
break
In [28]:
# add more to complete budget
while (True):
added = False
for c, channel in channels.iterrows():
if (budget + channel.cost > availableBudget):
continue
# find a possible offer in this channel for a customer not yet done
for o in offersR:
already = False
for product in products:
if gsol.get_value(index=o, col=product) == 1:
already = True
break
if already:
continue
possible = False
possibleProduct = None
for product in products:
if offers.get_value(index=o, col=product) == 1:
possible = True
possibleProduct = product
break
if not possible:
continue
#print "Assigning customer ", offers.get_value(index=o, col="id"), " with product ", product, " and channel ", channel['name']
gsol.set_value(index=o, col=possibleProduct, value=1)
i = i+1
added = True
budget = budget + channel.cost
revenue = revenue + channel.factor*value_per_product[product]
break
if not added:
print("FINISH BUDGET")
break
print(gsol.head())
In [30]:
a = gsol[gsol.Mortgage == 1]
b = gsol[gsol.Pension == 1]
c = gsol[gsol.Savings == 1]
abc = gsol[(gsol.Mortgage == 1) | (gsol.Pension == 1) | (gsol.Savings == 1)]
print("Number of clients: %d" %len(abc))
print("Numbers of Mortgage offers: %d" %len(a))
print("Numbers of Pension offers: %d" %len(b))
print("Numbers of Savings offers: %d" %len(c))
print("Total Budget Spent: %d" %budget)
print("Total revenue: %d" %revenue)
comp1_df = pd.DataFrame(data=[["Greedy", revenue, len(abc), len(a), len(b), len(c), budget]], columns=["Algorithm","Revenue","Number of clients","Mortgage offers","Pension offers","Savings offers","Budget Spent"])
The greedy algorithm only gives a revenue of \$50.8K.
In [31]:
import sys
import docplex.mp
There are two ways to solve the model:
In [35]:
url = None
key = None
docplex solve methods take various arguments: you can pass the api and key at each function call or you can put them at model declaration with a context.
In [36]:
from docplex.mp.context import Context
context = Context.make_default_context()
context.solver.docloud.url = url
context.solver.docloud.key = key
context.solver.agent = 'docloud'
In [37]:
from docplex.mp.model import Model
mdl = Model(name="marketing_campaign", checker='on', context=context)
channelVars, represent whether or not a customer will be made an offer for a particular product via a particular channel.totaloffers represents the total number of offers made.budgetSpent represents the total cost of the offers made.
In [38]:
channelVars = mdl.binary_var_cube(offersR, productsR, channelsR)
In [39]:
# At most 1 product is offered to each customer
mdl.add_constraints( mdl.sum(channelVars[o,p,c] for p in productsR for c in channelsR) <=1
for o in offersR)
# Do not exceed the budget
mdl.add_constraint( mdl.sum(channelVars[o,p,c]*channels.get_value(index=c, col="cost")
for o in offersR
for p in productsR
for c in channelsR) <= availableBudget, "budget")
# At least 10% offers per channel
for c in channelsR:
mdl.add_constraint(mdl.sum(channelVars[o,p,c] for p in productsR for o in offersR) >= len(offers) // 10)
mdl.print_information()
In [40]:
obj = 0
for c in channelsR:
for p in productsR:
product=products[p]
coef = channels.get_value(index=c, col="factor") * value_per_product[product]
obj += mdl.sum(channelVars[o,p,c] * coef* offers.get_value(index=o, col=product) for o in offersR)
mdl.maximize(obj)
In [41]:
mdl.parameters.timelimit = 30
In [42]:
s = mdl.solve()
assert s, "No Solution !!!"
In [43]:
print(mdl.get_solve_status())
print(mdl.get_solve_details())
In [44]:
totaloffers = mdl.sum(channelVars[o,p,c]
for o in offersR
for p in productsR
for c in channelsR)
mdl.add_kpi(totaloffers, "nb_offers")
budgetSpent = mdl.sum(channelVars[o,p,c]*channels.get_value(index=c, col="cost")
for o in offersR
for p in productsR
for c in channelsR)
mdl.add_kpi(budgetSpent, "budgetSpent")
for c in channelsR:
channel = channels.get_value(index=c, col="name")
kpi = mdl.sum(channelVars[o,p,c] for p in productsR for o in offersR)
mdl.add_kpi(kpi, channel)
for p in productsR:
product = products[p]
kpi = mdl.sum(channelVars[o,p,c] for c in channelsR for o in offersR)
mdl.add_kpi(kpi, product)
In [49]:
mdl.report()
comp2_df = pd.DataFrame(data=[["CPLEX", mdl.objective_value, mdl.kpi_value_by_name('nb_offers'), mdl.kpi_value_by_name('Mortgage'), mdl.kpi_value_by_name('Pension'), mdl.kpi_value_by_name('Savings'), mdl.kpi_value_by_name('budgetSpent')]], columns=["Algorithm","Revenue","Number of clients","Mortgage offers","Pension offers","Savings offers","Budget Spent"])
In [67]:
comp_df = comp1_df.append(comp2_df, ignore_index=True)
comp_df
comp_df.set_index("Algorithm", inplace=True)
my_plot = comp_df['Revenue'].plot(kind='bar')
With the mathematical optimization, we made a better selection of customers.
In [51]:
#get the hand on the budget constraint
ct = mdl.get_constraint_by_name("budget")
The following cell takes a relatively long term to run because the jobs are run sequentially. The standard subscriptions to DOcplexcloud solve service only allow one job at a time, but you can buy special subscriptions with parallel solves. If you have such a subscription, modify the following cell to benefit from it.
In [52]:
res = []
for i in range(20):
ct.rhs = availableBudget+1000*i
s = mdl.solve()
assert s, "No Solution !!!"
res.append((availableBudget+1000*i, mdl.objective_value, mdl.kpi_value_by_name("nb_offers"), mdl.kpi_value_by_name("budgetSpent")))
In [53]:
mdl.report()
In [54]:
pd.DataFrame(res, columns=["budget", "revenue", "nb_offers", "budgetSpent"])
Out[54]:
Due to the business constraints, we can address a maximum of 1680 customers with a \$35615 budget. Any funds available above that amount won't be spent. The expected revenue is \$87.1K.
In [55]:
ct.rhs = 0
In [56]:
s = mdl.solve()
In [57]:
if not s:
#rename the constraint with a "low" prefix to automatically put a low priority on it.
ct.name = "low_budget"
#setting all bool vars to 0 is an easy relaxation, so let's refuse it and force to offer something to 1/3 of the clients
mdl.add_constraint(totaloffers >= len(offers)//20, ctname="high")
# solve has failed, we try relaxation, based on constraint names
# constraints are prioritized according to their names
# if a name contains "low", it has priority LOW
# if a ct name contains "medium" it has priority MEDIUM
# same for HIGH
# if a constraint has no name or does not match any, it is not relaxable.
from docplex.mp.relaxer import Relaxer
relaxer = Relaxer(prioritizer='match', verbose=True)
relaxed_sol = relaxer.relax(mdl)
relaxed_ok = relaxed_sol is not None
assert relaxed_ok, "relaxation failed"
relaxer.print_information()
In [58]:
mdl.report()
print(mdl.get_solve_status())
print(mdl.get_solve_details())
We need a minimum of 15950\$ to be able to start a marketing campaign. With this minimal budget, we will be able to adress 825 possible clients.
Here are the results of the 2 algorithms:
| Algorithm | Revenue | Number of clients | Mortgage offers | Pension offers | Savings offers | Budget Spent |
|---|---|---|---|---|---|---|
| Greedy | 50800 | 1123 | 299 | 111 | 713 | 21700 |
| CPLEX | 72600 | 1218 | 381 | 117 | 691 | 25000 |
We need a minimum of \$16K to be able to start a valid campaign and we expect it will generate \$47.5K.
Due to the business constraints, we will be able to address 1680 customers maximum using a budget of \$36K. Any money above that amount won't be spent. The expected revenue is \$87K.
| Scenario | Budget | Revenue | Number of clients | Mortgage offers | Pension offers | Savings offers |
|---|---|---|---|---|---|---|
| Standard | 25000 | 72600 | 1218 | 381 | 117 | 691 |
| Minimum | 16000 | 47500 | 825 | 374 | 142 | 309 |
| Maximum | 35500 | 87000 | 1680 | 406 | 155 | 1119 |
Copyright © 2017 IBM. Sample Materials.