Active/Active Serverless App with Route 53 Weighted Routing

This notebook can be used to configure an active-standby two region serverless API project. This includes the following:

Creation of a common API key that can be installed in both regions. This is needed to ensure transparent request routing from the perspective of the service consumer.
Addition of custom domain names to each of the regional API gateway apps
Creation of route 53 cnames for the regional endpoints, and a weighted routing policy integrated with route53 health checks

Library Code



In [ ]:

    
# SDK Imports
import boto3

cformation_east = boto3.client('cloudformation', region_name='us-east-1')
cformation_west = boto3.client('cloudformation', region_name='us-west-2')

gw_east = boto3.client('apigateway', region_name='us-east-1')
gw_west = boto3.client('apigateway', region_name='us-west-2')



In [ ]:

    
def get_stack_name(service, stage):
    return '{}-{}'.format(service,stage)



In [ ]:

    
def get_endpoint(cf_client, stack_name):
    response = cf_client.describe_stacks(
        StackName=stack_name
    )
    
    outputs = response['Stacks'][0]['Outputs']
    endpoint =  [d for d in outputs if d['OutputKey'] == 'ServiceEndpoint'][0]['OutputValue']
    return endpoint



In [ ]:

    
def get_plan_and_api_ids(gw_client, service, stage):
    response = gw_client.get_usage_plans()
    plans = response['items']
    stack_name = get_stack_name(service, stage)
    plan =  [d for d in plans if d['name'] == stack_name][0]
    plan_id = plan['id']
    api_stage = [d for d in plan['apiStages'] if d['stage'] == stage][0]
    api_id = api_stage['apiId']
    return plan_id, api_id



In [ ]:

    
import uuid

def generate_api_key():
    return str(uuid.uuid4())



In [ ]:

    
def create_api_key_and_add_to_plan(gw_client, key_name, key_val, plan_id):
   
    create_key_response = gw_client.create_api_key(
        name=key_name,
        enabled=True,
        generateDistinctId=True,
        value=key_val
    )
    
    key_id = create_key_response['id']
    
    plan_key_response = gw_client.create_usage_plan_key(
        usagePlanId=plan_id,
        keyId=key_id,
        keyType='API_KEY'
    )
    
    return id, key_id



In [ ]:

    
def form_s3_url_prefix(region):
    prefix = ''
    if region == 'us-east-1':
        prefix = 'https://s3.amazonaws.com'
    else:
        prefix = 'https://s3-' + region + '.amazonaws.com'
    return prefix



In [ ]:

    
# Create a key and add it to the usage plan?
# - create_api_key - need key id output
# - you can get the usage plan id and the api id via get_usage_plan and matching the plan with same name
#   as the stack
# - create_usage_plan_key associates the key to the plan: inputs are plan id, key id

Application Context



In [ ]:

    
service = 'serverless-rest-api-with-dynamodb'
stage = 'dev'
cross_region_key_name = 'xregion_key'
bucket_name = 'xtds-cf-templates'
primary_region = 'us-east-1'



In [ ]:

    
stack_name = get_stack_name(service, stage)
east_endpoint = get_endpoint(cformation_east, stack_name)
print east_endpoint

west_endpoint = get_endpoint(cformation_west, stack_name)
print west_endpoint

Global Database Replication Group

At this moment, DynamoDB global tables does not appear to be supported by Cloud Formation. The SDK, however, allows forming a global table from like-named tables in multiple regions



In [ ]:

    
table_name = service + '-' + stage
print table_name



In [ ]:

    
ddb_client = boto3.client('dynamodb')



In [ ]:

    
response = ddb_client.create_global_table(
    GlobalTableName=table_name,
    ReplicationGroup=[
        {
            'RegionName': 'us-east-1'
        },
        {
            'RegionName': 'us-west-2'
        },
    ]
)

print response

Key Synchronization

This part of the notebook creates a common key for the gateway fronted app in both regions.



In [ ]:

    
key_val = generate_api_key()
print key_val



In [ ]:

    
# Create east key and add to plan
plan_id_east, api_id_east = get_plan_and_api_ids(gw_east, service, stage)
key_val_east, key_id_east = create_api_key_and_add_to_plan(gw_east, cross_region_key_name, key_val, plan_id_east)



In [ ]:

    
plan_id_west, api_id_west = get_plan_and_api_ids(gw_west, service, stage)
key_val_west, key_id_west = create_api_key_and_add_to_plan(gw_west, cross_region_key_name, key_val, plan_id_west)

Custom Domain Names

Now that API gateway deployments can be tagged as regional, we are free from the tyranny of cloud front certificate restrictions that prevented us from registering certificates with the same domain name in two different regions.

With regional API deployments, we can associated the same SSL cert with the endpoints in both regions, and use the certificate domain as the route 53 alias to define failover or weighted routing policies (or any others we desire).



In [ ]:

    
domain_name = 'superapi.elcaro.net'

East



In [ ]:

    
# Custom domains hang around even when the APIs the are associated with are deleted. In
# this cell we figure out if the following cells need to be executed.
regional_domain_name = ''
response = gw_east.get_domain_names()

items = response['items']

items = [x for x in items if x['domainName'] == domain_name]

if len(items) == 1:
    regional_domain_name = items[0]['regionalDomainName']
    print 'Custom domain name for API exists with regional domain name {}'.format(regional_domain_name)
    print '===> Skip the rest of the cells in this section'
else:
    print '===> Custom domain does not exist - continue executing the cells in this section of the notebook'



In [ ]:

    
# We need to select the certificate associated with out domain name
acm_client = boto3.client('acm')



In [ ]:

    
response = acm_client.list_certificates()

summaryList = response['CertificateSummaryList']
print summaryList

domain_cert = [x for x in summaryList if x['DomainName'] == domain_name][0]
print domain_cert

cert_arn = domain_cert['CertificateArn']
print cert_arn



In [ ]:

    
# Create the domain name
response = gw_east.create_domain_name(
    domainName=domain_name,
    regionalCertificateArn=cert_arn,
    endpointConfiguration={
        'types': [
            'REGIONAL'
        ]
    }
)

print response



In [ ]:

    
regional_domain_name = response['regionalDomainName']
print regional_domain_name



In [ ]:

    
# Get the rest api id
response = gw_east.get_rest_apis()
print response

items = response['items']
item = [x for x in items if x['name'] == stage + '-' + service][0]
print item

rest_api_id = item['id']
print rest_api_id



In [ ]:

    
# Create custom domain mapping for our stage - here we subsume the stage into the mapping
response = gw_east.create_base_path_mapping(
    domainName=domain_name,
    basePath='',
    restApiId=rest_api_id,
    stage=stage
)

print response



In [ ]:

    
# East health check endpoint
east_hc_cname = rest_api_id + '.execute-api.us-east-1.amazonaws.com'
print east_hc_cname

West



In [ ]:

    
# Custom domains hang around even when the APIs the are associated with are deleted. In
# this cell we figure out if the following cells need to be executed.
west_domain_name = ''
response = gw_west.get_domain_names()

items = response['items']

items = [x for x in items if x['domainName'] == domain_name]

if len(items) == 1:
    west_domain_name = items[0]['regionalDomainName']
    print 'Custom domain name for API exists with regional domain name {}'.format(west_domain_name)
    print '===> Skip the rest of the cells in this section'
else:
    print '===> Custom domain does not exist - continue executing the cells in this section of the notebook'



In [ ]:

    
acm_west = boto3.client('acm', region_name='us-west-2')



In [ ]:

    
response = acm_west.list_certificates()

summaryList = response['CertificateSummaryList']
print summaryList

domain_cert = [x for x in summaryList if x['DomainName'] == domain_name][0]
print domain_cert

cert_arn = domain_cert['CertificateArn']
print cert_arn



In [ ]:

    
# Create the domain name
response = gw_west.create_domain_name(
    domainName=domain_name,
    regionalCertificateArn=cert_arn,
    endpointConfiguration={
        'types': [
            'REGIONAL'
        ]
    }
)

print response



In [ ]:

    
west_domain_name = response['regionalDomainName']
print west_domain_name



In [ ]:

    
# Get the rest api id
response = gw_west.get_rest_apis()
print response

items = response['items']
item = [x for x in items if x['name'] == stage + '-' + service][0]
print item

rest_api_id = item['id']
print rest_api_id



In [ ]:

    
# Create custom domain mapping for our stage - here we subsume the stage into the mapping
response = gw_west.create_base_path_mapping(
    domainName=domain_name,
    basePath='',
    restApiId=rest_api_id,
    stage=stage
)

print response



In [ ]:

    
# West health check endpoint
west_hc_cname = rest_api_id + '.execute-api.us-west-2.amazonaws.com'
print west_hc_cname

Route 53 Set Up

For this deployment, we'll make east our primary region and west our secondary region. Note this is arbitrary, and also note we can set up other policies like weighted routing, latency based routing, etc.



In [ ]:

    
r53_client = boto3.client('route53')

Health Check - East



In [ ]:

    
caller_ref = generate_api_key() # Note this generates a uuid string that can be used as a key
print caller_ref

print regional_domain_name



In [ ]:

    
# East health check
response = r53_client.create_health_check(
    CallerReference=caller_ref,
     HealthCheckConfig={
        'Type':'HTTPS',
        'ResourcePath':'/' + stage + '/todos/health',
        'FullyQualifiedDomainName':east_hc_cname
    }
)

print response



In [ ]:

    
hc_id = response['HealthCheck']['Id']
print 'health check id: {}'.format(hc_id)



In [ ]:

    
# Now tag the health check name
tag_resp = r53_client.change_tags_for_resource(
    ResourceType='healthcheck',
    ResourceId=hc_id,
    AddTags=[
        {
            'Key':'Name',
            'Value':'east-api-hc'
        },
    ]
)

print tag_resp



In [ ]:

    
hc_resp = r53_client.get_health_check_status(
    HealthCheckId=hc_id
)

print hc_resp

Health Check - West



In [ ]:

    
caller_ref = generate_api_key() # Note this generates a uuid string that can be used as a key
print caller_ref

print west_domain_name



In [ ]:

    
# West health check
response = r53_client.create_health_check(
    CallerReference=caller_ref,
     HealthCheckConfig={
        'Type':'HTTPS',
        'ResourcePath': '/' + stage + '/todos/health',
        'FullyQualifiedDomainName':west_hc_cname
    }
)

print response



In [ ]:

    
hc_id = response['HealthCheck']['Id']
print 'health check id: {}'.format(hc_id)



In [ ]:

    
# Now tag the health check name
tag_resp = r53_client.change_tags_for_resource(
    ResourceType='healthcheck',
    ResourceId=hc_id,
    AddTags=[
        {
            'Key':'Name',
            'Value':'west-api-hc'
        },
    ]
)

print tag_resp

Route 53 CNames and Routing Policy



In [ ]:

    
hosted_zone = 'elcaro.net.'
response = r53_client.list_hosted_zones()

zones = response['HostedZones']

zones = [x for x in zones if x['Name'] == hosted_zone]

hosted_zone_id = zones[0]['Id']

print hosted_zone_id



In [ ]:

    
# Um, grab the health check ids again - we'll fix this later
response = r53_client.list_health_checks()

health_checks = response['HealthChecks']
print health_checks

east_check = [x for x in health_checks if x['HealthCheckConfig']['FullyQualifiedDomainName'] == east_hc_cname][0]['Id']
print east_check

west_check = [x for x in health_checks if x['HealthCheckConfig']['FullyQualifiedDomainName'] == west_hc_cname][0]['Id']
print west_check



In [ ]:

    
response = r53_client.list_resource_record_sets(
    HostedZoneId=hosted_zone_id
)
print response



In [ ]:

    
response = r53_client.change_resource_record_sets(
    HostedZoneId=hosted_zone_id,
    ChangeBatch={
        'Changes': [
            {
                'Action': 'CREATE',
                'ResourceRecordSet': {
                    'Name': domain_name + '.',
                    'Type': 'CNAME',
                    'SetIdentifier': 'east',
                    'Weight': 50,
                    'TTL': 30,
                    'ResourceRecords': [
                        {
                            'Value': regional_domain_name
                        },
                    ],
                    'HealthCheckId': east_check
                }
            },
            {
                'Action': 'CREATE',
                'ResourceRecordSet': {
                    'Name': domain_name + '.',
                    'Type': 'CNAME',
                    'SetIdentifier': 'west',
                    'Weight': 50,
                    'TTL': 30,
                    'ResourceRecords': [
                        {
                            'Value': west_domain_name
                        },
                    ],
                    'HealthCheckId': west_check
                }
            }
        ]
    }
)

print response



In [ ]: