The Docker Image Construction Problem

Part 1. Introduction

Last edited: August 13, 2015

Ryan J. O'Neil
Engineering @ Yhat, Inc.
SEOR @ George Mason University

Motivation

Consider the devops engineer.

A lot of devops revolves around the construction of environments for:

  • Software development
  • Testing & quality assurance
  • Running operational systems

There's even a Venn diagram about this...

A simple example is the upgrading of My Enterprise Java Application (TM) to a new Linux kernel. My Enterprise Java Application generates lots of \$\$\$\$ for My Enterprise Software Company. I don't want to just upgrade the kernel on our production instance without testing first. So I need two environments from my devops person.

One for running the production instance:

Command
$A$ Install the Java compiler and runtime environment.
$B$ Download a set of external dependencies.
$C$ Set of a database schema and populate it with data.

And another for testing it with the new kernel:

Command
$A$ Install the Java compiler and runtime environment.
$B$ Download a set of external dependencies.
$C$ Set of a database schema and populate it with data.
$D$ Upgrade the kernel to the latest version.

If My Enterprise Java Application (TM) integrates with a number of optional third party applications, I might need an additional test environment for each one.

Looks like my devops engineer will be working all weekend.

At a big software shop, devops may be responsible for the continual setup and teardown of dozens (or hundreds) of system configurations for a single software project.

This used to be done with physical hardware on (sometimes) fresh operating system installs.

Most medium-to-large software shops had an air conditioned room that looked like this.

Background

Docker Cache Mechanics

Problem Formulation