ServerSim Core Concepts

This document is a Jupyter notebook. See http://jupyter.org/ for more information on Jupyter notebooks.

This is an overview of ServerSim, a framework for the creation of discrete event simulation models to analyze the performance, throughput, and scalability of services deployed on computer servers.

ServerSim is a small framework based on SimPy, a well-known discrete-event simulation framework written in the Python language.

The reader should have at least cursory familiarity with Python and SimPy in order to follow the technical details in this document.

Python is well-suited to this kind of application due to its rapid development dynamic language characteristics and the availability of powerful libraries relevant for this kind of work. In addition to SimPy, we will use portions of SciPy, a powerful set of libraries for efficient data analysis and visualization that includes Matplotlib, which will be used for plotting graphs in our tutorial.

ServerSim consists of a several classes and utilities. The main classes are described below.

class Server

Represents a server -- physical, VM, or container, with a predetermined computation capacity. A server can execute arbitrary service request types. The computation capacity of a server is represented in terms of a number of hardware threads and a total speed number (computation units processed per unit of time). The total speed is equally apportioned among the hardware threads, to give the speed per hardware thread. A server also has a number of associated software threads (which must be no smaller than the number of hardware threads). Software threads are relevant for blocking computations only.

The simulations in this document assume non-blocking services, so the software threads will not be of consequence in the tutorial example.

Attributes:

env: SimPy Environment, used to start and end simulations, and used internally by SimPy to control simulation events.
max_concurrency: The maximum of _hardware threads for the server.
num_threads: The maximum number of software threads for the server.
speed: Aggregate server speed across all _hardware threads.
name: The server's name.
hw_svc_req_log: If not None, a list where hardware service requests will be logged. Each log entry is a triple ("hw", name, svc_req), where name is this server's name and svc_req is the current service request asking for hardware resources.
sw_svc_req_log: If not None, a list where software thread service requests will be logged. Each log entry is a triple ("sw", name, svc_req), where name is this server's name and svc_req is the current service request asking for a software thread.

class SvcRequest

A request for execution of computation units on one or more servers.

A service request submission is implemented as a SimPy Process.

A service request can be a composite of sub-requests. A service request that is part of a composite has an attribute that is a reference to its parent service request. In addition, such a composition is implemented through the gen attribute of the service request, which is a generator. That generator can yield service request submissions for other service requests.

By default, a service request is non-blocking, i.e., a thread is held on the target server only while the service request itself is executing; the thread is relinquished when the request finishes executing and it passes control to its sub-requests. However, blocking service requests can be modeled as well (see the Blkg class).

Attributes:

env: The SimPy Environment.
parent: The immediately containing service request, in case this is part of a composite service request. None othersise.
svc_name: Name of the service this request is associated with.
gen: Generator which defines the behavior of this request. The generator produces an iterator which yields simpy.Event instances. The submit() method wratps the iterator in a simpy.Process object to schedule the request for execution by SimPy..
server: The target server. May be None for composite service requests, i.e., those not produced by CoreSvcRequester.
in_val: Optional input value of the request.
in_blocking_call: Indicates whether this request is in the scope of a blocking call. When this parameter is true, the service request will hold a software thread on the target server while the service request itself and any of its sub-requests (calls to other servers) are executing. Otherwise, the call is non-blocking, so a thread is held on the target server only while the service request itself is executing; the thread is relinquished when this request finishes executing and it passes control to its sub-requests.
out_val: Output value produced from in_val by the service execution. None by default.
id: The unique numerical ID of this request.
time_log: List of tag-time pairs representing significant occurrences for this request.
time_dict: Dictionary with contents of time_log, for easier access to information.

class SvcRequester

Base class of service requesters.

A service requester represents a service. In this framework, a service requester is a factory for service requests. "Deploying" a service on a server is modeled by having service requests produced by the service requester sent to the target server.

A service requester can be a composite of sub-requesters, thus representing a composite service.

Attributes:

env: The SimPy Environment.
svc_name: Name of the service.
log: Optional list to collect all service request objects produced by this service requester.

class UserGroup

Represents a set of identical users or clients that submit service requests.

Each user repeatedly submits service requests produced by service requesters randomly selected from the set of service requesters specified for the group.

Attributes:

env: The Simpy Environment.
num_users: Number of users in group. This can be either a positive integer or a sequence of (float, int), where the floats are monotonically increasing. In this case, the sequence represents a step function of time, where each pair represents a step whose range of x values extend from the first component of the pair (inclusive) to the first component of the next pair (exclusive), and whose y value is the second component of the pair. The first pair in the sequence must have 0 as its first component. If the num_users argument is an int, it is transformed into the list [(0, num_users)].
name: This user group's name.
weighted_svcs: List of pairs of SvcRequester instances and positive numbers representing the different service request types issued by the users in the group and their weights. The weights are the relative frequencies with which the service requesters will be executed (the weights do not need to add up to 1, as they are normalized by this class).
min_think_time: The minimum think time between service requests from a user. Think time will be uniformly distributed between min_think_time and max_think_time.
max_think_time: The maximum think time between service requests from a user. Think time will be uniformly distributed between min_think_time and max_think_time.
quantiles: List of quantiles to be tallied. It defaults to [0.5, 0.95, 0.99] if not provided.
svc_req_log: If not None, a sequence where service requests will be logged. Each log entry is a pair (name, svc_req), where name is this group's name and svc_req is the current service request generated by this group.
svcs: The first components of weighted_svcs.

class CoreSvcRequester(SvcRequester)

This is the core service requester implementation that interacts with servers to utilize server resources.

All service requesters are either instances of this class or composites of such instances created using the various service requester combinators in this module

Attributes:

env: See base class.
svc_name: See base class.
fcompunits: A (possibly randodm) function that generates the number of compute units required to execute a service request instance produced by this object.
fserver: Function that produces a server (possibly round-robin, random, or based on server load information) when given a service request name. Models a load-balancer.
log: See base class.
f: An optional function that is applied to a service request's in_val to produce its out_val. If f is None, the constant function that always returns None is used.

Other service requester classes

Following are other service requester classes (subclasses of SvcRequester) in addition to CoreSvcRequester, that can be used to define more complex services, including blocking services, asynchronous fire-and-forget services, sequentially dependednt services, parallel service calls, and service continuations. These additional classes are not used in the simulations in this document.

class Async(SvcRequester)

Wraps a service requester to produce asynchronous fire-and-forget service requests.

An asynchronous service request completes and returns immediately to the parent request, while the underlying (child) service request is scheduled for execution on its target server.

Attributes:

env: See base class.
svc_requester: The underlying service requester that is wrapped by this one.
log: See base class.

class Blkg(SvcRequester)

Wraps a service requester to produce blocking service requests.

A blocking service request will hold a software thread on the target server until the service request itself and all of its non-asynchronous sub-requests complete.

Attributes:

env: See base class.
svc_requester: The underlying service requester that is wrapped by this one.
log: See base class.

class Seq(SvcRequester)

Combines a non-empty list of service requesters to yield a sequential composite service requester.

This service requester produces composite service requests.
A composite service request produced by this service requester consists of a service request from each of the provided service requesters. Each of the service requests is submitted in sequence, i.e., each service request is submitted when the previous one completes.

Attributes:

env: See base class.
svc_name: See base class.
svc_requesters: A composite service request produced by this service requester consists of a service request from each of the provided service requesters
cont: If true, the sequence executes as continuations of the first request, all on the same server. Otherwise, each request can execute on a different server.
log: See base class.

class Par(SvcRequester)

Combines a non-empty list of service requesters to yield a parallel composite service requester.

This service requester produces composite service requests.
A composite service request produced by this service requester consists of a service request from each of the provided service requesters. All of the service requests are submitted concurrently.

When the attribute cont is True, this represents multi-threaded execution of requests on the same server. Otherwise, each service request can execute on a different server.

Attributes:

env: See base class.
svc_name: See base class.
svc_requesters: See class docstring.
f: Optional function that takes the outputs of all the component service requests and produces the overall output for the composite. If None then the constant function that always produces None is used.
cont: If true, all the requests execute on the same server. Otherwise, each request can execute on a different server. When cont is True, the server is the container service request's server if not None, otherwise the server is picked from the first service request in the list of generated service requests.
log: See base class.