In [1]:
import model, modelfast, itertools, random

In [2]:
%load_ext autoreload

In [3]:
%autoreload 2

Switching Data

We make an action/observation sequence of length 500000/2. .01% of the time the world will change between 2 POMDPs generating the data.

The underlying POMDPs are pretty stupid and are built in another file.

In [5]:
import pomdp

In [112]:
seqswitching = list(itertools.islice(pomdp.Switcher(0.0001, pomdp.Room(), pomdp.Hallway()), 500000))


We make two models. Both are factored models so the predictors don't get confused between predicting actions and observations. We only compare about the second factor (the one predicting observations given the history).

The first is a CTW model.

The second is a FMN model.

We then run the models on our switching data.

In [154]:
MCTW = model.Factored((model.Dumb(), modelfast.CTW_KT(8)))

In [155]:
MFMN = model.Factored((model.Dumb(), model.FMN(lambda: modelfast.CTW_KT(8))))

In [156]:
MCTW.update_seq(seqswitching, [])


In [157]:
MFMN.update_seq(seqswitching, [])


Analyzing the Models

FMN extracts "useful" models from the history. One way to look at these extracted models is to compare the predictive ability on streams of data. In particular, we will examine the models by looking at their predictive ability on streams from the two underlying POMDP models.

In [158]:

<model.LogStore at 0x113e421d0>

In [159]:
B = MFMN.factors[1].model_factory()

In [160]:
seqhallway = list(itertools.islice(pomdp.Hallway(), 100000))

In [161]:
model_results = [ model.Factored((model.Dumb(), m)).log_predict_seq(seqhallway, []) for m in B.models ]

In [162]:
model_baseline = model.Factored((model.Dumb(), MCTW.factors[1])).log_predict_seq(seqhallway, [])

In [163]:
bar(range(len(B.models)), model_results);
plot((0, len(B.models)), (model_baseline,)*2, 'r');

In [164]:
max(model_results), model_baseline

(-10.513835471291582, -2169.112089504043)

In [165]:
seqroom = list(itertools.islice(pomdp.Room(), 100000))

In [166]:
model_results = [ model.Factored((model.Dumb(), m)).log_predict_seq(seqroom, []) for m in B.models ]

In [167]:
model_baseline = model.Factored((model.Dumb(), MCTW.factors[1])).log_predict_seq(seqroom, [])

In [168]:
bar(range(len(B.models)), numpy.array(model_results));
plot((0, len(B.models)), (model_baseline,)*2, 'r');

In [169]:
max(model_results), model_baseline

(-397.24810705354076, -707.7020135167986)

The first thing to notice in the two graphs is that the models are essentially specialized to perform well on one POMDP or the other. The one unspecialized model (Model 6) is the empty CTW model that is the base model for FMN.

The second thing to notice is that the best model from the set for each underlying POMDP is better than the CTW model that tries to predict over both POMDPs.

In [ ]: