Graphical representations of linear models

This notebook is intended to provide examples for how five functions in the seaborn plotting library, regplot, corrplot, lmplot, interactplot, and coefplot, can be used to informatively visualize the relationships between variables in a dataset. The functions are intended to produce plots that are attractive and that can be specified without much work. The goal of these visualizations, which the functions attempt to make achievable, is to emphasize important comparisons in the dataset and provide supporting information without distraction.

These functions are a a bit higher-level than the ones covered in the distributions and timeseries tutorials. Instead of plotting into an existing axis, most expect to have the whole figure to themselves, and they are frequently composed of multiple axes.

All of the functions covered here are Pandas-aware, and both lmplot and coefplot require that the data be in a Pandas dataframe.

Plotting a simple regression: regplot

First, we will show how to visualize the relationship between two variables. To do so, we'll create a very simple fake data set.

The regplot function accepts two arrays of data and draws a scatterplot of the relationship. It also fits a regression line to the data, bootstraps the regression to get a confidence interval (95% by default), and plots the marginal distributions of the two variables. It does all this with a very simple function call:

The function also works with pandas DataFrame objects, in which case the x and y values should be strings. Calling the function this way gets axis labels for free (otherwise you can use the xlabel and ylabel keyword arguments).

It's also easy to manipulate the appearance of the plot at a high level by setting a global color and altering the confidence interval for the regression estimate. By default, it plots the 95% confidence interval (this is computed with a bootstrap), but you may prefer to use a 68% CI, which corresponds to the standard error of the regression estimate.

You'll note that the a pearson correlation statistic is automatically computed and displayed in the scatterplot. If your data are not normally distributed, you can provide a different function to calculate a correlation metric; anything that takes two arrays of data and returns a stat numeric or (stat, p) tuple will work.

For large datasets, the bootstrap may become computationally intensive (and confidence intervals with large datasets will dramatically shrink), so you may want to turn it off.

The default behavior for the fit statistic is to use the function name, but sometimes you might want to use a different string. That's what the func_name keyword argument is for.

For finer control over the individual aspects of the plot, you can pass dictionaries with keyword arguments for the underlying seaborn or matplotlib functions.

Pairwise correlations in a large dataset: corrplot

Now let's explore a more complex dataset. We'll use the tips data that is provided with R's reshape2 package. This is a good example dataset in that it provides several quantitative and qualitative variables in a tidy format, but there aren't actually any interesting interactions so I am open to other suggestions for different data sets to use here.

Plotting correlation heatmaps

Once you have a tidy dataset, often the first thing you want is a very high-level summary of the relationships between the variables. Correlation matrix heatmaps can be very useful for this purpose. The corrplot function not only plots a color-coded correlation matrix, but it will also obtain a p value for each correlation using a permutation test to give you some indication of the significance of each relationship while correcting for multiple comparisons in an intelligent way.

Note that if you have a huge dataset, the permutation test will take a while. Of course, if you have a huge dataset, p values will not be particularly relevant, so you can turn off the significance testing.

It's also possible to choose a different colormap, but choose wisely! Don't even try using the "jet" map; you'll get a ValueError. The colorbar itself is also optional.

By default, the colormap is centered on 0 and uses a diverging map, which is appropriate since 0 is a meaningful boundary and both large positive and negative values are interesting.

Sometimes, though, you are only interested in either positive or negative values. In these cases, you can set the tail for the significance test, which will also change the default colormap to a sequential map.

It's also possible to specify the range for the colormap. Note that setting the test direction modifies the colormap and range, but the inverse is not true; the stars here will still correspond to a two-tailed test.

You might also have many variables, in which case the correlation coefficient annotation may not fit well. In this case, it can be turned off, and the variable names can be moved to the sides of the plot:

Visualizing multiple regression: lmplot

The lmplot function provides a more general interface for plotting linear relationships in a complex set of data. In its most basic usage, it does the same thing as the core of the regplot function. Note that lmplot only works with DataFrames.

The advantage to using lmplot over regplot is that you can visualize linear relationships among subsets of a larger data structure. There are a few ways to do this; but perhaps the most amenable to direct comparisons involves separating subgroups by color.

The default color palette is husl, but you can use any of the seaborn color palettes for the color factor.

It's not actually neccesary to fit a regression line to the data, if you don't want to. (Although I need to fix things so that the legend shows up when using color grouping -- this doesn't work at the moment).

Higher-order trends

You can also fit higher-order polynomials. Although there is not such a trend in this dataset, let's invent one to see what that might look like.

Logistic Regression

What if we want to fit a model where the response variable is categorical? (At the moment, it must be binary and numeric, so {0, 1} and {True, False} both work).

This plot suggests there is an relationship between the continuous value of x1 and the likelihood that y_flip will be positive, but it has a few problems. The first is that the individual observations are all plotted on top of each other, so their distribution is obscured. We can address this issue by adding a bit of jitter to the scatter plot.

A more fundamental problem follows from using basic linear regression with a binary response variable. The regression line implies that the probabilitiy of a positive y_flip with very large values of x1 is larger than 1. Of course, that doesn't make sense, which is why logistic regression was invented. lmplot can likewise plot a logistic curve over the data. You might want to use fewer bootstrap iterations, as the logistic regression fit is much more computationally intensive.

Faceted plots

There are several other ways to visualize fits of the model to sub-groups in the data.

You can also separate out factors into facet plots on the columns or rows.

Which doesn't mean you can't keep an association between colors and factors

By default, the same x and y axes are used for all facets, but you can turn this off if you have a big difference in intercepts that you don't care about.

Plotting with discrete predictor variables

Sometimes you will want to plot data where the independent variable is discrete. Although this works fine out of the box:

And can be improved with a bit of jitter:

It might be more informative to estimate the central tendency of each bin. This is easy to do with the x_estimator argument. Just pass any function that aggregates a vector of data into one estimate. The estimator will be bootstrapped and a confidence interval will be plotted -- 95% by default, as in other cases within these functions.

Sometimes you may want to plot binary factors and not extrapolate with the fitted line beyond your data points. (Here the fitted line doesn't make all that much sense for extrapolating within the range of the data either, but it does make the trend more visually obvious). Note that at the moment the independent variable must be "quantitative" (so, numerical or boolean typed), but in the future binary factors with string variables will be implemented.

With large datasets, the scatterplot can become overwhelming and fail to provide useful information about reliable linear trends. In these cases, it can be useful to bin the predictor variable into discrete values and plot an estimated central tendency and confidence interval.

Note that the regression estimate is still fit to the original data; the binning only applies to the visual representation of the observations.

Other faceting options

You can plot data on both the rows and columns to compare multiple factors at once.

And, of course, you can compose the color grouping with facets as well to facilitate comparisons within a complicated model structure.

If you have many of levels for some factor (say, your population of subjects), you may want to "wrap" the levels so that the plot is not too wide:

Removing nuisance variables

Because of how we generated our fake dataset, x1 and y3 appear to be related.

However, this relationship is being driven entirely by y2. We can thus ask what happens when we residualize our dependent variable against the confound before plotting.

Plotting interactions between continuous variables: interactplot

Faceting and binning the regression by color can make it easy to unpack interactions between the predictor variables in your dataset. However, these methods really only work when, at most, one of your variables is continuous. Two-way interactions between continuous variables are often interesting, though, but difficult to visualize.

Let's make some fake data with an interaction of this sort and explore how we might visualize it.

One approach is to bin one of the predictors and then plot the data as before, pretending the predictor is categorical.

This is servicable, but lacking in several ways. It requires several cumbersome steps, the choice of the bin size is arbitrary, and collapsing the continuous data into categories loses information.

An alternative approach plots the two independent variables on the x and y axes of a plot and color-encodes the model predictions with a contour plot. This maintains the continuous nature of the data. The seaborn function interactplot draws such a plot, with an interface similar to regplot:

Naturally, you can directly pass a dataframe, and also adjust the aesthetics of the plot.

The two underlying plot functions are contourf() and plot(), both of which can be tweaked with a keyword argument dictionary.

Note the appearance when we plot data that was not simulated with an interaction.

This works for logistic regression models, as well.

Plotting linear model parameters: coefplot

Although the above plots can be very helpful for understanding the structure of your data, they fail with more than about 4 variables or with more than one continuous predictor. To visually summarize this kind of model, it can be helpful to plot the point estimates for each coefficient along with confidence intervals. The coefplot function achieves this by using a Patsy formula specification for the model structure.

When you have repeated measures in your dataset (e.g. an experiment performed with multiple subjects), you can group by the levels of that variable and plot the model coefficients within each group. Note that the semantics of the resulting figure changes a little bit from the example above.