Plotting Keras Performance Trajectories

John Mount

2017-12-23

Below is the standard performance trajectory plot from Keras.

library("ggplot2")
library("WVPlots")
library("keras")

h <- readRDS("historyobject.rds")
plot(h)

We can also load the Keras metricsframe (a data.frame that is fairly standard to when working with Keras) and demonstrate or new WVPlots::plot_Keras_fit_trajectory() plot.

d <- readRDS("metricsframe.rds")

knitr::kable(head(d))

val_loss	val_acc	loss	acc
0.3769818	0.8722	0.5067290	0.7852000
0.2996994	0.8895	0.3002033	0.9040000
0.2963943	0.8822	0.2165675	0.9303333
0.2779052	0.8899	0.1738829	0.9428000
0.2842501	0.8861	0.1410933	0.9545333
0.3119754	0.8817	0.1135626	0.9656000

WVPlots::plot_Keras_fit_trajectory(
  d, 
  title = "model performance by epoch, dataset, and measure")

Obviously this plot needs some training to interpret, but that is pretty much the case for all visualizations.

The ideas of this plot include:

All facets have the same interpretation: “up is better.” This makes it easier to relate the shapes in one pane to the other.
The only horizontal solid curve rendered is the validation (or hold-out) performance. The validation performance is the primary item of interest, so it gets a unique presentation.
The training performance is represented as the upper boundary of the shaded region and with the pluses and vertical strokes.
A vertical line is added at the best epoch picked by the “minus binary cross entropy” metric (this can be turned off or changed).
An optional additional training over-fit penalized dashed curve is added. The default is: this is the validation score minus 10% of the excess generalization error (the difference in training and validation performance).