Quintile Plot
Why is it that when normalizing values for one model we divide by that model's overall average predicted value for the predicted and the actual data but when normalizing values for two models we divide each model by their own average overall predicted value and divide the actual values by the average overall actual value? Referring to Goldburd_Validation_PS2 Q1 and Q4.
Comments
For a quantile plot, the source text (p.77) actually says all we need to do is plot the average actual pure premium and the average predicted pure premium for each quantile. Since we have two separate plots with potentially very different scales, it helps to normalize them. The easiest way to do this is to scale everything relative to the average predicted pure premium across all quantiles. If we normalize by each column then we lose information about the difference between the average predicted and average actual on an overall basis. This can make it harder for us to tell which model is performing better.
For a double lift chart we don't have to worry about losing this information because we took it into account via the sort ratio.