Gaussian Process prior models, as used in Bayesian non-parametric statistical models methodology are applied to implement a nonlinear adaptive control law. The expected value of a quadratic cost function is minimised, without ignoring the variance of the model predictions. This leads to implicit regularisation of the control signal (caution) in areas of high uncertainty. As a consequence, the controller has dual features, since it both tracks a reference signal and learns a model of the system from observed responses. The general method and its unique features are illustrated on simulation examples.