We propose a method to optimise the parameters of a policy which will be used to safely perform a given task in a data-efficient manner. We train a Gaussian process model to capture the system dynamics, based on the PILCO framework. The model has useful analytic properties, which allow closed form computation of error gradients and the probability of violating given state space constraints. Even during training, only policies that are deemed safe are implemented on the real system, minimising the risk of catastrophic failure.