Optimization problems constrained by partial differential equations (PDEs) arise in a variety of fields. However, in many applications, uncertainties arise in the specification of these models, such as uncertainty in parameters or uncertainty in the boundary and initial conditions. Accounting for these uncertainties introduces challenges to the PDE-constrained problem, as the PDE generally has to be solved for various values of the control variable as well as the uncertain parameters. A recent approach to PDE constrained optimization under uncertainty is to build machine learning based surrogate models, mapping both the control and uncertain parameters to the solution of the PDE or a quantity of interest. Then, optimization is performed using this surrogate model to minimize some risk measure on the quantity of interest. However, such an approach requires an initial data collection step, where the solution of the PDE is first evaluated on points (x,ξ), for controls xand unknown parameter ξ. However, this process may include superfluous evaluations at points distant from the region of interest (e.g., far away from the minima or far away from the current optimization iterate). Furthermore, optimizing the surrogate model may lead to inaccurate results if the surrogate model is inaccurate at certain regions of the control space, even if it is “globally” accurate. To tackle these challenges, we propose a hybrid online/offline procedure to tackle PDE constrained optimization under uncertainty. In the online step, during optimization, we use the framework of inexact trust region methods to specify the desired accuracy of the machine learning-based surrogate model. When necessary, additional data are collected locally and used to update the surrogate model. We discuss this inexact trust region approach and show numerical results on a set of PDE-constrained optimization problems under uncertainty.