The Shifted Boundary Method (SBM) is an alternative to traditional unfitted FEM approaches that addresses the small cut-cell problem by eliminating cell cutting altogether. The SBM introduces surrogate domains which contain only fully formed elements. The true boundary conditions are shifted onto the location of the surrogate domain boundary by extension operators constructed with Taylor expansions, preserving accuracy. The Shifted Boundary Method is simple for Dirichlet boundary conditions, but more challenging for Neumann boundary conditions, since higher-order terms in the Taylor expansion are unavailable. We propose an approach to Neumann boundary conditions that does not require a mixed formulation, maintaining the same data structure requirements as for Dirichlet boundary conditions. This approach yields optimal accuracy, achieved using approximate integration formulas of the weak form in the gap between the true and surrogate boundary. Note that no cut-cell integration is performed, but only integrals on the surrogate boundary.