In the field of topology optimization, it is commonplace to use discretization schemes with low convergence orders. This is motivated by a desire for fine resolution and short time-to-solution, combined with the fact that the resulting optimized designs are often highly irregular. To investigate potential benefits of higher-order discretizations, this work considers higher-order time-integration schemes, specifically applied to Navier-Stokes flow including a Brinkman penalty. Four different types of ODE-solvers are considered: implicit $\theta$-schemes, the Backward Differentiation Formula (BDF), Diagonally Implicit Runge-Kutta methods (DIRK), and Spectral Deferred Corrections (SDC). For an ODE with N degrees of freedom, these ODE-solvers consist of several subproblems, each with N degrees of freedom. Additionally, the considered integrators are applicable to stiff problems and differential-algebraic equations, like the semi-discretized Navier-Stokes equations with large Brinkman penalties. The three latter higher-order ODE-solvers exhibit better accuracy-to-cost relationships than first-order implicit theta-schemes, even when the error tolerance is very large. Additionally, the higher-order integrators sometimes achieve reasonably small errors when the CFL-number is very large. In general, the investigated higher-order integrators obtain smaller errors using fewer time steps, thus reducing both the time-to-solution and memory demands, at the expense of greater implementation costs.