The dynamics of constrained multibody systems are effectively modeled by differential-algebraic equations (DAEs) defined on a manifold with Lie group structure - referred to here as geometrically-exact multibody systems. While geometrically-exact theories can handle such DAEs efficiently and accurately, real-time simulation, parametric studies, and optimization of high-dimensional systems with large rigid-body motions and deformations remain challenging. The challenge arises not only from the high computational cost of high-dimensional models, but also from the inability of conventional (linear) reduction methods to capture the strongly nonlinear dynamics induced by large rigid-body motions and large deformations. Model reduction based on spectral submanifolds (SSMs) offers a promising approach for constructing reduced-order models of high-dimensional constrained multibody systems. At present, SSM reduction is not applicable to geometrically-exact multibody systems. This is because SSMs can only be computed for systems posed on Banach spaces and the kinematics of geometrically-exact multibody systems evolve on a matrix Lie group. We present a method that enables the computation of SSMs for geometrically-exact multibody systems without relying on local coordinates, while still preserving the configuration constraints. As we will show, local-coordinates formulations introduce at least cubic nonlinearities in the kinematic equations and standard lower-pair joints. In contrast, the proposed approach produces only quadratic nonlinearities, which significantly simplifies SSM computation. Using the proposed approach, we construct SSM reduced-order models for rigid and flexible benchmark multibody systems and demonstrate that they accurately reproduce the nonlinear dynamics of the full-order model. The proposed approach establishes SSM model reduction for geometrically-exact multibody systems and opens the door to efficient nonlinear dynamical analysis of constraint multibody systems modeled with geometrically-exact theories.