The Vlasov-Maxwell system describes the dynamics of a plasma in its self-consistent and external magnetic fields. Each plasma species is modeled by a distribution function in phase-space with the velocity advection governed by the electromagnetic fields, for which the self-consistent parts are obtained from Maxwell's equations. Numerical schemes that preserve the structure of the underlying kinetic equations can provide new insights into the long time behavior of plasmas. In this talk, we consider the relativistic formulation of the Vlasov-Maxwell equations. Compared to the non-relativistic case, the gamma factor adds a non-linear coupling of the various velocity dimensions. Structure-preserving spatial discretizations can be obtained by discretizing the magnetic fields based on the finite element exterior calculus framework and a standard particle ansatz for the distribution function. A semi-discretization can be derived from a variational principle. In this talk, we will discuss various temporal discretizations: an explicit splitting preserving the divergence constraints and a semi-implicit splitting scheme additionally preserving the energy to order six in the fraction velocity/speed of light. Finally, we will also discuss subcycling strategy for fast gyration motion.