Gaussian process (GP) models provide a flexible Bayesian framework for modelling dynamic systems. Their probabilistic formulation yields a posterior distribution over predictions, supporting informed decision-making in applications such as structural health monitoring and maintenance planning through principled uncertainty quantification. However, this expressiveness comes at a substantial computational cost: training a full (exact) GP scales as O(N 3) with the number of data points, limiting its applicability to large datasets. Sparse GP methods [1], often building on inducing-variable formulations introduced by Titsias [2], have been developed to alleviate this limitation by trading exactness for scalability. In the context of structural dynamics, this work instead explores alternative routes to improved computational efficiency that retain exact GP inference, by embedding physical knowledge directly within structured kernel representations. This approach achieves computational speed-ups comparable to sparse methods, while avoiding the additional implementation complexity and optimisation sensitivity associated with inducing point-based approximations. Structured kernel representations often provide a tradeoff; increased computational efficiency is achieved at the expense of restricting modelling applications. Here, two forms of structured kernel are exploited, ensuring that their modelling constraints remain satisfied. Firstly, celerite kernels [3] are used to efficiently model 1D oscillatory time series. Kronecker kernels [4] are then employed, which are suited to grid-structured inputs i.e. spatio-temporal structural response data. These structured kernels are combined with a single-degree-of-freedom (SDOF) oscillator-derived kernel [5], which enables the retrieval of physically interpretable system parameters, providing insight into the dynamics of the structure. Data from the experimental beam and strut assembly shown in Figure 1 is used to evaluate both the accuracy and computational cost of approaches. Multi-axial loading and a variable preload introduce a range of simulated operating conditions. The proposed method is able to effectively upsample response data to a high fidelity at a fraction of the computational cost of exact GP approaches.