The Bayesian approach to solving inverse problems updates a prior distribution to a posterior distribution of the parameters through data [1]. The likelihood of the data needed to obtain the posterior distribution depends on a computational model for the numerical solution of a partial differential equation. Standard procedures for solving the Bayesian inverse problem are based on multiple evaluations of the likelihood function and, hence, the underlying computational model. The application of model order reduction enables a cheaper solution of the inverse problem for resource demanding computational models and many-query contexts such as digital twins.\newline We focus on Bayesian inference of static structural loads with Gaussian parameter priors and sparse data. As limited data is only informative in a low-dimensional subspace of the parameter space, the posterior in such problems can be expressed as a low-rank update of the prior distribution [2]. In this work, we introduce a framework that leverages the low-rank mapping between the measurements and the load itself to construct a reduced-order model for inverse problems [3]. This approach is based on a Petrov-Galerkin projection onto the leading eigendirections of the Fisher information matrix and the prior precision, requiring no snapshot data and, hence, leading to minimal offline costs. Through numerical examples drawn from structural engineering, we demonstrate superiority of the LIS reduced-order model compared to reduced-order models obtained using proper orthogonal decomposition (POD).