The solution of inverse problems is inherent in many applications in computational science and engineering. A typical scenario arises, for example, when inferring parameters from noisy data in the context of material properties estimation or detecting damages in structural health monitoring. The integration of physical models based on partial differential equations (PDEs) can be beneficial, in particular when measurements are incomplete. For the discretization of PDEs, spline-based and isogeometric methods offer clear advantages and higher accuracy on a per-degree-of-freedom basis. Recent advances in surrogate and reduced order models [1] show promise in multi-query settings although the cost of training and generation of high-fidelity data can be high. In this contribution we explore the method of optimizing a discrete loss (ODIL) [2] in combination with spline-based numerical solvers for the solution of inverse problems. The framework minimizes a cost function for spline approximations of PDEs using machine learning tools and automatic differentiation, while it inherits the properties of the underlying numerical scheme. The solution of the inverse problem is obtained by minimizing a loss that involves: i) the residuals of the PDE evaluated at the control points and ii) an additional term representing the deviation between the solution field and measurements. We evaluate the framework on forward and inverse problems involving linear elliptic PDEs formulated on spline discretizations to assess both accuracy and computational efficiency. [1] M. Chasapi, P. Antolin., A. Buffa, Reduced order modelling of nonaffine problems on parameterized NURBS multipatch geometries, Lecture Notes in Computational Science and Engineering, (2024) 151, 67-87. [2] P. Karnakov, S. Litvinov, P. Koumoutsakos, Solving inverse problems in physics by optimizing a discrete loss: Fast and accurate learning without neural networks, PNAS Nexus, (2024), 2, 1-16.