Bayesian inference provides a probabilistic framework for quantifying uncertainties and updating them from data, and is widely used to address inverse problems in science and engineering. In this setting, uncertain input parameters of computational models are inferred from noisy and scarce measurements, through the specification of a prior distribution and its update into a posterior distribution via conditioning on observations. However, the choice of the prior distribution constitutes a critical modeling assumption, and may significantly influence inference results. Robust Bayesian Analysis aims at studying the sensitivity of Bayesian inference results, including the prior distribution. In particular, Bayesian perturbed-law based sensitivity indices (BPLI) were recently introduced to quantify the influence of prior marginal distributions. These indices rely on perturbations defined using the Fisher distance, which provides a rigorous mathematical framework. The impact of a prior marginal is characterized by estimating extremal values of BPLI over a Fisher sphere. Nevertheless, these extremal values are estimated through discretizing the Fisher sphere into a finite set of samples, which may scale poorly with dimension. In this work, we propose a more robust and efficient method for estimating extremal values of BPLI, based on numerical optimization over Fisher spheres. We show that the problem can be reformulated as an optimization over the unit sphere of R^n, enabling the use of Riemannian optimization algorithms. The method is applied to a Bayesian inverse problem from engineering, involving the identification of uncertain parameters of a finite element model of a nuclear fuel assembly, from dynamic impact experiments. The forward model relies on nonlinear transient simulations performed with the code_aster finite element solver. Results demonstrate the tractability of the approach for nonlinear Bayesian inverse problems and emphasize that BPLI effectively identify parameters for which prior distribution choices have a significant impact.