Dynamical systems are used in real-world applications to model complex time-varying systems. Despite being broadly used, exploiting the historical effects carried by the data for inversion purposes is an understudied question. Traditional methods often assume simplified settings (linear or instantaneous models for instance) and do not fully exploit history while more sophisticated ones often lack explicit Bayesian formalism. In this work, we are interested in reconstructing the time evolution of latent environment variables whose effects are reflected with a certain inertia on a system’s output (ie. the latter is a function that rely not only on the present or instantaneous input but also on its past values). More precisely, we propose a simple and yet effective formalism for a precise Bayesian inversion for time-series with historical outputs, and investigate three inversion set-ups. We first consider the scenario where the input-output relationship is approximated based on a standard non-linear model relying solely on the instantaneous input. In such a set-up, the model captures a part of historical effects via the temporal correlation of the input process and although it can achieve good prediction results, we highlight that its usability becomes limited for inversion tasks. Then, we consider another class of models based on linear history (also called distributed lag models) that are often exclusively used for prediction in the literature. We demonstrate to what extent they are useful to handle the inversion problematic being studied and are sometimes sufficient to tackle it. Finally, for an enhanced accuracy and more versatility, we consider a combination of the two aforementioned set-ups where the linear history model is complemented with instantaneous non-linearities. We evaluate this framework on synthetic data and demonstrate its effectiveness in providing better inversion results in different application cases inspired from models describing the dynamics of biological systems and/or engineering systems.