Autoregressive and recurrent sequence models underpin many recent advances in artificial intelligence, enabling applications ranging from large language models to molecular generation and neural surrogate modeling for partial differential equations. Despite their strong empirical performance, these models typically lack reliable mechanisms for assessing predictive uncertainty, particularly when operating outside the training distribution. This limitation poses critical challenges in scientific and engineering applications, where understanding model confidence is as important as predictive accuracy. In this work, we introduce an efficient Bayesian approach for uncertainty quantification in sequence models based on variational adaptive dropout. The proposed method provides a principled and scalable way to convert any neural surrogate model— including autoregressive and recurrent architectures—into a Bayesian model capable of producing meaningful and well-calibrated uncertainty estimates. By leveraging variational inference, the approach remains computationally efficient and readily applicable to large-scale sequence models commonly used in practice. We evaluate the proposed framework on a range of Scientific Machine Learning tasks, with a particular focus on neural PDE surrogate modeling and molecular generation. Experimental results demonstrate that variational adaptive dropout yields accurate predictions while significantly improving uncertainty awareness, especially in out-of-distribution regimes.