Filling the Gaps: Video Inpainting for Fluid Flow Reconstruction
Please login to view abstract download link
Access to reliable and complete experimental data is essential for understanding fluid dynamics phenomena. However, due to experimental limitations, the data obtained are often limited or incomplete, exhibiting gaps, missing entries, and other imperfections that hinder their analysis. As a result, the development of methods for repairing and/or augmenting experimental data is of great interest [1,2]. From a data-driven perspective, this reconstruction task can be seen within the paradigm of image inpainting. In the machine learning and deep learning literature, image inpainting refers to the process of filling missing regions of an image with synthetically generated content that is consistent with the observed data. Successful inpainting requires the reconstructed pixels to preserve both local and global characteristics of the original image, including colour distributions, shapes, textures, and semantic structures. When extended to time-resolved data (i.e. videos), this paradigm becomes video inpainting, which further imposes the requirement of temporal coherence across consecutive frames. Similarly, repairing a gappy dataset of a fluid flow requires to preserve fluid structures and temporal coherence. Therefore, the use of video inpainting techniques seems adequate. In this work, we employ the Spatial-Temporal Transformer Network (STTN) [3] to address the reconstruction of incomplete fluid flow data. The STTN leverages a transformer-based architecture to model long-range spatio-temporal dependencies, enabling coherent reconstruction across missing regions. We demonstrate that this approach yields improved pointwise accuracy and better preservation of spectral content compared to classic reconstruction methods, highlighting its potential for the repair and augmentation of experimental fluid flow data. [1] Buzzicotti, M. (2023). Data reconstruction for complex flows using AI: Recent progress, obstacles, and perspectives. Europhysics Letters, 142(2), 23001. [2] Saini, P., Arndt, C. M., & Steinberg, A. M. (2016). Development and evaluation of gappy-POD as a data reconstruction technique for noisy PIV measurements in gas turbine combustors. Experiments in Fluids, 57(7), 122. [3] Zeng, Y., Fu, J., & Chao, H. (2020, August). Learning joint spatial-temporal transformations for video inpainting. In European conference on computer vision (pp. 528-543). Cham: Springer International Publishing.
