Compaction control is carried out to verify that the material has been sufficiently compacted and that as little post-compaction as possible will occur. However, current control methods are heavily dependent on project-specific conditions. A critical knowledge gap exists regarding a key performance indicator: the remaining compaction potential. This parameter, which indicates how much further the material can be densified, has previously received limited attention. In this study, the highly nonlinear and dynamic response of a roller-soil interaction system during vibratory compaction was investigated using a coupled multibody dynamics and finite element method (MBD-FEM) approach. The soil behavior was simulated using a hypoplastic constitutive model with intergranular strain to capture density evolution. Both the achieved compaction effect and the remaining compaction potential were quantified by normalized void ratio changes. Extensive simulations were conducted to generate a dataset, involving varying soil properties and compaction parameters. These results were then used to train a machine learning (ML) model that estimates relative void ratios based on roller parameters and selected drum acceleration features. The results demonstrate that the MBD-FEM model realistically simulates surface deformation, depth-dependent strain response, and densification profiles. Furthermore, the developed ML model successfully estimates both the achieved compaction effect and the remaining compaction potential, showing general applicability across various soil conditions and machine types. These findings enhance the fundamental understanding of vibratory compaction mechanisms and contribute to the advancement of continuous compaction control (CCC) technology.