When dynamical systems are coarse-grained, unresolved dynamics naturally manifest as stochasticity, dissipation, and history effects that must be treated exactly to preserve emergent non-equilibrium dynamics. Metriplectic brackets provide a rich framework for machine learning models which capture these effects, prescribing stochastic dynamics which consistent with bulk evolution of energy and entropy. We present a finite element framework to fit metriplectic brackets to observations of observable field evolution, using a discretize-then-optimize procedure to learn metriplectic perturbations of reversible brackets. While generally applicable to fluid and plasma systems, we specialize here to data-driven fracture mechanics for hierarchical materials, building on variational phase-field formulations of brittle fracture. Given observations of displacements, we present a self-supervised technique to identify a phase field damage evolution which exactly preserves the balance between kinetic, strain, and Griffiths energies while guaranteeing irreversible damage evolution. A cross-attention transformer parameterizing the bracket allows these structure preserving dynamics to be conditioned upon subgrid descriptions of microstructure. While fracture mechanics remains a pathological application for general neural operators due to minimal regularity, this approach yields data-driven fracture models which may extrapolate beyond training data.