We present a thermodynamically consistent neural constitutive framework for anisotropic inelasticity at finite strains that generalizes classical dual potential approaches beyond the restriction to convex potentials [1]. The formulation is based on a Helmholtz free energy and a dual inelastic potential defined on invariant sets of elastic, inelastic, and structural tensors, ensuring frame indifference, material symmetry, and independence of the intermediate configuration. Physical admissibility is enforced directly at the architectural level by embedding convexity and monotonicity constraints through a combination of Input Convex Neural Networks and newly proposed Input Monotonic Neural Networks. This enables the representation of non-convex yet thermodynamically admissible dissipation potentials. To address numerical instabilities associated with exponential-map integration during training, we introduce a Liquid Neural Network that approximates the implicit evolution of inelastic variables while preserving the multiplicative decomposition and the positive definiteness of inelastic stretches. For inference, the exponential update is reinstated to recover full physical consistency. The framework is assessed using synthetic data generated from anisotropic finite strain viscoelastic models and is integrated into a finite element solver. Numerical examples at both the material point and structural levels demonstrate accurate prediction of stresses, displacements, and reaction forces, as well as robust extrapolation to previously unseen loading paths. [1] Holthusen, H., & Kuhl, E. (2026). A complement to neural networks for anisotropic inelasticity at finite strains. Computer Methods in Applied Mechanics and Engineering, 450, 118612. https://doi.org/10.1016/j.cma.2025.118612