In recent years, the field of Bayesian inference has seen an explosion of interacting particle methods. Such methods generally require a number of particles, initial position for each particle, and prescribed dynamics; the subsequent particle interactions result from comparing low- and high-probability regions of the target density. Like many Bayesian inference methods, these approaches face significant challenges when dealing with highly concentrated posteriors, multi-modality, non-differentiable likelihoods, and high-dimensional parameter spaces. To help mitigate these problems, we introduce interacting particle systems for Bayesian inference and generative modeling based on preconditioned gradient descent dynamics, particularly aiming to minimize the maximum mean discrepancy between the target density and the set of particles. By preconditioning a pseudo-physical system, we are able to achieve high-performance sampling methods which work in many settings still challenging Bayesians today. We demonstrate the efficacy of our methods on challenging low-dimensional examples, scaling it up to higher-dimensional PDE-based inference and, finally, the training of high-dimensional Bayesian neural networks.