This work presents a novel approach for designing multi-alloy structures by simultaneously optimizing the material and topology. The proposed computational framework employs a material latent space integrated with density-based topology optimization in a two-step process. In the first step, a variational autoencoder (VAE), a type of neural network, maps a database of materials and their properties to a continuous, low-dimensional latent space. In the second step, the latent space is coupled with topology design variables (pseudo-densities) to simultaneously optimize material and topology. A gradient-based optimizer, specifically the method of moving asymptotes (MMA), traverses the latent space to select the material while simultaneously optimizing the topology. Optionally, one can add a penalization term that will drive the latent points towards real materials from the dataset. The framework is illustrated through 2D and 3D numerical examples using up to twenty materials, involving more than a million degrees of freedom.