The National Research Institute for Earth Science and Disaster Resilience (NIED) has been developing the E-Simulator, a numerical simulation framework based on detailed finite element (FE) models to replicate the nonlinear seismic response of structures. However, due to its high computational cost, it is impractical to apply the E-Simulator directly to seismic risk assessment, which requires a large number of simulations, or to areal simulations involving many structures. Therefore, this study proposes a surrogate modeling approach for detailed FE models of building structures. In the proposed method, a machine learning approach is employed to reproduce nonlinear behaviour arising from elastoplasticity and damage. However, constructing a training dataset for an entire structure would require an impractically large number of large-scale simulation results. Therefore, surrogate models are constructed at the level of individual structural member units, such as beams, columns, beam-to-column connections, slabs, and walls. Each surrogate model of a member unit is then regarded as an element and assembled into a surrogate model of the entire structure. This framework is referred to as Surrogate Element Analysis. For surrogate modeling of each member unit, physical variable fields are reduced in dimensionality using proper orthogonal decomposition (POD), and deep neural networks incorporating physical laws, namely energy balance and stress-elastic strain relations, are employed. For assembling the member units, consistency conditions at the interfaces between units as well as Dirichlet boundary conditions are treated as constraints, and the constrained problem is reduced to a low-dimensional system using the POD–Galerkin method. To demonstrate the feasibility of the proposed method, a simplified model consisting of eight elements is selected as a member unit, and a surrogate element analysis model is constructed by assembling two member units. The prediction performance of both the surrogate model for the member unit and proposed assembling approach is then investigated.