A new numerical model is proposed by combining the framework of the extended multiscale FEM and the scaled boundary finite element method (SBFEM) for the multiscale heterogeneous analysis of mechanical and heat transfer problems. Scaled Boundary Finite Element (SBFE) and image-based quadtree or octree gridding, the construction of base functions can be conveniently and efficiently conducted particularly for the heterogeneous media with complex geometries or singularity of stress or heat flux at the small-scale, and the solution accuracy at the large-scale can be improved by increasing nodes of coarse element only without increasing new nodes inside. Consequently, the SBFEM based solutions at the large-scale can be solved with a reduced order and sufficient accuracy, and the solutions at the small-scale can be achieved using SBFEM based base functions. We have applied the proposed method into steady and transient heat transfer problems, elastic and viscoelastic problems, and developed special computational scheme for nonlinear problems, and both 2D and 3D problems can be solved. The proposed method provides an efficient approach for multiscale analysis with automatic mesh generation and fast computation in different scales, having potential in a serials of important engineering problems.