There are few studies on topology optimization (TO) for unsteady or transient phenomena, especially those based on topology derivatives. First, following Bonnet's previous research on scalar wave problems as well as elastodynamic problems, this presentation derives topology derivatives for the damped wave equation (Cattaneo equation) using the adjoint variable method and constructs TO using the level set method. The time-domain boundary element method, which the author recently developed, is used to solve the primal and adjoint problems, enabling TO to be performed for external scattering problems. Furthermore, the construction of TO for the time-domain Maxwell's equations for perfect conductors is also mentioned with regard to the fast time-domain boundary element method.