An acoustic metasurface is a kind of two-dimensional metamaterials with microstructures of subwavelength. It can modulate the phase and amplitude of acoustic waves and thus has potential applications in various fields. Its design is generally based on Snell’s law, lattice diffraction or surface impedance theory. However, when a metasurface is used to realize complex functionalities, such as holographic imaging, its design is so challenging that it has to be implemented through inverse design. In this paper, we develop a customized inverse design methodology of acoustic metasurefaces for holographic imaging and particle manipulation. The metasurfaces are constructed by elements of several types with the required phase and amplitude modulation ability. Firstly, the mapping relationship from the metasurface to the holographic plane or radiation force/torque are established by using Huygens-Fresnel's principle. Then a customized inverse design framework for the structures of elements is developed based on the optimization method with the required phase-shift and transmittance as targets. The layout of the elements with different phase shifts and transmittances is optimized to achieve holographic imaging. For particle manipulation, the element arrangement is optimized to achieve required radiation force and torque to suspend and rotate a small particle simultaneously. Numerical simulations and experiments are performed to verify the developed design methodology.