Transfer operators such as the Koopman and Perron–Frobenius operators provide a powerful linear framework for analyzing nonlinear dynamical systems, with their spectral properties revealing dominant timescales, long-term behavior, and coherent structures. Despite their theoretical appeal, approximating transfer operators remains challenging. RaNNDy is a randomized neural network architecture to approximate spectral decompositions of transfer operators from data. The weights and biases of the hidden layers of the network in RaNNDy are randomly initialized and kept fixed; only the output layer is trained. This has several advantages over fully optimized neural networks, notably a significantly lower computational cost and a closed-form solution for the output layer. Despite these advantages, it is, however, restricted to the initial selection of weights and biases that act as the basis functions for the operator approximation. Selecting the optimal weights and biases of the hidden layers is important, as they have a direct influence on the quality of the operator approximation. We propose an algorithm to optimize the activation function itself while keeping the weights and biases of the randomized neural network fixed. The efficacy of the algorithm is illustrated with the aid of different benchmark problems, including stochastic differential equations and random walks on graphons.