Artificial intelligence is transforming industry, society, and science at an unprecedented pace, yet its long-term success depends on addressing two key challenges: reliability and sustainability. In this talk, we examine these challenges from a foundational perspective, highlighting intrinsic limits of current computing architectures. We first highlight the limitations of deep learning on traditional digital hardware, modeled as Turing machines, for inverse problems [1], which motivates the exploration of alternative paradigms such as analog and neuromorphic computing. In particular, we show that finite-dimensional inverse problems are not Banach-Mazur computable for small relaxation parameters. Even more, our results introduce a lower bound on the accuracy that can be obtained algorithmically. These results reveal unavoidable performance limits imposed by conventional digital architectures and underscore the need to rethink hardware design to enable more reliable and energy-efficient AI systems. Motivated by this observation, we consider AI systems on neuromorphic hardware architectures, an event-driven alternative to digital hardware. Spiking neural networks (SNNs) naturally arise as suitable models for neuromorphic AI implementation, though their theoretical understanding remains limited compared to artificial neural networks. We analyze discrete-time leaky integrate-and-fire (LIF) SNNs with static inputs, showing that they implement piecewise constant functions over polyhedral partitions of the input space and deriving bounds on network size required to approximate continuous functions [2]. We show how latency and depth affect partition complexity, highlighting differences from ANNs with piecewise linear activations. Finally, we examine (LIF) SNNs through Boolean function analysis, focusing on noise sensitivity and stability in classification [3]. Wide LIF-SNN classifiers are stable on average due to concentration of their Fourier spectrum on low-frequency components. This implies spectral simplicity (in the Fourier-Walsh sense) and offers insight into generalization in spiking networks. These results reveal fundamental constraints of digital computation and provide deeper theoretical insight into AI systems implemented on neuromorphic hardware.