The problem of learning operators from input-output data is fundamental to many areas of scientific computing and system identification. In this work, we consider the specific inverse problem of recovering an unknown kernel function that governs a nonlocal operator, given a dataset of noisy observations. This formulation encompasses a wide range of applications, from constitutive modeling in continuum mechanics to the identification of nonlocal interaction laws in particle systems. Based on this problem, we aim to address two fundamental questions in scientific machine learning: 1) Is the smoother or rougher data more favorable for a robust operator learning? 2) How is data quality, such as its regularity and noise levels, going to impact the learning results? The primary objective of this work is to rigorously quantify how the roughness of input data influences the learnability of kernels in nonlocal operators. We posit that rough data expands the range of learnable kernels by slowing the spectral decay of the data operator, thereby increasing the effective dimension of the inverse problem. Firstly, a theoretical bridge between the continuous regularity of the input data and the spectral decay rate of the discretized operator is established. We define the effective dimension of the inverse problem via the spectrum of the normal operator, and analyze how it scales with the regularization parameter. It was found that rougher data leads to a slower polynomial decay of eigenvalues, denoted by a smaller decay parameter. Second, we derive explicit convergence rates for the estimation error in the small noise limit. By analyzing the bias-variance trade-off under a source condition characterized by the kernel's smoothness, we demonstrate that the optimal convergence rate depends critically on the interplay between the kernel smoothness and the data roughness. The analysis is numerical verified using both Tikhonov regularized estimators and neural network-based approaches. We confirm that as the data becomes rougher, the effective dimension increases, enabling the accurate recovery of increasingly complex kernel functions and the corresponding nonlocal operators.