Crack initiation at stress concentrators is a dominant cause of catastrophic and unstable failure in brittle and quasi-brittle structures. For safety-critical components, reliable prediction therefore requires not only accurate physical modeling but also a systematic treatment of uncertainties in material properties and geometry. In this context, uncertainty quantification (UQ) is essential to move beyond purely deterministic safety factors toward risk-informed assessments of crack initiation. Finite Fracture Mechanics (FFM) provides an efficient and physically well-founded framework for modeling crack initiation, as it drops the incremental crack growth assumption of classical Linear Elastic Fracture Mechanics by accounting for the instantaneous formation of cracks of finite size. Crack initiation is governed by a coupled stress and energy criterion (CC) that simultaneously enforces a local strength condition and an energetic fracture condition. For known stress and energy functions, this CC can be solved very efficiently. Based on this formulation, sensitivity analyses are first performed to identify the dominant parameters governing crack initiation under varying structural conditions. Owing to the high computational efficiency of the CC-FFM framework, probabilistic uncertainty quantification can subsequently be carried out using sampling-based approaches. Monte Carlo simulations with Latin Hypercube Sampling are employed and implemented using the open-source UQpy package to propagate uncertainties in material and geometric parameters. The methodology is applied to several representative structural configurations, including center cracks and elliptical holes in infinite domains, negative geometries characterized by non-monotonic energy release behavior, and adhesively bonded joints. The results highlight variations in sensitivity patterns between strength-dominated and energy-dominated crack initiation and demonstrate how these differences govern the uncertainty propagation in probabilistic crack initiation analyses. Overall, the study shows that Finite Fracture Mechanics provides a robust and physically transparent basis for combined deterministic and probabilistic analyses of crack initiation in varying structural systems.