Vibration signals from machinery are widely utilized for equipment fault diagnosis. The acquired vibration signals often carry a significant amount of noise, resulting in a low signal-to-noise ratio (SNR), which poses challenges to the diagnostic process. Existing diagnostic methods generally perform well in diagnosing faults from vibration signals with high SNR, but they often exhibit poor performance under low SNR conditions. In this paper, a fault diagnosis method based on the Minimize Residual strategy for extracting sub-Gaussian-like distribution components is proposed (MRSE). This method leverages the characteristic of fault signals tending to approximate Gaussian distributions at low SNRs. By using variance as the objective function and minimizing residuals through Lanczos decomposition for iterative filtering, the variance of the filtered signal is reduced to extract components in the signal that approximate Gaussian distributions, thereby obtaining the filtered signal containing the fault information. Simulation experiments demonstrate that MRSE outperforms traditional methods across all metrics. In practical experiments, MRSE has been shown to more effectively extract fault features from bearings and gearboxes compared to traditional methods.