132 / 2025-05-21 15:25:29

Towards Low-speed Machinery Incremental Fault Diagnosis based on Inverted Transformer with Learnable Pruning Mechanism

Inverted transformer, lifelong learning, multiple sensors, stability-plasticity, low-speed fault diagnosis

Special Sessions > SSL 16 Physics-inspired Multi-modal Intelligent Fault Diagnosis of Machinery

it is a non-negligible issue for catastrophic forgetting to perform low-speed bearing fault diagnosis, whereas previously learned feature significantly affects the model’s performance facing with challenges related to the fault information increments. In terms of issues, a new lifelong learning based on inverted transformer with learnable pruning mechanism is proposed to enhance adaptability facing with multiple fault information increments. The backbone of diagnosis model effectively learned global information perception and local information refinement in signals of multiple sensors through the multi-head inverted attention in inverted transformer. The learnable pruning mechanism, consisting with dynamic exemplar selection and pruning mechanism, effectively assists in balancing the memory and learning capabilities, that is, consolidating the stability-plasticity of the model. The former is performed to adjust the retention and utilization of exemplars in the memory bank, thereby keeping memory through the exemplars' diversity, mitigating catastrophic forgetting. Furthermore, the latter is applied to address the dilemma caused by predefined and fixed structures in the previous stage throughout the entire training process of the model. The effectiveness and feasibility of the proposed method is validated on low-speed machinery (two cases).