基于残差注意力TCN与vision transformer的齿轮剩余寿命预测

胡爱军; 李晨阳; 邢磊; 周卓浩; 向玲

doi:10.13224/j.cnki.jasp.20240284

基于残差注意力TCN与vision transformer的齿轮剩余寿命预测

doi: 10.13224/j.cnki.jasp.20240284

华北电力大学能源动力与机械工程学院机械工程系河北省电力机械装备健康维护与失效预防实验室，河北保定 071003

基金项目: 国家自然科学基金（52175092）

详细信息

作者简介:
胡爱军（1971-），男，教授、博士生导师，博士，主要从事机械设备状态监测与故障诊断研究

中图分类号: V240.2；TH132；TH133.33
计量
- 文章访问数: 535
- HTML浏览量: 421
- PDF量: 47
- 被引次数: 0
出版历程
- 收稿日期: 2024-05-07
- 网络出版日期: 2025-09-24

Gear remaining life prediction based on residual attention TCN and vision transformer

Hebei Key Laboratory of Electric Machinery Health Maintenance and Failure Prevention，Department of Mechanical Engineering，School of Energy Power and Mechanical Engineering，North China Electric Power University，Baoding Hebei 071003，China

摘要

摘要:
齿轮系统的运行状况受到多个因素的影响，这些因素在时间上存在长期依赖关系，并在局部和全局特征之间存在差异。为了有效地捕捉数据中的时间依赖性并自适应调整对特征的关注度，提出具有残差卷积块注意力机制的时间卷积网络（RCMTCN）。通过在卷积块注意力机制中引入残差连接，模型能够同时关注原始输入和注意力加权的信息，提高了模型对局部信息的感知能力。在此基础上，将vision transformer（ViT）模型与RCMTCN相结合对齿轮的剩余使用寿命（RUL）预测，ViT模型能有效地捕获数据中的全局信息。两者融合后能充分展现在处理时间序列数据局部特征提取能力和全局信息关注方面的优势，提高对多维度特征的感知能力。最后，通过在两种工况齿轮性能退化数据集上对模型进行验证，选用点蚀故障数据进行训练，分别对点蚀和断齿故障进行测试。实验结果表明：与其他方法相比，所提出的方法能更充分地提取关键特征信息，在点蚀故障上评分函数得分为0.8898，且在断齿故障上得分为0.8587，表现出良好的工况、故障适应能力。
- 齿轮 /
- 剩余使用寿命 /
- 时序网络 /
- 注意力机制 /
- vision transformer模型
Abstract:
The operating condition of a gear system is influenced by multiple factors exhibiting long-term dependencies over time and differences between local and global features. In order to effectively capture the temporal dependence in data and adaptively adjust the attention to features, a temporal convolutional network (RCMTCN) with residual convolutional block attention mechanism was proposed. By introducing residual connections into the convolutional block attention mechanism, the model can jointly emphasize the original input and attention-weighted information, and improve the model’s ability to perceive local information. On this basis, the vision transformer (ViT) model was combined with RCMTCN to predict the remaining service life (RUL) of gears. The ViT model can effectively obtain the global information in the data. The fusion of these two can fully demonstrate its advantages in local feature extraction capabilities and global information attention in processing time series data, and improve the perception of features of various scales. Finally, the model was verified on two working conditions gear performance degradation datasets, pitting corrosion fault data were selected for training, and pitting corrosion and tooth broken faults were tested respectively. Experimental results showed that compared with other methods, the proposed method can more fully extract key feature information. The scoring function achieved 0.8898 for pitting failure and 0.8587 for broken tooth failure, indicating excellent operational conditions and fault adaptability.
- gear /
- remaining useful life /
- temporal network /
- attention mechanism /
- vision transformer model

HTML

图 1 RCMTCN模型框架

Figure 1. RCMTCN model framework

下载: 全尺寸图片幻灯片

图 2 残差卷积块注意力机制模块

Figure 2. Residual convolutional block attention module

下载: 全尺寸图片幻灯片

图 3 ViT模型框架

Figure 3. ViT model framework

下载: 全尺寸图片幻灯片

图 4 RCMTCN-ViT整体网络结构图

Figure 4. RCMTCN-ViT overall network structure diagram

下载: 全尺寸图片幻灯片

图 5 频域累积幅值特征

Figure 5. Cumulative amplitude characteristics in frequency domain

下载: 全尺寸图片幻灯片

图 6 描述齿轮退化趋势的3种标签

Figure 6. Three labels describing gear degradation trends

下载: 全尺寸图片幻灯片

图 7 重庆大学齿轮性能退化数据集

Figure 7. Gear performance degradation dataset from Chongqing University

下载: 全尺寸图片幻灯片

图 8 不同工况齿轮预测结果

Figure 8. Gear prediction results under different working conditions

下载: 全尺寸图片幻灯片

图 9 不同工况的MAE与RMSE雷达图

Figure 9. MAE and RMSE radar charts under different working conditions

下载: 全尺寸图片幻灯片

图 10 隐藏层的特征可视化

Figure 10. Feature visualization of hidden layers

下载: 全尺寸图片幻灯片

图 11 不同方法的RMSE、MAE和Score结果图

Figure 11. RMSE, MAE and Score result graphs of different methods

下载: 全尺寸图片幻灯片

表 1 数据集的详细信息

Table 1. Dataset details

项目	数值或说明
数据集名称	G1	G2	G3	G4	G5
故障类型	点蚀				断齿
转矩/（N·m）	1300	1300	1300	1300	1400
转速/（r/min）	1000	1000	1000	1000	500
样本数	600	600	600	600	400
数据集标识	训练集			测试集

下载: 导出CSV

表 2 超参数设置

Table 2. Hyperparameter settings

超参数	数值或说明
批次大小	128
迭代次数	200
学习率	0.0001
优化器	Adam
损失函数	MSE
卷积核大小	3×3
编码器层数	8
多头注意力头数	8
随机失活率	0.1
隐藏层神经元数	256

下载: 导出CSV

表 3 所提方法与其他5种模型的评价指标结果

Table 3. Evaluation index results of the proposed method and other five solutions

预测模型	G4			G5
预测模型	RMSE	MAE	Score	RMSE	MAE	Score
CMTCN	0.0491	0.0398	0.5677	0.0413	0.0338	0.5717
RCMTCN	0.0226	0.0203	0.6715	0.0281	0.0226	0.6353
ViT	0.0333	0.0334	0.6853	0.0358	0.0310	0.5995
TCN-ViT	0.0270	0.0230	0.7447	0.0295	0.0262	0.6626
CMTCN-ViT	0.0343	0.0313	0.6919	0.0316	0.0270	0.6920
RCMTCN-ViT	0.0110	0.0089	0.8898	0.0126	0.0099	0.8587

下载: 导出CSV

表 4 所提方法与其他方法的评价指标结果

Table 4. Evaluation index results of the proposed method and other methods

方法	G4			G5
方法	RMSE	MAE	Score	RMSE	MAE	Score
WDCNN	0.0815	0.0652	0.3756	0.079	0.0591	0.4474
LSTM	0.0403	0.0328	0.5817	0.0491	0.0398	0.5677
GRU	0.0427	0.0379	0.5591	0.0576	0.0443	0.4698
TCN	0.0565	0.0468	0.4699	0.0327	0.0373	0.5463
RCMTCN-ViT	0.0110	0.0089	0.8898	0.0126	0.0099	0.8587

下载: 导出CSV

参考文献(29)

[1]	CHEN Jiaxian, LI Dongpeng, HUANG Ruyi, et al. Aero-engine remaining useful life prediction method with self-adaptive multimodal data fusion and cluster-ensemble transfer regression[J]. Reliability Engineering & System Safety, 2023, 234: 109151.
[2]	朱永超, 朱才朝, 谭建军, 等. 考虑数据特征差异的风电齿轮箱群组健康状态预测研究[J]. 机械工程学报, 2024, 60(18): 64-75. ZHU Yongchao, ZHU Caichao, TAN Jianjun, et al. Health status prediction of wind turbine gearbox cluster considering data distribution discrepancy[J]. Journal of Mechanical Engineering, 2024, 60(18): 64-75. (in Chinese ZHU Yongchao, ZHU Caichao, TAN Jianjun, et al. Health status prediction of wind turbine gearbox cluster considering data distribution discrepancy[J]. Journal of Mechanical Engineering, 2024, 60(18): 64-75. (in Chinese)
[3]	DING Wanmeng, LI Jimeng, MAO Weilin, et al. Rolling bearing remaining useful life prediction based on dilated causal convolutional DenseNet and an exponential model[J]. Reliability Engineering & System Safety, 2023, 232: 109072.
[4]	NIE Lei, ZHANG Lvfan, XU Shiyi, et al. Remaining useful life prediction for rolling bearings based on similarity feature fusion and convolutional neural network[J]. Journal of the Brazilian Society of Mechanical Sciences and Engineering, 2022, 44(8): 328. doi: 10.1007/s40430-022-03638-0
[5]	张超, 庞永志, 王巍智, 等. 基于CNN-GRU模型的齿轮寿命预测[J]. 机床与液压, 2023, 51(2): 11-16. ZHANG Chao, PANG Yongzhi, WANG Weizhi, et al. Gear life prediction based on CNN-GRU model[J]. Machine Tool & Hydraulics, 2023, 51(2): 11-16. (in Chinese ZHANG Chao, PANG Yongzhi, WANG Weizhi, et al. Gear life prediction based on CNN-GRU model[J]. Machine Tool & Hydraulics, 2023, 51(2): 11-16. (in Chinese)
[6]	WAN Lanjun, CHEN Keyu, LI Yuanyuan, et al. A novel remaining useful life prediction method based on CEEMDAN-IFTC-PSR and ensemble CNN/BiLSTM model for cutting tool[J]. IEEE Access, 2022, 10: 2182-2195. doi: 10.1109/ACCESS.2021.3140165
[7]	ZHANG Jiusi, TIAN Jilun, LI Minglei, et al. A parallel hybrid neural network with integration of spatial and temporal features for remaining useful life prediction in prognostics[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 72: 3501112.
[8]	HUANG Chenggeng, HUANG Hongzhong, LI Yanfeng, et al. A novel deep convolutional neural network-bootstrap integrated method for RUL prediction of rolling bearing[J]. Journal of Manufacturing Systems, 2021, 61: 757-772. doi: 10.1016/j.jmsy.2021.03.012
[9]	ZHAO Minghang, ZHONG Shisheng, FU Xuyun, et al. Deep residual shrinkage networks for fault diagnosis[J]. IEEE Transactions on Industrial Informatics, 2020, 16(7): 4681-4690. doi: 10.1109/TII.2019.2943898
[10]	PASCANU R, MIKOLOV T, BENGIO Y. On the difficulty of training recurrent neural networks[EB/OL]. (2013-02-16)[2024-05-07]. https://arxiv.org/abs/1211.5063.
[11]	BAI Shaojie, ZICO KOLTER J, KOLTUN V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling[EB/OL]. (2018-04-19)[2024-05-07]. https://arxiv.org/abs/1803.01271.
[12]	WU Pin, SUN Junwu, CHANG Xuting, et al. Data-driven reduced order model with temporal convolutional neural network[J]. Computer Methods in Applied Mechanics and Engineering, 2020, 360: 112766. doi: 10.1016/j.cma.2019.112766
[13]	CAO Y, DING Y, JIA M, et al. A novel temporal convolutional network with residual self-attention mechanism for remaining useful life prediction of rolling bearings[J]. Reliability Engineering & System Safety, 2021, 215: 107813.
[14]	HE Ke, SU Zuqiang, TIAN Xiaoqing, et al. RUL prediction of wind turbine gearbox bearings based on self-calibration temporal convolutional network[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 3501912.
[15]	SUN Heng, XIA Min, HU Yawei, et al. A new sorting feature-based temporal convolutional network for remaining useful life prediction of rotating machinery[J]. Computers and Electrical Engineering, 2021, 95: 107413. doi: 10.1016/j.compeleceng.2021.107413
[16]	WANG Wei, ZHOU Gongbo, MA Guoqing, et al. A novel competitive temporal convolutional network for remaining useful life prediction of rolling bearings[J]. IEEE Transactions on Instrumentation and Measurement, 2023, 72: 3523612.
[17]	WANG Yiwei, DENG Lei, ZHENG Lianyu, et al. Temporal convolutional network with soft thresholding and attention mechanism for machinery prognostics[J]. Journal of Manufacturing Systems, 2021, 60: 512-526. doi: 10.1016/j.jmsy.2021.07.008
[18]	WANG Haitao, YANG Jie, WANG Ruihua, et al. Remaining useful life prediction of bearings based on convolution attention mechanism and temporal convolution network[J]. IEEE Access, 2023, 11: 24407-24419. doi: 10.1109/ACCESS.2023.3255891
[19]	ZHANG X, LIU C, YANG D, et al. RFAConv: innovating spatial attention and standard convolutional operation[EB/OL]. (2024-03-28)[2024-05-07]. https://arxiv.org/abs/2304.03198.
[20]	ZHAI Xiaohua, KOLESNIKOV A, HOULSBY N, et al. Scaling vision transformers[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, US: IEEE, 2022: 1204-1213.
[21]	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: transformers for image recognition at scale[EB/OL]. (2021-06-03)[2024-05-07]. https://arxiv.org/abs/2010.11929?featured_on=pythonbytes.
[22]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, US: IEEE, 2016: 770-778.
[23]	YAO Qingtao, LI Xianze, XIANG Ling, et al. A new condition-monitoring method based on multi-variable correlation learning network for wind turbine fault detection[J]. Measurement Science and Technology, 2023, 34(2): 024009. doi: 10.1088/1361-6501/ac9f5d
[24]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[R]. Munich, Germany: 15th European Conference on Computer Vision, 2018.
[25]	向玲, 王凯伦, 胡爱军, 等. 基于Dy Res Net-CBAM网络的滚动轴承剩余寿命预测[J]. 中国工程机械学报, 2023, 21(1): 6-11. XIANG Ling, WANG Kailun, HU Aijun, et al. Remaining life prediction of rolling bearing based on Dy Res Net-CBAM network[J]. Chinese Journal of Construction Machinery, 2023, 21(1): 6-11. (in Chinese XIANG Ling, WANG Kailun, HU Aijun, et al. Remaining life prediction of rolling bearing based on Dy Res Net-CBAM network[J]. Chinese Journal of Construction Machinery, 2023, 21(1): 6-11. (in Chinese)
[26]	DENG Feiyue, BI Yan, LIU Yongqiang, et al. Remaining useful life prediction of machinery: a new multiscale temporal convolutional network framework[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 2516913.
[27]	LI Xiang, DING Qian, SUN Jianqiao. Remaining useful life estimation in prognostics using deep convolution neural networks[J]. Reliability Engineering & System Safety, 2018, 172: 1-11.
[28]	JIANG Changhong, LIU Xinyu, LIU Yizheng, et al. A method for predicting the remaining life of rolling bearings based on multi-scale feature extraction and attention mechanism[J]. Electronics, 2022, 11(21): 3616. doi: 10.3390/electronics11213616
[29]	QIN Yi, XIANG Sheng, CHAI Yi, et al. Macroscopic-microscopic attention in LSTM networks based on fusion features for gear remaining life prediction[J]. IEEE Transactions on Industrial Electronics, 2020, 67(12): 10865-10875. doi: 10.1109/TIE.2019.2959492