基于多尺度CNN和Transformer的航空发动机寿命预测方法

段佳俊; 陆中; 王捷

doi:10.13224/j.cnki.jasp.20240311

基于多尺度CNN和Transformer的航空发动机寿命预测方法

doi: 10.13224/j.cnki.jasp.20240311

南京航空航天大学民航学院，南京 211106

基金项目: 国家重点研发计划（2023YFB4302403）；南京航空航天大学基本科研业务费国际科研合作伙伴培育基金（NG2023003）；南京航空航天大学科研与实践创新计划（xcxjh20230735）

详细信息

作者简介:
段佳俊（1999-），男，硕士，研究方向为航空发动机剩余寿命预测。E-mail：nostalgic_jj@nuaa.edu.cn

通讯作者:
陆中（1980-），男，教授，博士，研究方向为飞机系统安全性评估、系统可靠性工程。E-mail：luzhong@nuaa.edu.cn

中图分类号: V240.2
计量
- 文章访问数: 807
- HTML浏览量: 561
- PDF量: 48
- 被引次数: 0
出版历程
- 收稿日期: 2024-05-15
- 网络出版日期: 2025-12-26

Aero-engine life prediction method based on multi-scale CNN and Transformer

College of Civil Aviatio，Nanjing University of Aeronautics and Astronautics，Nanjing 211106，China

摘要

摘要:
目前在航空发动机剩余寿命预测中，通常仅考虑了单一尺度退化特征，并且在处理具有长期依赖关系的时序数据时效率低下。提出了一种基于多尺度CNN和Transformer的航空发动机剩余寿命预测方法。在传统CNN的基础上设计了多尺度特征提取模块，挖掘并融合了不同尺度下的退化特征；构造了特征连接模块，弥补了传统特征提取模块只能从同一层面提取特征的不足；基于Transformer的位置编码、多头自注意力机制和前馈神经网络实现了时序数据长期依赖特征的学习，利用多头注意力机制的并行计算能力降低了网络的运行时间。在C-MAPSS数据集上进行了实验验证，结果表明：所提方法与未提取多尺度特征的网络相比误差更小；并且相对于按顺序处理时序数据的RNN、LSTM和GRU等网络，运算时间分别降低了65.77%、48.41%和45.02%。
- 剩余寿命预测 /
- 卷积神经网络 /
- Transformer架构 /
- 多尺度特征提取 /
- 注意力机制
Abstract:
Multi-scale degradation features are seldom considered in remaining life prediction for aero-engine, and it is inefficient in dealing with time-series data with long-term dependencies. A remaining life prediction method is proposed based on multi-scale CNN and Transformer. A multi-scale feature extraction module is designed by traditional CNN, which extracts and integrates degradation features at different scales. A feature connection module is constructed to make up for the shortage that traditional feature extraction module can only extract features from the same level. The learning of long-term dependency features for time-series data is realized based on the position encoding, the multi-head self-attention mechanism and the feed-forward neural network of the Transformer. The parallel computing of the multi-head attention mechanism is utilized to reduce network runtime. Experimental validation is carried out on the C-MAPSS dataset, and the results show that our proposed method has less error compared with the network without extracting multi-scale features. Compared with RNN, LSTM and GRU, the operation time is reduced by 65.77%, 48.41%, and 45.02%, respectively.
- remaining life prediction /
- convolutional neural network /
- Transformer architecture /
- multi-scale feature extraction /
- attention mechanism

HTML

图 1 模型训练过程

Figure 1. Flow chat of model training process

下载: 全尺寸图片幻灯片

图 2 多传感器的滑动时间窗口处理

Figure 2. Multi-sensor sliding time window processing

下载: 全尺寸图片幻灯片

图 3 MSCNNformer框架图

Figure 3. MSCNNformer framework diagram

下载: 全尺寸图片幻灯片

图 4 多尺度特征提取模块

Figure 4. Multi-scale feature extraction module

下载: 全尺寸图片幻灯片

图 5 多头注意力机制

Figure 5. Multi-head attention mechanism

下载: 全尺寸图片幻灯片

图 6 不同MFE与Encoder层数的结果

Figure 6. Results from different MFE & Encoder layers

下载: 全尺寸图片幻灯片

图 7 不同卷积核大小组合的结果

Figure 7. Results of different convolution kernel size combinations

下载: 全尺寸图片幻灯片

图 8 测试集的预测偏差分布（FD001～FD004）

Figure 8. Prediction deviation distribution of the test set （FD001—FD004）

下载: 全尺寸图片幻灯片

图 9 单台发动机的RUL预测结果（FD001～FD004）

Figure 9. RUL prediction for a single engine （FD001—FD004）

下载: 全尺寸图片幻灯片

表 1 预处理后的输入样本基本情况

Table 1. Preprocessed input sample details and labels

数据集	FD001	FD002	FD003	FD004
训练集中发动机数	100	260	100	249
训练集中样本数	17731	48819	21820	56518
测试集中发动机数	100	259	100	248
运行工况	1	6	1	6
故障模式	HPC	HPC	HPC&FAN	HPC&FAN
输入时间窗尺寸	30×9	20×10	30×9	20×10

下载: 导出CSV

表 2 不同网络的训练时间

Table 2. Training time of different network

方法	时间/s
本文	1241.41
RNN	3627.04
LSTM	2257.98
Bi-LSTM	2846.60
GRU	2406.51
Bi-GRU	2657.36

下载: 导出CSV

表 3 本文网络的最优参数

Table 3. Optimal parameters of the network

参数	FD001	FD002	FD003	FD004
最大周期数	100	100	100	100
批大小	512	512	512	512
窗口长度T	30	20	30	20
L2正则化	0.001	0.001	0.0006	0.008
R_early	125	125	125	125
d_model	512	512	512	512
头的数量	8	8	8	8
学习率	0.0001	0.0001	0.0003	0.0001
等待周期数	10	10	10	10
激活函数	ReLU	ReLU	ReLU	ReLU
前馈层上的神经元数量	512/128	512/128	512/128	512/128

下载: 导出CSV

表 4 不同方法的RMSE对比

Table 4. RMSE comparison of different methods

方法	FD001	FD002	FD003	FD004
CNN^[3]	18.45	30.29	19.82	29.16
DCNN^[4]	12.61	22.36	12.64	23.31
HDNN^[6]	13.02	15.24	12.22	18.16
BLSTM-CNN^[8]	12.13	16.01	11.96	18.10
BiGRU-TSAM^[10]	12.56	18.94	12.45	20.47
MS-DCNN^[11]	11.44	19.35	11.67	22.22
MSTformer^[15]		14.48		15.03
MSFMTP^[20]	13.24	14.83	11.17	14.09
本文	11.26	13.65	12.63	13.53

下载: 导出CSV

表 5 不同方法的Score得分对比

Table 5. Score comparison of different methods

方法	FD001	FD002	FD003	FD004
CNN^[3]	1287	13570	1596	7886
DCNN^[4]	273	10412	284	12466
HDNN^[6]	245	1282	288	1527
BLSTM-CNN^[8]	174	1230	242	1513
BiGRU-TSAM^[10]	213	2264	233	3610
MS-DCNN^[11]	196	3747	241	4844
MSTformer^[15]		1099		1012
MSFMTP^[20]	34051	131190	32817	254290
本文	251	1200	724	946

下载: 导出CSV

参考文献(20)

[1]	ORSAGH R F, SHELDON J, KLENKE C J. Prognostics/diagnostics for gas turbine engine bearings[M]. Atlanta, US: American Society of Mechanical Engineers. 2003.
[2]	CHIACHÍO J, CHIACHÍO M, SANKARARAMAN S, et al. Condition-based prediction of time-dependent reliability in composites[J]. Reliability Engineering & System Safety, 2015, 142: 134-147.
[3]	SATEESH B G, ZHAO Peilin, Li Xiaoli. Deep convolutional neural network based regression approach for estimation of remaining useful life[C]// Database Systems for Advanced Applications. 21st International Conference. Dallas, USA: Springer International Publishing, 2016: 214-228.
[4]	LI Xiang, DING Qian, SUN Jianqiao. Remaining useful life estimation in prognostics using deep convolution neural networks[J]. Reliability Engineering & System Safety, 2018, 172: 1-11.
[5]	LIU Zhenyu, LIU Hui, JIA Weiqiang, et al. A multi-head neural network with unsymmetrical constraints for remaining useful life prediction[J]. Advanced Engineering Informatics, 2021, 50: 101396. doi: 10.1016/j.aei.2021.101396
[6]	陈保家, 郭凯敏, 陈法法, 等. 基于残差NLSTM网络和注意力机制的航空发动机剩余使用寿命预测[J]. 航空动力学报, 2023, 38(5): 1176-1184. CHEN Baojia, GUO Kaimin, CHEN Fafa, et al. Prediction of remaining useful life of aero-engine based on residual NLSTM neural network and attention mechanism[J]. Journal of Aerospace Power, 2023, 38(5): 1176-1184. (in Chinese doi: 10.13224/j.cnki.jasp.20210728 CHEN Baojia, GUO Kaimin, CHEN Fafa, et al. Prediction of remaining useful life of aero-engine based on residual NLSTM neural network and attention mechanism[J]. Journal of Aerospace Power, 2023, 38(5): 1176-1184. (in Chinese) doi: 10.13224/j.cnki.jasp.20210728
[7]	AL-DULAIMI A, ZABIHI S, ASIF A, et al. A multimodal and hybrid deep neural network model for Remaining Useful Life estimation[J]. Computers in Industry, 2019, 108: 186-196. doi: 10.1016/j.compind.2019.02.004
[8]	郭晓静, 徐晓慧, 郭佳豪. 基于改进GRU的航空发动机寿命预测自注意力优化算法[J]. 航空动力学报, 2024, 39(12): 20220984. GUO Xiaojing, XU Xiaohui, GUOJIA Hao. Improved GRU-based self-attention optimization algorithm for aero-engine remaining useful life prediction[J]. Journal of Aerospace Power, 2024, 39(12): 20220984. (in Chinese doi: 10.13224/j.cnki.jasp.20220984 GUO Xiaojing, XU Xiaohui, GUOJIA Hao. Improved GRU-based self-attention optimization algorithm for aero-engine remaining useful life prediction[J]. Journal of Aerospace Power, 2024, 39(12): 20220984. (in Chinese) doi: 10.13224/j.cnki.jasp.20220984
[9]	SONG J W, PARK Y I, HONG J J, et al. Attention-based bidirectional LSTM-CNN model for remaining useful life estimation[C]//2021 IEEE International Symposium on Circuits and Systems. Daegu, Korea: IEEE, 2021: 1-5.
[10]	ZHANG Jiusi, JIANG Yuchen, WU Shimeng, et al. Prediction of remaining useful life based on bidirectional gated recurrent unit with temporal self-attention mechanism[J]. Reliability Engineering & System Safety, 2022, 221: 108297.
[11]	LI Han, ZHAO Wei, ZHANG Yuxi, et al. Remaining useful life prediction using multi-scale deep convolutional neural network[J]. Applied Soft Computing, 2020, 89: 106113. doi: 10.1016/j.asoc.2020.106113
[12]	MO Yu, WU Qianhui, LI Xiu, et al. Remaining useful life estimation via transformer encoder enhanced by a gated convolutional unit[J]. Journal of Intelligent Manufacturing, 2021, 32(7): 1997-2006. doi: 10.1007/s10845-021-01750-x
[13]	SAXENA A, GOEBEL K, SIMON D, et al. Damage propagation modeling for aircraft engine Run-to-failure simulation[C]//2008 International Conference on Prognostics and Health Management. Denver, US: IEEE, 2008: 1-9.
[14]	FREDERICK D K, DECASTRO J A, LITT J S. User's guide for the commercial modular aero-propulsion system simulation (C-MAPSS): E-16205 [R]. Cleveland, USA: Glenn Research Center, 2007.
[15]	XU Dan, XIAO Xiaoqi, LIU Jie, et al. Spatio-temporal degradation modeling and remaining useful life prediction under multiple operating conditions based on attention mechanism and deep learning[J]. Reliability Engineering & System Safety, 2023, 229: 108886.
[16]	ADNAN R M, KHOSRAVINIA P, KARIMI B, et al. Prediction of hydraulics performance in drain envelopes using Kmeans based multivariate adaptive regression spline[J]. Applied Soft Computing, 2021, 100: 107008. doi: 10.1016/j.asoc.2020.107008
[17]	车畅畅, 王华伟, 倪晓梅, 等. 基于1D-CNN和Bi-LSTM的航空发动机剩余寿命预测[J]. 机械工程学报, 2021, 57(14): 304-312. CHE Changchang, WANG Huawei, NI Xiaomei, et al. Residual life prediction of aeroengine based on 1D-CNN and Bi-LSTM[J]. Journal of Mechanical Engineering, 2021, 57(14): 304-312. (in Chinese doi: 10.3901/JME.2021.14.304 CHE Changchang, WANG Huawei, NI Xiaomei, et al. Residual life prediction of aeroengine based on 1D-CNN and Bi-LSTM[J]. Journal of Mechanical Engineering, 2021, 57(14): 304-312. (in Chinese) doi: 10.3901/JME.2021.14.304
[18]	AL-DULAIMI A, ZABIHI S, ASIF A, et al. NBLSTM: noisy and hybrid convolutional neural network and BLSTM-based deep architecture for remaining useful life estimation[J]. Journal of Computing and Information Science in Engineering, 2020, 20(2): 021012. doi: 10.1115/1.4045491
[19]	VASWANI A, SHAZIER N, PARMAR N, et al. Attention is all you need[J]. Advances in Neural Information Processing Systems, 2017, 30: 6000-6010.
[20]	ZHOU Liang, WANG Huawei, XU Shanshan. Aero-engine prognosis strategy based on multi-scale feature fusion and multi-task parallel learning[J]. Reliability Engineering & System Safety, 2023, 234: 109182.