Prediction of remaining useful life of aero-engine based on residual NLSTM neural network and attention mechanism
-
摘要:
针对长短期记忆（LSTM）网络对于多维数据特征识别和提取上存在不足的问题，在其改进模型嵌套式长短期记忆（NLSTM）网络的基础上，提出了一种基于注意力机制和残差NLSTM网络的剩余使用寿命预测方法。该方法将双层NLSTM网络代替残差块中的主网络，保留捷径连接中的卷积神经网络结构，既能充分提取时序特征又能保证有用数据在网络层中的跳层传递，并融入注意力机制构建多层残差网络，注意力机制的使用能够选择出对预测结果有重要影响的信息，有效提高预测的准确率。在航空发动机退化实验数据集上进行实验分析，结果表明：所述方法能有效建立监测数据与发动机健康状态之间的关系，剩余使用寿命预测误差较未改进残差结构方法平均降低10.8%，比未融入注意力机制方法平均降低18.9%，有效提高了预测精度。
-
关键词:
- 残差网络 /
- 剩余使用寿命 /
- 注意力机制 /
- 预测模型 /
- 嵌套式长短期记忆神经网络
Abstract:The remaining useful life prediction method based on attention mechanism and residual nested long-short-term memory （NLSTM） neural network was employed to address the shortcomings of traditional long-short-term memory （LSTM） neural network in the recognition and extraction of multi-dimensional data features. Two NLSTM neural network layers were used to replace the main structure of the residual block, and the shortcut connection of the one-dimensional convolutional network in the residual block was retained, which can fully extract the temporal feature and use the jump layer to transfer useful data in the network layer. The method also added the attention mechanism to construct multilayer network, and the important information influencing the result of prediction can be chosen to improve the prediction accuracy by the attention mechanism. The proposed method was verified by experiments in the aero-engines degradation dataset. The results showed that the method can effectively establish the relationship between the monitoring data and the engines health. The prediction error was reduced by 10.8% compared with the method without residual structure and reduced 18.9% compared with the method without attention mechanism, which improved the prediction accuracy effectively.
-
图 6 ResNLSTM-attention RUL预测流程图
Figure 6. Flow chart of ResNLSTM-attention RUL prediction
表 1 C-MAPSS数据集描述
Table 1. Description of C-MAPSS dataset
数据集 训练集
（发动机数量）测试集
（发动机数量）操作
条件故障
模式FD001 100 100 1 1 FD002 260 259 6 1 FD003 100 100 1 2 FD004 248 249 6 2 
下载: 导出CSV
表 2 ResNLSTM-attention网络参数
Table 2. ResNLSTM-attention network parameters
网络层 层参数 输出尺寸 输入层 时间步：36 （36，14） NLSTM层 单元数：14 （36，14） Conv1D层 单元数：14，核尺寸：1 （36，14） NLSTM层 单元数：7 （36，7） Conv1D层 单元数：7，核尺寸：1 （36，7） Attention层 单元数：32 （36，7） Attention输出层 单元数：32 （None，32） Dense层 单元数：1 （None，1）
下载: 导出CSV
表 3 方均根误差结果
Table 3. Result of root mean square error
子数据集 σrms 平均值 第1次 第2次 第3次 FD001 12.52 12.50 12.56 12.53 FD002 19.37 20.34 21.82 20.51 FD003 11.94 12.03 12.48 12.15 FD004 22.29 20.99 23.80 22.36
下载: 导出CSV
表 4 评分函数结果
Table 4. Result of score function
子数据集 Sf 平均值 第1次 第2次 第3次 FD001 279 259 257 265 FD002 761 496 2328 1195 FD003 249 258 345 284 FD004 3444 1724 2907 2692
下载: 导出CSV
表 5 C-MAPSS数据集消融实验结果对比
Table 5. Comparison of the results of ablation experiments in the C-MAPSS dataset
方法 FD001 FD002 FD003 FD004 NLSTM-attention 13.49 24.61 12.58 24.18 ResNLSTM 15.32 23.50 16.68 24.80 本文方法预测平均值 12.53 20.51 12.15 22.36 本文方法预测最优值 12.50 20.34 11.94 20.99
下载: 导出CSV
表 6 C-MPASS数据集下不同方法的方均根误差结果对比
Table 6. Comparison of root mean square error results of different methods in the C-MPASS dataset
方法 σrms FD001 FD002 FD003 FD004 CNN[3], 2016 18.45 30.29 19.82 29.16 LSTM-FNN[4], 2017 16.14 24.49 16.18 28.17 RBM-LSTM-FNN[6], 2019 12.56 22.73 12.10 22.66 Auto-Encoder[7], 2019 14.74 22.07 17.48 23.49 CMRSA[9], 2021 15.00 23.22 17.23 23.74 CNN-FNN[19], 2018 12.61 22.36 12.64 22.43 Auto-Encoder[20], 2020 13.58 19.59 19.16 22.15 DCNN-FNN[21], 2020 12.61 28.51 12.62 30.73 本文方法预测平均值 12.53 20.51 12.15 22.36 本文方法预测最优值 12.50 20.34 11.94 20.99
下载: 导出CSV
表 7 C-MPASS数据集下不同方法的评分函数结果对比
Table 7. Comparison of score function results of different methods in the C-MPASS data set
方法 Sf FD001 FD002 FD003 FD004 CNN[3], 2016 1290 13570 1596 7886 LSTM-FNN[4], 2017 338 4450 852 5550 RBM-LSTM-FNN[6], 2019 231 3370 251 2840 Auto-Encoder[7], 2019 273 3099 574 3202 CMRSA[9], 2021 381 21766 994 13564 DBN[18], 2017 640 10851 683 7210 CNN-FNN[19], 2018 274 10412 284 12466 Auto-Encoder[20], 2020 220 2650 1727 2901 本文方法预测平均值 265 1195 284 2692 本文方法预测最优值 257 496 249 1724
下载: 导出CSV
-
[1] 彭宇,刘大同. 数据驱动故障预测和健康管理综述[J]. 仪器仪表学报,2014,35(3): 481-495. doi: 10.19650/j.cnki.cjsi.2014.03.001PENG Yu,LIU Datong. Data-driven prognostics and health management: a review of recent advances[J]. Chinese Journal of Scientific Instrument,2014,35(3): 481-495. (in Chinese) doi: 10.19650/j.cnki.cjsi.2014.03.001 [2] LEI Yaguo,LI Naipeng,GUO Liang,et al. Machinery health prognostics: a systematic review from data acquisition to RUL prediction[J]. Mechanical Systems and Signal Processing,2018,104(5): 799-834. [3] BABU G S, ZHAO Peilin, LI Xiaoli. Deep convolutional neural network based regression approach for estimation of remaining useful life[C]// International Conference on Database Systems for Advanced Applications. Cham, Switzerland: Springer, 2016: 214-228. [4] ZHENG Shuai, RISTOVSKI K, FARAHAT A, et al. Long short-term memory network for remaining useful life estimation[C]// IEEE International Conference on Prognostics and Health Management (ICPHM). Dallas, US: IEEE, 2017: 88-95. [5] WANG Jiujian, WEN Guilin, YANG Shaopu, et al. Remaining useful life estimation in prognostics using deep bidirectional lstm neural network[C]// Prognostics and System Health Management Conference (PHM-Chongqing). Chongqing, China: IEEE, 2018: 1037-1042. [6] ELLEFSEN A L,BJORLYKHAUG E,AESOY V,et al. Remaining useful life predictions for turbofan engine degradation using semi-supervised deep architecture[J]. Reliability Engineering and System Safety,2019,183(3): 240-251. [7] YU W,KIM I I Y,MECHEFSKE C. Remaining useful life estimation using a bidirectional recurrent neural network based autoencoder scheme[J]. Mechanical Systems and Signal Processing,2019,129(8): 764-780. [8] 朱霖,宁芊,雷印杰,等. 基于遗传算法选优的集成手段与时序卷积网络的涡扇发动机剩余寿命预测[J]. 计算机应用,2020,40(12): 3534-3540.ZHU Lin,NING Qian,LEI Yinjie,et al. Remaining useful life prediction for turbofan engines by genetic algorithm-based selective ensembling and temporal convolutional network[J]. Journal of Computer Applications,2020,40(12): 3534-3540. (in Chinese) [9] 牟含笑, 郑建飞, 胡昌华, 等. 基于CDBN与BiLSTM 的多元退化设备剩余寿命预测[J/OL]. [2022-06-27]. http://kns.cnki.net/kcms/detail/11.1929.v.20210510.1354.004.html. [10] 叶瑞达,王卫杰,何亮,等. 基于残差自注意力机制的航空发动机RUL预测[J]. 光学精密工程,2021,29(6): 1482-1490. doi: 10.37188/OPE.20212906.1482YE Ruida,WANG Weijie,HE Liang,et al. RUL prediction of aero-engine based on residual self-attention mechanism[J]. Optics and Precision Engineering,2021,29(6): 1482-1490. (in Chinese) doi: 10.37188/OPE.20212906.1482 [11] CAI Yongping,CEN Yuefeng,CEN Gang,et al. Temperature prediction of PMSMs using pseudo-siamese Nested LSTM[J]. World Electric Vehicle Journal,2021,12(2): 57-71. doi: 10.3390/wevj12020057 [12] MONIZ J R A, KRUEGER D. Nested LSTMS[C]// Asian Conference on Machine Learning. Seoul, Korea: The Journal of Machine Learning Research, 2017: 530-544. [13] HE K, ZHANG X, REN S, et al. Identity mappings in deep residual networks[C]// European Conference on Computer Vision. Cham, Switzerland: Springer, 2016: 630-645. [14] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// Proceedings of the 31st Conference on Neural Information Processing Systems. Long Beach, US: Curran Associates Inc, 2017: 5998-6008. [15] SAXENA A, GOEBEL K, SIMON D, et al. Damage propagation modeling for aircraft engine run-to-failure simulation[C]// 2008 international Conference on Prognostics and Health Management. Denver, US: IEEE, 2008: 1-9. [16] WU Yuting,YUAN Mei,DONG Shaopeng,et al. Remaining useful life estimation of engineered systems using vanilla LSTM neural networks[J]. Neurocomputing,2018,275(1): 167-179. [17] HEIMES F O. Recurrent neural networks for remaining useful life estimation[C]// 2008 International Conference on Prognostics and Health Management. Denver, US: IEEE, 2008: 6-12. [18] CHONG Z,LIM P,QIN A K,et al. Multiobjective deep belief networks ensemble for remaining useful life estimation in prognostics[J]. Transactions on Neural Networks and Learning Systems,2017,28(10): 2306-2318. doi: 10.1109/TNNLS.2016.2582798 [19] LI Xiang,DING Qian,SUN Jianqiao. Remaining useful life estimation in prognostics using deep convolution neural networks[J]. Reliability Engineering and System Safety,2018,172(4): 1-11. [20] YU W,KIM I Y,MECHEFSKE C. An improved similarity-based prognostic algorithm for RUL estimation using an RNN autoencoder scheme[J]. Reliability Engineering and System Safety,2020,199(7): 1-12. [21] XU Xin,WU Qianhui,LI Xiu,et al. Dilated convolution neural network for remaining useful life prediction[J]. Journal of Computing and Information Science in Engineering,2019,20(2): 1-14. -
点击查看大图
计量
- 文章访问数: 154
- HTML浏览量: 93
- PDF量: 54
- 被引次数: 0