LLM-STDAN: SPATIO-TEMPORAL DATA AUGMENTATION NETWORK OF A LARGE LANGUAGE MODEL FOR UNSUPERVISED RAILWAY SAFETY RISK DIVISION

Yifan Wang, Lingbin Bu, Liang Zhang, Qiming Ma, Zhiyuan Li, Qi Wang, Fanliang Bu

References

  1. [1] V.B. Getanda, H. Oya, T. Kubo, and Y. Sato, Data groupingtechniques’ performance analysis in GM (1, 1)’s prediction accu-racy improvement for forecasting traffic parameters, MechatronicSystems and Control, 48(1), 2020, 25–34.
  2. [2] W. Zhou, X. Chen, and B. Cui, A case study of environmentalmonitoring data analysis and forecasting model, MechatronicSystems and Control, 46(3), 2018, 127–131.
  3. [3] J. Liu, G. Liu, Y. Wang, and W. Zhang, Artificial-intelligent-powered safety and efficiency improvement for controlling andscheduling in integrated railway systems, High-speed Railway, 2,2024, 172–179.
  4. [4] H Hadj-Mabrouk, Analysis and prediction of railway accidentrisks using machine learning, AIMS Electronics and ElectricalEngineering, 4(1), 2020, 19–46.
  5. [5] J. Liu, K. Chen, H. Duan, and C. Li, A knowledge graph-basedhazard prediction approach for preventing railway operationalaccidents, Reliability Engineering & System Safety, 247, 2024,110126.
  6. [6] D. Zhang, Y. Peng, Y. Xu, C. Du, Y. Zhang, N. Wang,Y. Chong, H. Wang, D. Wu, J. Liu, H. Zhang, L. Lu,and J. Liu, A high-speed railway network dataset from trainoperation records and weather data, Scientific Data, 9(1),2022, 244.14
  7. [7] A. Kasraei, A.H.S. Garmabaki, J. Odelius, S.M. Famurewa,K.S. Chamkhorami, and G. Strandberg, Climate changeimpacts assessment on railway infrastructure in urbanenvironments, Sustainable Cities and Society, 101, 2024,105084.
  8. [8] T. Marteau, S. Afanon, D. Sodoyer, and S. Ambellouis, Violencedetection in railway environment with modern deep learningapproaches and small dataset, Transportation Research Procedia,72, 2023, 87–92.
  9. [9] I.K. Nti, A.F. Adekoya, and B.A. Weyori, A novel multi-sourceinformation-fusion predictive framework based on deep neuralnetworks for accuracy enhancement in stock market prediction,Journal of Big Data, 8(1), 2021, 1–28.
  10. [10] J. Xie, R. Girshick, and A. Farhadi, Unsuperviseddeep embedding for clustering analysis, in Proceedingsof International Conference on Machine Learning, 2016,478–487.
  11. [11] J. Buitrago, A. Abdulaal, and S. Asfour, Electric load patternclassification using parameter estimation, clustering and artificialneural networks, International Journal of Power and EnergySystems, 35(4), 2015, 167–174.
  12. [12] R.K. Esfahani, F. Shahbazi, and M. Akbarzadeh, Three-phaseclassification of an uninterrupted traffic flow: A k-means clusteringstudy, Transportmetrica B: Transport Dynamics, 7(1), 2018,546–558.
  13. [13] D. Baranovskyi, L. Muradian, and M. Bulakh, The methodof assessing traffic safety in railway transport, IOP SciNotes,666(4), 2021, 42075.
  14. [14] Y. Jinbao, M. Min, F. Yu, and G. Yanhong, LSTM-attention textclassification method combined with key information, MechatronicSystems and Control, 50(10), 2022, 1–7.
  15. [15] S. Kobayashi, Contextual augmentation: Data augmentationby words with paradigmatic relations, in Proceedings of NAACLHLT, 2018, 452–457.
  16. [16] D. Park and C.W. Ahn, Self-supervised contextual dataaugmentation for natural language processing, Symmetry, 11(11),2019, 1393.
  17. [17] C.T. Lin, Y.K. Wang, P.L. Huang, Y. Shi, and Y.C.Chang, Spatial-temporal attention-based convolutional networkwith text and numerical information for stock price pre-diction, Neural Computing and Applications, 34(17), 2022,14387–14395.
  18. [18] V. Sanh, L. Debut, J. Chaumond, and T. Wolf, DistilBERT,a distilled version of BERT: smaller, faster, cheaper and lighter,2019, arXiv:1910.01108.
  19. [19] A.I. K´aroly, R. Full´er, and P Galambos, Unsupervised clusteringfor deep learning: A tutorial survey, Acta Polytechnica Hungarica,15(8), 2018, 29–53.
  20. [20] M. van de Velden, A. I. D’Enza, and A. Markos, Distance-based clustering of mixed data, Wiley Interdisciplinary Reviews:Computational Statistics, 11(3), 2019, e1456.
  21. [21] P.H. Le-Khac, G. Healy, and A.F. Smeaton, Contrastiverepresentation learning: A framework and review, IEEE Access,8, 2020, 193907–193934.
  22. [22] A. van den Oord, S. Dieleman, H. Zen, K. Simonyan,O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K.Kavukcuoglu, Wavenet: A generative model for raw audio, 2016,arXiv:1609.03499.
  23. [23] Federal Railroad Administration Office of Safety Analysis,Accident data as reported by railroads, (2024-01-31) 2024-03-15.https://safetydata.fra.dot.gov.
  24. [24] N. Reimers and I. Gurevych, Sentence-BERT: Sentenceembeddings using Siamese BERT-networks, in Proceedingsof the 2019 Conference on Empirical Methods in NaturalLanguage Processing and the 9th International Joint Conferenceon Natural Language Processing (EMNLP-IJCNLP), 2019,3982–3992.
  25. [25] J. Xu, B. Xu, P. Wang, S. Zheng, G. Tian, and J. Zhao, Self-taught convolutional neural networks for short text clustering,Neural Networks, 88, 2017, 22–31.
  26. [26] F. Galatioto, M. Catalano, N. Shaikh, E. McCormick, andR. Johnston, Advanced accident prediction models and impactsassessment, IET Intelligent Transport Systems, 12(9), 2018, 1131–1141.
  27. [27] D. Kehagias, A. Salamanis, and D. Tzovaras, Speed patternrecognition technique for short-term traffic forecasting based ontraffic dynamics, IET Intelligent Transport Systems, 9(6), 2015,646–653.

Important Links:

Go Back