TY - GEN

T1 - Revolutionizing Log Parsing for Modern Software Systems

AU - Petrescu, Stefan

N1 - Funding Information: This research project, initiated at TU Delft, is currently underway at Leibniz University of Hannover under the supervision of Prof. Dr. Jan Rellermeyer. Furthermore, the industry experiments were carried out with the valuable support of Floris den Hengst at the AI for Fintech Research lab, ING Netherlands.

PY - 2023

Y1 - 2023

N2 - With the tremendous log volume generated by modern systems, automated log analysis becomes indispensable for discovering crucial insights into the behavior of running systems. The foremost step subsumed in an automated log analysis pipeline, called log parsing, significantly influences its entire performance. Despite its significance, log parsing lacks quality implementations and, in practice, suffers from fundamental limitations, thereby creating a bottleneck for discovering valuable insights at the log line level. As a consequence, in my PhD thesis, I endeavor to explore a novel paradigm, called entity parsing, which goes beyond previous work by not only addressing the current limitations of log parsing but by exploring a new avenue for log parsing that could lead to significant advances in how we perform log analysis. As preliminary results show, entity parsing, despite solving a more difficult problem than conventional log parsing, is viable and obtains a significantly better accuracy on comparable datasets. By applying a first-principles approach, entity parsing is based on three fundamental components: data, machine learning, and source code. These components, leveraged within the framework of entity parsing, process logs in a way that could significantly improve the overall dependability of systems, enhancing their availability, security, and reliability.

AB - With the tremendous log volume generated by modern systems, automated log analysis becomes indispensable for discovering crucial insights into the behavior of running systems. The foremost step subsumed in an automated log analysis pipeline, called log parsing, significantly influences its entire performance. Despite its significance, log parsing lacks quality implementations and, in practice, suffers from fundamental limitations, thereby creating a bottleneck for discovering valuable insights at the log line level. As a consequence, in my PhD thesis, I endeavor to explore a novel paradigm, called entity parsing, which goes beyond previous work by not only addressing the current limitations of log parsing but by exploring a new avenue for log parsing that could lead to significant advances in how we perform log analysis. As preliminary results show, entity parsing, despite solving a more difficult problem than conventional log parsing, is viable and obtains a significantly better accuracy on comparable datasets. By applying a first-principles approach, entity parsing is based on three fundamental components: data, machine learning, and source code. These components, leveraged within the framework of entity parsing, process logs in a way that could significantly improve the overall dependability of systems, enhancing their availability, security, and reliability.

KW - log analysis

KW - log parsing

KW - reliability

UR - http://www.scopus.com/inward/record.url?scp=85178504104&partnerID=8YFLogxK

U2 - 10.1109/ISSREW60843.2023.00036

DO - 10.1109/ISSREW60843.2023.00036

M3 - Conference contribution

AN - SCOPUS:85178504104

T3 - Proceedings - 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshop, ISSREW 2023

SP - 17

EP - 21

BT - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 34th IEEE International Symposium on Software Reliability Engineering Workshop, ISSREW 2023

Y2 - 9 October 2023 through 12 October 2023

ER -