ARCHIVES

Research Article

Efficient Log Parsing and Visualization System for Large- Scale Logs: A Data-Centric Approach Using Custom Data Structures

V M Suhas1V M Subash2V S Arjun3

¹² Final Year B.Tech Students, Department of Computer Science and Engineering, Reva University, Bangalore, Karnataka, India. ³Technical Program Manager – Wireless, Tessolve Semiconductor Private Limited, Bangalore, Karnataka, India.

Published Online: September-October 2024

Pages: 22-26

Abstract

View PDF

Abstract: In the realm of debugging and system optimization, testers frequently deal with large-scale logs from systems such as 5G communication networks. Handling, filtering, and visualizing these logs efficiently are crucial for understanding system performance and identifying issues. This paper presents an efficient log parsing and visualization system that leverages custom data structures to handle logs more effectively, with significant improvements in filtering and querying after initial parsing. The system provides multiple ways to filter logs, including by parent attributes, supports inline scripting to manipulate data tables, and offers various visualization techniques. The tool is optimized for fast filtering and querying, compared to other approaches, and will be available as an open-source project to the broader testing and debugging community. Key Word : PySide6, Matplotlib, Mpl Cursors, Log Parsing, QT Threads, Log Visualization.

Related Articles

2024

Embedding Artificial Intelligence for Personal Voice Assistant Using NLP

2024

Analysis of Pedestrian Steel Bridge subjected the Seismic Load and Wind Load using Damper at different Span

2024

Review Paper on Comparison of Asymmetric and Symmetric RCC Building with Soil Structure Interaction by Dynamic Loading

2024

BLYNK RFID and Retinal Lock Access System

2024

ML-Driven Facial Synthesis from Spoken Words Using Conditional GANs

2024

Research on smart baby cradle using sensor technology

2024

Multimodal Biometric Login System Using Face and Signature

2024

Optimization of Geometrical Characteristics of Building by Using Diagrid System against Lateral Forces

2024

PDF-Chat SaaS Platform Using MERN Stack

2024

Social Media App for Connecting Similar Interests People Using flutter

Efficient Log Parsing and Visualization System for Large- Scale Logs: A Data-Centric Approach Using Custom Data Structures | IJIRE