Skip to main content

Log Mechanism

Overview​

In order to make the best use of the AIOps log analysis capabilities, let us start by understanding the log mechanism in AIOps.

The log mechanism in AIOps includes the following major steps:

  1. Log Ingestion: The raw logs are first ingested in AIOps. There are multiple ways to configure a logging source.. The first step in log monitoring is log ingestion, which involves collecting the raw logs from various sources and feeding them into the AIOps system. There are several ways to configure a logging source, including syslog, and agents. Log ingestion can be done in real-time or through batch processing. Real-time ingestion involves processing the logs as they are generated, while batch processing involves processing the logs at specified intervals.

  2. Log Parsing and Filtering: Once the logs are ingested into the AIOps system, the logs are parsed and filtered based on the parser assigned to the source device. The parser identifies the log format and extracts the relevant information from the log messages. The extracted information is then stored in a structured format for further analysis. There are several inbuilt log parsers available in AIOps, but users can also create their own parsers to parse logs from specific devices or applications that may not be supported by the inbuilt parsers.

  3. Log Dump in Database: After the logs are parsed, the parsed logs are stored in a database. The database is usually structured in a way that is optimized for log storage and retrieval. The logs are categorized based on the parser assigned to the log source, making it easy to locate and analyze specific logs.

  4. Log Exploration and Analysis: The final step in the log mechanism is Log exploration and analysis. Once the logs are in the database, they can be explored and analyzed using Log Explorer. Log exploration and analysis involves searching and filtering logs, creating dashboards and reports, and identifying patterns and trends in the log data. Motadata AIOps uses machine learning and AI algorithms to analyze log data and identify anomalies or potential issues. This can help organizations proactively address issues before they become major problems.

We will now look into all the steps in detail in the next sections.