- 5 minutes to read

How to automate monitoring of correlated events

This guide will teach you how to monitor correlated events spanning one or more text-based log files using the Nodinite Log File Parser Monitoring Agent.

Let's look at a quick example to help you understand a use case. Below is a sample text-based log file where the record for ORDER with Id 456 does not provide the expected record with the matching (correlated) ORDERRESPONSE 456. Late, and or a missing Order Response is probably inadequate for your business, and you might want to get an alert that this situation exists. In your real-world scenario, you might have other entities that you would like to monitor, and since the Nodinite Log File Parser Monitoring Agent uses RegEx expressions, most uses cases should be possible to cover.

2019-04-10 13:37:00.000 ORDERS #123
2019-04-10 13:39:00.123 ORDERS #456
2019-04-10 14:12:34.456 ORDERRESPONSE #123

Example where the order response for order 456 is missing

Before you begin

Ensure you comply with the prerequisites and install the Nodinite Log File Parser Monitoring Agent.

Step 1: Add a new monitoring configuration

To add a new monitoring entry, click the Add button from the Log File Events tab.
tab

  1. Name the configuration
  2. Provide an optional description
  3. Set the Application ID (You must have a matching entry in the Applications Tab)

General Tab

Step 2: Set the monitor path

Next, enter the path to where the log files with events to correlate, are located (for example, C:\Temp\Log File Parser).

  1. Enter the path
  2. Enter the RegEx-based filter (\.txt$)

Path

Below is a table with some common RegEx file filter examples:

Filter Example Comment
\.xml$ XML Files All XML files with suffix ".xml"
\.txt$ Text Files All text files with suffix ".txt"
^ONLYME\.data$ Specific file Only this file "ONLYME.data"
^PrefixedFileName.*\.csv$ Matching a file name pattern Files with prefix ^PrefixedFileName, and suffix .csv

Step 3: Set the start match

Next, enter the start match configuration.

  1. The line contains content X (The line that starts the correlation must have this content) - For example ORDERS
  2. Match date (the date format) - ([0-9]{4}-[0-9]{2}-[0-9]{2}[T\\s]?[0-9]{2}:[0-9]{2}:[0-9]{2}\\.[0-9]{3}(\\+[0-9]{2}:[0-9]{2})?)
  3. In this example, leave the 'Date Time Format (Optional)' field empty
  4. Matched date groups - 1
  5. Match value is what content to use for the start of correlation. In this example, we are looking for a number preceded by a # sign - #([0-9]{1,})
  6. Matched value groups with this RegEx expression is 1
  7. Optionally, you can enter a RegEx to use if some other content matches this line. In this example, we leave the field empty.

Start match

Step 4: Set the end match

Next, enter the end match configuration:

  1. Line contains (The end of the correlation is...) - For example ORDERRESPONSE
  2. Match date (the date) - ([0-9]{4}-[0-9]{2}-[0-9]{2}[T\\s]?[0-9]{2}:[0-9]{2}:[0-9]{2}\\.[0-9]{3}(\\+[0-9]{2}:[0-9]{2})?)
  3. Matched date groups - 1
  4. Match value is what content to use for the start of correlation. In this example, we are looking for a number preceded by a # sign - #([0-9]{1,})
  5. Matched value groups with this RegEx expression is 1
  6. Optionally, you can enter a RegEx to use if some other content matches this line. In this example, we leave this field empty.
    End match
    The time-span format is days.hours:minutes:seconds, for example, 0.00:10:00 for ten minutes.
  7. Warning Time-Span - Enter the time-span allowed before a Warning alert is issued. For example, 0.00:10:00
  8. Error Time-Span - Enter the time-span allowed before an Error alert is issued. For example, 0.00:30:00

Next, select what files to include in the Monitoring.

  1. Leave the Clear Date Time field empty for now. This field is populated by the system when an end-user is clearing previous problems.
  2. Select the time option (Modified after clear date-time)
    Timed options
    Here is an example of the 'Time options' tab.

Save

You must click 'Save' or 'Save and close' for changes to persist. The new settings/thresholds are evaluated on the next synchronization.
Save and Close buttons
Here's an example of the Save options.

Note

The delay in presenting the new evaluated state depends on the monitoring agent's synchronization interval

Save and close, save, and close the dialogue. Cancel, closes the dialogue without saving any changes.

Step 6: Configure the Monitor View

Next, add the named Resource to a Monitor View.

Errors detected
Here's an example of a Monitor View with errors detected in correlated log files

You can further review the details about the errors by clicking on the Actions button, and then clicking on the Error Report menu item.
Show Error Report Action

Clicking on this action opens a new modal with a list of all log files where an error is still active (> last clear date-time).
Error Report

Additional information is available if you expand the row.
Details
Do note that the correlation may span multiple files(!)

Tip

You can ignore old errors; by clicking on the Clear Errors menu item in the Actions button. The Resource is then in the OK state until a new entry matches the configuration.

Next Step

Nodinite File Monitoring Agent