RegEx with Capturing Groups Search Field Expression Plugin

Easily extract unique values from messages using the RegEx with Capturing Groups Search Field Expression Plugin. This page shows you how to configure, test, and use this plugin with clear input, configuration, and result examples.

✅ Extract single or multiple unique values from messages (payload) ✅ Use powerful RegEx expressions with capturing groups for advanced extraction ✅ Support for global and multiline matching options ✅ Designed for high performance and self-service log views

🎯 Design Note: Perfect for extracting specific parts of matched patterns using parentheses groups!

What does the RegEx with Capturing Groups plugin do?

The RegEx with Capturing Groups plugin lets you extract one or more unique values from the payload (Body) of Log Events. Unlike the basic RegEx plugin that returns the entire match, this plugin extracts only the content within capturing groups (parentheses).

Key Features:

• Extract specific parts of RegEx matches using capturing groups ()
• Support for global matching (find all occurrences)
• Support for multiline mode
• Specify which capturing group to extract (1, 2, 3, etc.)
• Ideal for complex extraction scenarios

Use Case: Extract specific parts of matched patterns, like extracting just the ID from CityId="054" → 054, or the 4th field from delimited data.

Tip: The basic RegEx plugin returns the entire match. This plugin extracts only what's inside the parentheses.

How it works

The RegEx with Capturing Groups plugin processes messages in four steps:

1. Input: The message payload
2. Expression: A RegEx pattern with capturing groups ()
3. Group Selection: Which capturing group to extract (1, 2, 3, etc.)
4. Result: Unique value(s) from the specified capturing group

The plugin matches the full pattern but extracts only the content within the specified capturing group.

Example

Input:

<ns0:Orders xmlns:ns0="Common.Schemas/Nodinite/1.0">
    <Order>
        <Id>101</Id>
        <Amount>1337</Amount>
        <City CityId="054">Karlstad</City>
    </Order>
    <Order>
        <Id>102</Id>
        <Amount>10</Amount>
        <City CityId="08">Stockholm</City>
    </Order>
</ns0:Orders>

RegEx Expression:

CityId="([0-9]+)"

Configuration:

• RegEx groups = 1
• Global = Checked
• Multi line = Unchecked

Result:

054
08

Tip: The pattern matches CityId="054" but extracts only 054 (the content inside the parentheses). See RegEx for extracting the full match.

Examples

Example 1: Extract City IDs from XML Using RegEx

Input:

<ns0:Orders xmlns:ns0="Common.Schemas/Nodinite/1.1">
    <Order>
        <Id>101</Id>
        <Amount>1000</Amount>
        <City CityId="054">Karlstad</City>
    </Order>
    <Order>
        <Id>102</Id>
        <Amount>10</Amount>
        <City CityId="08">Stockholm</City>
    </Order>
</ns0:Orders>

RegEx Expression:

CityId="([0-9]+)"

• RegEx groups = 1
• Global = Checked
• Multi line = Unchecked

Result:

054
08

Example 2: Extract 4th Field from Pipe-Delimited Flat File

Input:

ORD|101|ExampleCompany1|456|Company Name ACME |Dieselgate valley 1|123 45|Flameburg|
ORD|102|ExampleCompany2|789|Company Name MECA |Radiator Spring 420|543 21|Apocalypsenburg|

RegEx Expression:

^(?:(.*?)\|){4}

• RegEx groups = 1
• Global = Checked
• Multi line = Checked

Result:

456
789

Example: Extract multiple customer codes from a flat file.

Example 3: Extract Value from Flat File with Tag at Start of Line

Input:

ST|123
KV|Z|AN|1337|ABC123
ID|1912121212|

RegEx Expression:

^KV(?:(.*?)\|){4}

• RegEx groups = 1
• Global = Checked or Unchecked
• Multi line = Checked

Result:

1337

Example: Extract value from flat file with tag KV.

Example 4: Extract 3rd Field from Semicolon-Delimited Flat File with Tag

Input:

HDR;ORDER;42
KV;Z;AN;1337;ABC123
ID;1912121212;

RegEx Expression:

^HDR(?:(.*?)[;|\n|\r]){3}

• RegEx groups = 1
• Global = Checked or Unchecked
• Multi line = Checked

Result:

42

Example: Extract value from flat file with tag HDR.

Features

• Extract single or multiple unique values from messages (payload)
• Use RegEx expressions with capturing groups, global, and multiline options
• Designed for small payloads and high performance

Note: This plugin loads the entire message into RAM. Only use it for small payloads.

How to Use

1. Select the 'RegEx with capturing groups' expression type plugin when configuring a Search Field.
2. Enter the RegEx expression to extract the desired value(s).
   • RegEx groups: Enter the number or name of RegEx group(s) to return (leave empty for all matches)
   • Global: When checked, do not return on first match
   • Multi line: When checked, use ^ for start of line and $ for end of line
3. Finalize the configuration and select the Message Types to apply the expression.
4. Optionally, perform a re-index operation (Add or manage Search Field).

Extracted values are stored for as long as the Message Type retention is configured.

Test Expression

You can test your configuration in the 'Test Expression' tab when setting up a Search Field:

1. Enter a sample payload in the 'Message Body' tab
2. Select the 'RegEx with capturing groups' plugin
   
   Here's an example selecting the 'RegEx with capturing groups' expression type plugin.
3. Enter your RegEx expression (click on elements/attributes to get suggestions)
4. Review the output and adjust as needed

If the expression is valid, you will see the unique values and total count:

Valid expression with state output, unique values, and total count.

If the expression is invalid or does not match any data, you will see:

Example of an invalid expression yielding no result.

Related Plugins and Topics

• RegEx
• RegEx on Message Context
• RegEx on Message Context with capturing groups
• XPath with RegEx
• Formula – For advanced options using nested expressions

Mermaid Diagram: Search Field Expression Flow

This diagram shows how the RegEx with capturing groups plugin fits into the overall search field expression process.

Next Step

• Add or manage Search Fields
• Add or manage Log Views

Learn More

• What are Search Fields?
• What are Search Field Expressions?
• What are Message Types?
• What are Log Views?