- 5 minutes to read

RegEx with Capturing Groups Search Field Expression Plugin

The RegEx with capturing groups plugin runs a regular expression on the message body of a Log Event and returns only the text inside the capturing group you choose, not the whole match. Use it to pick a value out of its surroundings, such as 054 from CityId="054" or the 4th field of a delimited flat file record.

  • ✅ Returns one, several, or all capturing groups of each match
  • ✅ Global and Multi line options for record-by-record flat files
  • ✅ Works on any text format in the message body

When to use it

Four RegEx plugins cover the combinations of where the text is and what part of the match you want. For every plugin, see Choose a plugin.

Your data You want Use
Message body The whole match RegEx
Message body Only the part inside a capturing group ( ) RegEx with capturing groups (this page)
One context value (you know the key) The whole match RegEx on Message Context
One context value (you know the key) Only the part inside a capturing group RegEx on Message Context with capturing groups
One context value (you know the key) The value as it is, no pattern Message Context Key
A group you need to transform or combine Nested functions Formula with regexGroup

Expression syntax

Select RegEx with capturing groups in the Expression Type list. The plugin shows these fields:

Expression Type list filtered on regex, with RegEx with capturing groups selected
The RegEx with capturing groups plugin in the Expression Type list.

Field What goes in it
Expression The regular expression, with at least one capturing group ( ).
RegEx groups The group or groups to return. The UI accepts a single number (1), a list (1,3,4,5,6), a range (1,3-6), or group names (value1,value2); leave it empty to return all groups.
Global Checked: return every match. Unchecked: stop after the first match.
Multi line Checked: ^ and $ match the start and end of each line, not only of the whole body.
  • Groups are numbered from 1 in the order their opening parentheses appear. (?: ) is a non-capturing group and gets no number.
  • A repeated group keeps the value of its last repetition. ^(?:(.*?)\|){4} therefore returns the 4th pipe-delimited field of a line, counting from 1.
  • Matching ignores case: the pattern cityId matches CityId in Example 1.
  • The plugin reads the whole message body; for large XML or JSON payloads, prefer XPath or JSON Path.

Configure and test the expression as described in Add or manage Search Field and Test Your Expression. When nothing matches, Test Expression reports a warning:

Test Expression result warning that the expression yielded no result
Test result when the pattern matches nothing in the message body.

Examples

Example 1: Extract city IDs from an order message

Input 

<ns0:Orders xmlns:ns0="Common.Schemas/Nodinite/1.0">
    <Order>
        <Id>101</Id>
        <Amount>1337</Amount>
        <City CityId="054">Karlstad</City>
    </Order>
    <Order>
        <Id>102</Id>
        <Amount>10</Amount>
        <City CityId="08">Stockholm</City>
    </Order>
</ns0:Orders>

Expression 

cityId="([0-9]+)"
  • RegEx groups: 1
  • Global: checked
  • Multi line: unchecked

Result 

054
08

The same pattern in the RegEx plugin returns CityId="054" and CityId="08".

Test Expression for RegEx with capturing groups returning 054 and 08 from an Orders XML message
Test Expression: group 1 of each match, two unique values.

Example 2: Extract the 4th field from each pipe-delimited order record

Input 

ORD|101|ExampleCompany1|456|Company Name ACME |Dieselgate valley 1|123 45|Flameburg|
ORD|102|ExampleCompany2|789|Company Name MECA |Radiator Spring 420|543 21|Apocalypsenburg|

Expression 

^(?:(.*?)\|){4}
  • RegEx groups: 1
  • Global: checked
  • Multi line: checked

Result 

456
789

The group repeats four times per line and keeps the last value: ORD, 101, ExampleCompany1, then 456. Multi line makes ^ match at the start of each record, and Global returns both records.

Test Expression for RegEx with capturing groups returning 456 and 789 from two pipe-delimited records
Test Expression: the 4th field of each flat file record.

Example 3: Extract a field from the line that starts with a tag

Input 

ST|123
KV|Z|AN|1337|ABC123
ID|1912121212|

Expression 

^KV(?:(.*?)\|){4}
  • RegEx groups: 1
  • Global: checked
  • Multi line: checked

Result 

1337

After KV, the first repetition captures the empty text before the first |, then Z, AN, and 1337. Counting the tag KV as field 1, {4} returns the 4th field. Only one line starts with KV, so Global makes no difference here.

Test Expression for RegEx with capturing groups returning 1337 from the KV line of a flat file
Test Expression: the 4th field of the line tagged KV.

Example 4: Extract the 3rd field from a semicolon-delimited header line

Input 

HDR;ORDER;42
KV;Z;AN;1337;ABC123
ID;1912121212;

Expression 

^HDR(?:(.*?)[;\r\n]){3}
  • RegEx groups: 1
  • Global: unchecked
  • Multi line: unchecked

Result 

42

Each repetition ends at a ; or a line break, so the last field of the line is also captured. Counting the tag HDR as field 1, {3} returns the 3rd field: the empty text before the first ;, then ORDER, then 42. Without Multi line, ^ matches only at the start of the body, where the HDR line is.

Test Expression for RegEx with capturing groups returning 42 from the HDR line of a flat file
Test Expression with the older form [;|\n|\r]; inside a character class | is a literal pipe, not "or", and doesn't change this result.

Next Steps