Flat File CSV Search Field Expression Plugin
The Flat file CSV plugin extracts Search Field values from delimited text in the Message Body of a Log Event, such as CSV exports and semicolon-separated flat files. You pick the columns by zero-based index, or by column name when the file has header rows.
- ✅ Any Delimiter, such as
;,,, or a longer one such as|| - ✅ Fields by zero-based index, list, range, or header name
- ✅ Skips header rows and commented-out lines
Animated walkthrough: Number of header rows 2, Delimiter , and Fields CustomerName on a logged Order-to-Cash order confirmation return Northwind Traders.
When to use it
| Your data | Use |
|---|---|
| CSV or other delimited text in the Message Body, value in a known column | Flat file CSV (this plugin) |
| Fixed-width records, value at a known character position | Flat file Fixed width |
| Delimited text in a context property, or a column you need to transform or combine further | Formula with the csv function |
| Text without a consistent column structure | RegEx |
For all plugins side by side, see Choose a plugin.
Expression syntax
Select Flat file CSV in the Type list, then fill in these fields:
The Flat file CSV plugin in the Type list.
| Field | What goes in it | Example |
|---|---|---|
| Number of header rows | How many rows at the top of the message are headers. When set, you can use the header names in Fields. With several header rows, the last one names the columns. | 0, 1 |
| Delimiter | The text that separates the fields. Usually one character, but longer delimiters such as \|\| work. For tab, paste a real tab character; the text \t doesn't work. Never leave it empty: the test then fails with Index was outside the bounds of the array. |
;, ,, \|\| |
| Fields | The columns to extract, zero-based: 0 is the first column, 1 the second. Use a list (0,2), a range (1,3-4), header names (OrderId,Customer, requires Number of header rows of at least 1), or a mix. Don't put spaces before header names: Name, City looks for a header named "City" with a leading space. |
1 |
| Ignore missing fields | Checked: rows that lack a listed column are skipped. Unchecked: the whole expression fails on the first row that lacks the column, and returns no values at all. | Checked |
| Quote | The single character that wraps (escapes) a field, so the field can contain the Delimiter. Defaults to ". |
" |
| Comment | The single character that marks a line as commented out; the plugin skips those lines. It applies only when set. | # |
- There is no row selection: every listed column is read from every row after the header rows.
- Values are returned exactly as they are in the file, spaces included: in
a; b ;c, column1is "b" with a space on each side. Unlike Flat file Fixed width, the CSV plugin doesn't trim values. A value that is only white space is dropped. - A file with several sections, each with its own column layout (a header section, a details section, a footer), is read as one table. A column index that exists in the later sections too picks up their values as well. In the walkthrough above,
Fields = OrderIdwould also returnQuantityOrdered,5,TrackingEnabled, andtrue; onlyCustomerName, a column that exists in the header section alone, gives a clean value. For such files, use Formula with the csv function, or RegEx with capturing groups.
The plugin returns each unique value once (see How expressions run). Configure the plugin in Add or manage Search Field and check the result with Test Your Expression. When no field matches, the test reports Processed with warning: The Expression yields no result instead of values. When the plugin fails, for example on an empty Delimiter, the error message is shown below it:
The test result when the expression finds no value.
Examples
Example 1: Extract order numbers from a semicolon-separated file
Input
ORD;101;ACME;456;Company Name ACME ;Dieselgate valley 1;123 45;Flameburg;
ORD;102;Northwind;789;Company Name Northwind;Radiator Spring 420;543 21;Apocalypsenburg;
Expression
Number of header rows = 0
Delimiter = ;
Fields = 1
Ignore missing fields = checked
Quote = "
Comment = #
Result
101
102
Field 1 is the second column, because Fields are zero-based.
Field 1 returns the order numbers 101 and 102.
Example 2: Extract a column by its header name
Input
OrderId;Customer;Amount
5001;ACME;1200.50
5002;Northwind;89.90
5003;ACME;310.00
Expression
Number of header rows = 1
Delimiter = ;
Fields = Customer
Result
ACME
Northwind
The header row names the columns, so Customer selects the second column. ACME appears twice in the data but is returned once.
With one header row, the Fields value Customer selects the column by name.