diff --git a/README.md b/README.md index dcbf1b1c..f7cf5f31 100644 --- a/README.md +++ b/README.md @@ -12,13 +12,14 @@ _This is the source repository for **lnav**, visit [https://lnav.org](https://ln # LNAV -- The Logfile Navigator The Logfile Navigator is a log file viewer for the terminal. Given a -set of files, **lnav** will: +set of files/directories, **lnav** will: - decompress as needed; - detect their format; - merge the files together by time into a single view; -- tail the files and follow renames; -- build an index of errors and warnings. +- tail the files, follow renames, find new files in directories; +- build an index of errors and warnings; +- [pretty-print JSON-lines](https://docs.lnav.org/en/latest/formats.html#json-lines). Then, in the **lnav** TUI, you can: diff --git a/docs/source/formats.rst b/docs/source/formats.rst index 45dc2c31..c405b633 100644 --- a/docs/source/formats.rst +++ b/docs/source/formats.rst @@ -1,7 +1,11 @@ .. _log_formats: +*********** Log Formats -=========== +*********** + +Built-in Formats +================ Log files loaded into **lnav** are parsed based on formats defined in configuration files. Many @@ -19,6 +23,9 @@ The following log formats are built into **lnav**: :widths: 8 5 20 :file: format-table.csv +XSV Formats +----------- + In addition to the above formats, the following self-describing formats are supported: @@ -33,6 +40,24 @@ supported: self-describing, so **lnav** will read the header to determine the shape of the file. +JSON-lines +---------- + +Logs encoded as `JSON-lines `_ can be parsed and +pretty-printed in lnav by creating a log format file. The format file +is a bit simpler to create since it doesn't require a regular expression +to match plain text. Instead, the format defines the relevant fields +and provides a :code:`line-format` array that specifies how the fields +in the JSON object should be displayed. + +See the following formats that are built into lnav as examples: + +* `cloudflare_log.json `_ +* `github_events_log.json `_ + +logfmt +------ + There is also basic support for the `logfmt `_ convention for formatting log messages. Files that use this format must have the entire line be key/value pairs and the timestamp contained in a @@ -41,9 +66,8 @@ quite follow this formatting, but wraps logfmt data with another recognized format, you can use the :ref:`logfmt2json` SQL function to convert the data into JSON for further analysis. - Defining a New Format ---------------------- +===================== New log formats can be defined by placing JSON configuration files in subdirectories of the :file:`/etc/lnav/formats` and :file:`~/.lnav/formats/` @@ -55,7 +79,7 @@ modify existing ones. Format directories can also contain '.sql' and '.lnav' script files that can be used automate log file analysis. Creating a Format Using Regex101.com (v0.11.0+) -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +----------------------------------------------- For plain-text log files, the easiest way to create a log format definition is to create the regular expression that recognizes log messages using @@ -81,7 +105,7 @@ default to :code:`string`, but you'll want to change them to the appropriate type. Format File Reference -^^^^^^^^^^^^^^^^^^^^^ +--------------------- An **lnav** format file must contain a single JSON object, preferably with a :code:`$schema` property that refers to the @@ -515,7 +539,7 @@ header: :eval :filter-out ${pattern} VSCode Extension -^^^^^^^^^^^^^^^^ +---------------- The `lnav VSCode Extension `_ can be installed to add syntax highlighting to lnav scripts. @@ -566,7 +590,7 @@ Executing the format file should then install it automatically: .. _format_order: Format Order When Scanning a File ---------------------------------- +================================= When **lnav** loads a file, it tries each log format against the first 15,000 lines [#]_ of the file trying to find a match. When a match is found, that log @@ -594,7 +618,7 @@ will win. :ref:`tuning` section for more details. Automatic File Conversion -------------------------- +========================= File formats that are not naturally understood by **lnav** can be automatically detected and converted to a usable form using the diff --git a/test/expected/test_text_file.sh_5b51b55dff7332c5bee2c9b797c401c5614d574a.out b/test/expected/test_text_file.sh_5b51b55dff7332c5bee2c9b797c401c5614d574a.out index 6673f486..a8182dd7 100644 --- a/test/expected/test_text_file.sh_5b51b55dff7332c5bee2c9b797c401c5614d574a.out +++ b/test/expected/test_text_file.sh_5b51b55dff7332c5bee2c9b797c401c5614d574a.out @@ -22,13 +22,17 @@ LNAV – The Logfile Navigator The Logfile Navigator is a log file viewer for the terminal. Given a -set of files, lnav will: +set of files/directories, lnav will: • decompress as needed; • detect their format; • merge the files together by time into a single view; - • tail the files and follow renames; - • build an index of errors and warnings. + • tail the files, follow renames, find new files in + directories; + • build an index of errors and warnings; + • ]8;;https://docs.lnav.org/en/latest/formats.html#json-lines\pretty-print JSON-lines]8;;\[1]. + + ▌[1] - https://docs.lnav.org/en/latest/formats.html#json-lines Then, in the lnav TUI, you can: