[docs] more sections

pull/69/head
Timothy Stack 11 years ago
parent fb7d1f1dc9
commit f716d5a533

@ -4,3 +4,26 @@ Extracting Data
**Note**: This feature is still in **BETA**, you should expect bugs and
incompatible changes in the future.
Log messages contain a good deal of useful data, but it's not always easy to
get at. The log parser built into **lnav** is able to extract data as
described by log formats as well as discovering data in plain text messages.
This data can then be queried and processed using the SQLite front-end that is
also incorporated into **lnav**. As an example, the following Syslog message
from :cmd:`sudo` will be parsed and several
Jul 31 11:42:26 Example-MacBook-Pro.local sudo[87024]: testuser : TTY=ttys004 ; PWD=/Users/testuser/github/lbuild ; USER=root ; COMMAND=/usr/bin/make install
Current Time: 2013-07-31T11:42:26.000 Original Time: 2013-07-31T11:42:26.000 Offset: +0.000
Known message fields:
├ log_hostname = Example-MacBook-Pro.local
├ log_procname = sudo
├ log_pid = 87024
Discovered message fields:
├ col_0 = testuser
├ TTY = ttys004
├ PWD = /Users/testuser/github/lbuild
├ USER = root
└ COMMAND = /usr/bin/make install

@ -30,10 +30,10 @@ can consult that file when writing your own formats or if you need to modify
existing ones.
The contents of the format configuration should be a JSON object with a field
for each format defined by the file.
The symbolic name of the format. This value will also be
used as the SQL table name for the log.
for each format defined by the file. Each field name should be the symbolic
name of the format. This value will also be used as the SQL table name for
the log. The value for each field should be another object with the following
fields:
:title: The short and human-readable name for the format.
:description: A longer description of the format.
@ -51,10 +51,22 @@ The symbolic name of the format. This value will also be
in order of severity, are: **fatal**, **critical**, **error**,
**warning**, **info**, **debug**, **trace**.
:value:
:value: This object contains the definitions for the values captured by the
regexes.
:kind: The type of data that was captured **string**, **integer**,
**float**.
:collate: The collation function for this value.
:identifier: A boolean that indicates whether or not this field represents
an identifier and should be syntax colored.
:foreign-key: A boolean that indicates that this field is a key and should
not be graphed. This should only need to be set for integer fields.
:sample: A list of objects that contain sample log messages. All formats
must include at least one sample and it must be matched by one of the
included regexes. Each object must contain the following field:
:kind: **string**, **integer**, **float**.
:collate:
:line: The sample message.
Example format::
@ -89,3 +101,5 @@ Example format::
Modifying an Existing Format
----------------------------

@ -22,6 +22,7 @@ Contents:
config
hotkeys
commands
sqlext
Indices and tables
==================

@ -0,0 +1,138 @@
SQLite Extensions Reference
===========================
To make it easier to analyze log data from within **lnav**, there are several
built-in extensions that provide extra functions and collators beyond those
`provided by sqlite <http://www.sqlite.org/lang_corefunc.html>`_. The majority
of the functions are from the
`extensions-functions.c <http://www.sqlite.org/contrib>`_ file available from
the `sqlite.org <http://sqlite.org>`_ web site.
*Tip*: You can include a SQLite database file on the command-line and use
**lnav**'s interface to perform queries. The database will be attached with
a name based on the database file name.
Math
----
Basic mathematical functions:
* cos(n)
* sin(n)
* tan(n)
* cot(n)
* cosh(n)
* sinh(n)
* coth(n)
* acos(n)
* asin(n)
* atan(r1,r2)
* atan2(r1,r2)
* exp(n)
* log(n)
* log10(n)
* power(x,y)
* sign(n) - Return one of 3 possibilities +1,0 or -1 when the argument is
respectively positive, 0 or negative.
* sqrt(n)
* square(n)
* ceil(n)
* floor(n)
* pi()
* degrees - Convert radians to degrees
* radians - Convert degrees to radians
Aggregate functions:
* stddev
* variance
* mode
* median
* lower_quartile
* upper_quartile
String
------
Additional string comparison and manipulation functions:
* difference(s1,s2) - Computes the number of different characters between the
soundex value fo 2 strings.
* replicate(s,n) - Given a string (s) in the first argument and an integer (n)
in the second returns the string that constains s contatenated n times.
* proper(s) - Ensures that the words in the given string have their first
letter capitalized and the following letters are lower case.
* charindex(s1,s2), charindex(s1,s2,n) - Given 2 input strings (s1,s2) and an
integer (n) searches from the nth character for the string s1. Returns the
position where the match occured. Characters are counted from 1. 0 is
returned when no match occurs.
* leftstr(s,n) - Given a string (s) and an integer (n) returns the n leftmost
(UTF-8) characters if the string has a length<=n or is NULL this function is
NOP.
* rightstr(s,n) - Given a string (s) and an integer (n) returns the n rightmost
(UTF-8) characters if the string has a length<=n or is NULL this function is
NOP
* reverse(s) - Given a string returns the same string but with the characters
in reverse order.
* padl(s,n) - Given an input string (s) and an integer (n) adds spaces at the
beginning of (s) until it has a length of n characters. When s has a length
>=n it's a NOP. padl(NULL) = NULL
* padr(s,n) - Given an input string (s) and an integer (n) appends spaces at
the end of s until it has a length of n characters. When s has a length >=n
it's a NOP. padr(NULL) = NULL
* padc(s,n) - Given an input string (s) and an integer (n) appends spaces at
the end of s and adds spaces at the begining of s until it has a length of n
characters. Tries to add has many characters at the left as at the right.
When s has a length >=n it's a NOP. padc(NULL) = NULL
* strfilter(s1,s2) - Given 2 string (s1,s2) returns the string s1 with the
characters NOT in s2 removed assumes strings are UTF-8 encoded.
* regexp(re,s) - Return 1 if the regular expression 're' matches the given
string.
* regexp_replace(str, re, repl) - Replace the portions of the given string
that match the regular expression with the replacement string. **NOTE**:
The arguments for the string and the regular expression in this function are
reversed from the plain regexp() function. This is to be somewhat compatible
with functions in other database implementations.
* startswith(s1,prefix) - Given a string and prefix, return 1 if the string
starts with the given prefix.
* endswith(s1,suffix) - Given a string and suffix, return 1 if the string ends
with the given suffix.
File Paths
----------
File path manipulation functions:
* basename(s) - Return the file name part of a path.
* dirname(s) - Return the directory part of a path.
* joinpath(s1,s2,...) - Return the arguments joined together into a path.
Networking
----------
Network information functions:
* gethostbyname - Convert a host name into an IP address. The host name could
not be resolved, the input value will be returned.
* gethostbyaddr - Convert an IPv4/IPv6 address into a host name. If the
reverse lookup fails, the input value will be returned.
Internal State
--------------
The following functions can be used to access **lnav**'s internal state:
* log_top_line() - Return the line number at the top of the log view.
* log_top_datetime() - Return the timestamp of the line at the top of the log
view.
Collators
---------
* naturalcase - Compare strings "naturally" so that number values in the string
are compared based on their numeric value and not their character values.
For example, "foo10" would be considered greater than "foo2".
* naturalnocase - The same as naturalcase, but case-insensitive.
* ipaddress - Compare IPv4/IPv6 addresses.

@ -165,6 +165,10 @@ int fs_extension_functions(const struct FuncDef **basic_funcs,
{ "dirname", 1, 0, SQLITE_UTF8, 0, sql_dirname },
{ "joinpath", -1, 0, SQLITE_UTF8, 0, sql_joinpath },
/*
* TODO: add other functions like readlink, normpath, ...
*/
{ NULL }
};

Loading…
Cancel
Save