[prql] some more docs/tweaks

circleci-project-setup
Tim Stack 2 months ago
parent c075da3eaa
commit 991972b62f

@ -80,6 +80,30 @@ and lnav will switch accordingly. A major advantage of using PRQL is that
lnav can show previews of the results of the pipeline stages and provide
better tab completion options.
A PRQL query starts with the :code:`from` keyword that specifies the table
to use as a data source. The next stage of a pipeline is started by
entering a pipe symbol (:code:`|`) followed by a
`PRQL transform <https://prql-lang.org/book/reference/stdlib/transforms/index.html>`_.
As you build the query in the prompt, lnav will display any relevant
help and preview for the current and previous stages of the pipeline.
The following is a screenshot of lnav viewing a web access log with a
query in progress:
.. figure:: ../assets/images/lnav-prql-preview.png
:align: center
Screenshot of a PRQL query in progress
The top half of the window is the usual log message view. Below that is
the online help panel showing the documentation for the :code:`stats.count_by`
PRQL function. lnav will show the help for what is currently under the
cursor. The next panel shows the preview data for the pipeline stage
that precedes the stage where the cursor is. In this case, the
results of :code:`from access_log`, which is the contents of the access
log table. The second preview window shows the result of the
pipeline stage where the cursor is located.
Log Tables
----------

@ -29,6 +29,7 @@
#include "dump_internals.hh"
#include "bound_tags.hh"
#include "lnav.events.hh"
#include "lnav.hh"
#include "lnav_config.hh"
@ -42,6 +43,9 @@ namespace lnav {
void
dump_internals(const char* internals_dir)
{
static const auto* sql_cmd_map
= injector::get<readline_context::command_map_t*, sql_cmd_map_tag>();
for (const auto* handlers :
std::initializer_list<const json_path_container*>{
&lnav_config_handlers,
@ -87,6 +91,13 @@ dump_internals(const char* internals_dir)
unique_sql_help.insert(sql.second);
format_help_text_for_rst(*sql.second, eval_example, sql_file.get());
}
for (const auto& cmd_pair : *sql_cmd_map) {
if (cmd_pair.second->c_help.ht_name == nullptr) {
continue;
}
format_help_text_for_rst(
cmd_pair.second->c_help, eval_example, sql_file.get());
}
}
}

@ -56,7 +56,7 @@ have been loaded, you can use the following options:
* `-c cmd` A command, query, or file to execute. The first character
determines the type of operation: a colon (`:`) is used for the
built-in commands; a semi-colon (`;`) for SQL queries; and a
built-in commands; a semi-colon (`;`) for SQL/PRQL queries; and a
pipe symbol (`|`) for executing a file containing other
commands. For example, to open the file "foo.log" and go
to the tenth line in the file, you can do:
@ -315,7 +315,7 @@ mark lines of text and move the view by grabbing the scrollbar.
NOTE: You need to manually enable this feature by setting the LNAV_EXP
environment variable to "mouse". F2 toggles mouse support.
## SQL Queries
## Log Analysis
Lnav has support for performing SQL queries on log files using the
SQLite3 "virtual" table feature. For all supported log file types,
@ -415,6 +415,37 @@ example of a top ten query into the "/tmp/topten.db" file, you can do:
LIMIT 10;
```
### PRQL Support
The Pipelined Relational Query Language
[(PRQL)](https://prql-lang.org) is an alternative database query
language that compiles to SQL. The main advantage of PRQL,
in the context of lnav, is that it is easier to work with
interactively compared to SQL. For example, lnav can provide
previews of different stages of the pipeline and provide more
accurate tab-completions for the columns in the result set.
You can execute a PRQL query in the SQL prompt. A PRQL query
starts with the `from` keyword that specifies the table to use as
a data source. The next stage of a pipeline is started by entering
a pipe symbol (`|`) followed by a
[PRQL transform](https://prql-lang.org/book/reference/stdlib/transforms/index.html).
As you build the query in the prompt, lnav will display any relevant
help and preview for the current and previous stages of the pipeline.
Using the top ten URLs query from earlier as an example, the PRQL
version would be as follows:
```lnav
;from access_log | stats.count_by cs_uri_stem | take 10
```
The first stage selects the data source, the web `access_log` table
in this case. The `stats.count_by` transform is a convenience
provided by lnav that groups by the given column, counts the rows
in each group, and sorts by count in descending order. The `take 10`
turns into the `LIMIT 10`.
## Dynamic logline Table (experimental)
(NOTE: This feature is still very new and not completely reliable yet,

@ -634,7 +634,14 @@ link_name(const help_text& ht)
if (is_sql_infix) {
scrubbed_name = "infix";
} else {
scrubbed_name = ht.ht_name;
if (ht.ht_context == help_context_t::HC_PRQL_TRANSFORM) {
scrubbed_name += "prql_";
}
scrubbed_name += ht.ht_name;
if (scrubbed_name[0] == '.') {
scrubbed_name.erase(scrubbed_name.begin());
scrubbed_name.insert(0, "dot_");
}
}
if (ht.ht_function_type == help_function_type_t::HFT_AGGREGATE) {
scrubbed_name += "_agg";
@ -667,7 +674,7 @@ format_help_text_for_rst(const help_text& ht,
return;
}
bool is_sql_func = false, is_sql = false;
bool is_sql_func = false, is_sql = false, is_prql = false;
switch (ht.ht_context) {
case help_context_t::HC_COMMAND:
prefix = ":";
@ -688,6 +695,8 @@ format_help_text_for_rst(const help_text& ht,
case help_context_t::HC_PRQL_TRANSFORM:
case help_context_t::HC_PRQL_FUNCTION:
is_sql = true;
is_prql = true;
prefix = "";
break;
default:
prefix = "";
@ -721,6 +730,11 @@ format_help_text_for_rst(const help_text& ht,
out_count += fmt::fprintf(rst_file, "\\[");
}
out_count += fmt::fprintf(rst_file, "%s", param.ht_name);
if (is_prql && param.ht_default_value) {
out_count += fmt::fprintf(rst_file, ":");
out_count
+= fmt::fprintf(rst_file, "%s", param.ht_default_value);
}
if (param.ht_nargs == help_nargs_t::HN_OPTIONAL) {
out_count += fmt::fprintf(rst_file, "\\]");
}

@ -80,7 +80,7 @@
:append-to /tmp/interesting-lines.txt
**See Also**
:ref:`echo`, :ref:`echoln`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`redirect_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_view_to`
:ref:`dot_dump`, :ref:`dot_read`, :ref:`echo`, :ref:`echoln`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`redirect_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_view_to`
----
@ -476,7 +476,7 @@
:echo Hello, World!
**See Also**
:ref:`alt_msg`, :ref:`append_to`, :ref:`cd`, :ref:`echoln`, :ref:`eval`, :ref:`export_session_to`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`rebuild`, :ref:`redirect_to`, :ref:`redirect_to`, :ref:`sh`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_to`, :ref:`write_view_to`, :ref:`write_view_to`
:ref:`alt_msg`, :ref:`append_to`, :ref:`cd`, :ref:`dot_dump`, :ref:`dot_read`, :ref:`echoln`, :ref:`eval`, :ref:`export_session_to`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`rebuild`, :ref:`redirect_to`, :ref:`redirect_to`, :ref:`sh`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_to`, :ref:`write_view_to`, :ref:`write_view_to`
----
@ -551,7 +551,7 @@
* **path\*** --- The path to the file to write
**See Also**
:ref:`alt_msg`, :ref:`append_to`, :ref:`cd`, :ref:`echo`, :ref:`echo`, :ref:`echoln`, :ref:`eval`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`rebuild`, :ref:`redirect_to`, :ref:`redirect_to`, :ref:`sh`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_to`, :ref:`write_view_to`, :ref:`write_view_to`
:ref:`alt_msg`, :ref:`append_to`, :ref:`cd`, :ref:`dot_dump`, :ref:`dot_read`, :ref:`echo`, :ref:`echo`, :ref:`echoln`, :ref:`eval`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`rebuild`, :ref:`redirect_to`, :ref:`redirect_to`, :ref:`sh`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_to`, :ref:`write_view_to`, :ref:`write_view_to`
----
@ -982,7 +982,7 @@
:pipe-line-to sed -e 's/foo/bar/g'
**See Also**
:ref:`append_to`, :ref:`echo`, :ref:`echoln`, :ref:`export_session_to`, :ref:`pipe_to`, :ref:`redirect_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_view_to`
:ref:`append_to`, :ref:`dot_dump`, :ref:`dot_read`, :ref:`echo`, :ref:`echoln`, :ref:`export_session_to`, :ref:`pipe_to`, :ref:`redirect_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_view_to`
----
@ -1005,7 +1005,7 @@
:pipe-to sed -e s/foo/bar/g
**See Also**
:ref:`append_to`, :ref:`echo`, :ref:`echoln`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`redirect_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_view_to`
:ref:`append_to`, :ref:`dot_dump`, :ref:`dot_read`, :ref:`echo`, :ref:`echoln`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`redirect_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_view_to`
----
@ -1131,7 +1131,7 @@
:redirect-to /tmp/script-output.txt
**See Also**
:ref:`alt_msg`, :ref:`append_to`, :ref:`cd`, :ref:`echo`, :ref:`echo`, :ref:`echoln`, :ref:`eval`, :ref:`export_session_to`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`rebuild`, :ref:`sh`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_to`, :ref:`write_view_to`, :ref:`write_view_to`
:ref:`alt_msg`, :ref:`append_to`, :ref:`cd`, :ref:`dot_dump`, :ref:`dot_read`, :ref:`echo`, :ref:`echo`, :ref:`echoln`, :ref:`eval`, :ref:`export_session_to`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`rebuild`, :ref:`sh`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_to`, :ref:`write_view_to`, :ref:`write_view_to`
----
@ -1554,7 +1554,7 @@
:write-table-to /tmp/table.txt
**See Also**
:ref:`alt_msg`, :ref:`append_to`, :ref:`cd`, :ref:`create_logline_table`, :ref:`create_search_table`, :ref:`echo`, :ref:`echo`, :ref:`echoln`, :ref:`eval`, :ref:`export_session_to`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`rebuild`, :ref:`redirect_to`, :ref:`redirect_to`, :ref:`sh`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_to`, :ref:`write_to`, :ref:`write_view_to`, :ref:`write_view_to`, :ref:`write_view_to`
:ref:`alt_msg`, :ref:`append_to`, :ref:`cd`, :ref:`create_logline_table`, :ref:`create_search_table`, :ref:`dot_dump`, :ref:`dot_read`, :ref:`echo`, :ref:`echo`, :ref:`echoln`, :ref:`eval`, :ref:`export_session_to`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`rebuild`, :ref:`redirect_to`, :ref:`redirect_to`, :ref:`sh`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_to`, :ref:`write_to`, :ref:`write_view_to`, :ref:`write_view_to`, :ref:`write_view_to`
----
@ -1578,7 +1578,7 @@
:write-csv-to /tmp/table.csv
**See Also**
:ref:`alt_msg`, :ref:`append_to`, :ref:`cd`, :ref:`create_logline_table`, :ref:`create_search_table`, :ref:`echo`, :ref:`echo`, :ref:`echoln`, :ref:`eval`, :ref:`export_session_to`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`rebuild`, :ref:`redirect_to`, :ref:`redirect_to`, :ref:`sh`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_to`, :ref:`write_view_to`, :ref:`write_view_to`, :ref:`write_view_to`
:ref:`alt_msg`, :ref:`append_to`, :ref:`cd`, :ref:`create_logline_table`, :ref:`create_search_table`, :ref:`dot_dump`, :ref:`dot_read`, :ref:`echo`, :ref:`echo`, :ref:`echoln`, :ref:`eval`, :ref:`export_session_to`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`rebuild`, :ref:`redirect_to`, :ref:`redirect_to`, :ref:`sh`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_to`, :ref:`write_view_to`, :ref:`write_view_to`, :ref:`write_view_to`
----
@ -1602,7 +1602,7 @@
:write-json-to /tmp/table.json
**See Also**
:ref:`alt_msg`, :ref:`append_to`, :ref:`cd`, :ref:`create_logline_table`, :ref:`create_search_table`, :ref:`echo`, :ref:`echo`, :ref:`echoln`, :ref:`eval`, :ref:`export_session_to`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`rebuild`, :ref:`redirect_to`, :ref:`redirect_to`, :ref:`sh`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_to`, :ref:`write_view_to`, :ref:`write_view_to`, :ref:`write_view_to`
:ref:`alt_msg`, :ref:`append_to`, :ref:`cd`, :ref:`create_logline_table`, :ref:`create_search_table`, :ref:`dot_dump`, :ref:`dot_read`, :ref:`echo`, :ref:`echo`, :ref:`echoln`, :ref:`eval`, :ref:`export_session_to`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`rebuild`, :ref:`redirect_to`, :ref:`redirect_to`, :ref:`sh`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_to`, :ref:`write_view_to`, :ref:`write_view_to`, :ref:`write_view_to`
----
@ -1626,7 +1626,7 @@
:write-jsonlines-to /tmp/table.json
**See Also**
:ref:`alt_msg`, :ref:`append_to`, :ref:`cd`, :ref:`create_logline_table`, :ref:`create_search_table`, :ref:`echo`, :ref:`echo`, :ref:`echoln`, :ref:`eval`, :ref:`export_session_to`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`rebuild`, :ref:`redirect_to`, :ref:`redirect_to`, :ref:`sh`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_to`, :ref:`write_view_to`, :ref:`write_view_to`, :ref:`write_view_to`
:ref:`alt_msg`, :ref:`append_to`, :ref:`cd`, :ref:`create_logline_table`, :ref:`create_search_table`, :ref:`dot_dump`, :ref:`dot_read`, :ref:`echo`, :ref:`echo`, :ref:`echoln`, :ref:`eval`, :ref:`export_session_to`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`rebuild`, :ref:`redirect_to`, :ref:`redirect_to`, :ref:`sh`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_to`, :ref:`write_view_to`, :ref:`write_view_to`, :ref:`write_view_to`
----
@ -1651,7 +1651,7 @@
:write-raw-to /tmp/table.txt
**See Also**
:ref:`alt_msg`, :ref:`append_to`, :ref:`cd`, :ref:`create_logline_table`, :ref:`create_search_table`, :ref:`echo`, :ref:`echo`, :ref:`echoln`, :ref:`eval`, :ref:`export_session_to`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`rebuild`, :ref:`redirect_to`, :ref:`redirect_to`, :ref:`sh`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_to`, :ref:`write_view_to`, :ref:`write_view_to`, :ref:`write_view_to`
:ref:`alt_msg`, :ref:`append_to`, :ref:`cd`, :ref:`create_logline_table`, :ref:`create_search_table`, :ref:`dot_dump`, :ref:`dot_read`, :ref:`echo`, :ref:`echo`, :ref:`echoln`, :ref:`eval`, :ref:`export_session_to`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`rebuild`, :ref:`redirect_to`, :ref:`redirect_to`, :ref:`sh`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_to`, :ref:`write_view_to`, :ref:`write_view_to`, :ref:`write_view_to`
----
@ -1675,7 +1675,7 @@
:write-screen-to /tmp/table.txt
**See Also**
:ref:`alt_msg`, :ref:`append_to`, :ref:`cd`, :ref:`create_logline_table`, :ref:`create_search_table`, :ref:`echo`, :ref:`echo`, :ref:`echoln`, :ref:`eval`, :ref:`export_session_to`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`rebuild`, :ref:`redirect_to`, :ref:`redirect_to`, :ref:`sh`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_to`, :ref:`write_view_to`, :ref:`write_view_to`, :ref:`write_view_to`
:ref:`alt_msg`, :ref:`append_to`, :ref:`cd`, :ref:`create_logline_table`, :ref:`create_search_table`, :ref:`dot_dump`, :ref:`dot_read`, :ref:`echo`, :ref:`echo`, :ref:`echoln`, :ref:`eval`, :ref:`export_session_to`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`rebuild`, :ref:`redirect_to`, :ref:`redirect_to`, :ref:`sh`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_to`, :ref:`write_view_to`, :ref:`write_view_to`, :ref:`write_view_to`
----
@ -1699,7 +1699,7 @@
:write-to /tmp/interesting-lines.txt
**See Also**
:ref:`alt_msg`, :ref:`append_to`, :ref:`cd`, :ref:`echo`, :ref:`echo`, :ref:`echoln`, :ref:`eval`, :ref:`export_session_to`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`rebuild`, :ref:`redirect_to`, :ref:`redirect_to`, :ref:`sh`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_view_to`, :ref:`write_view_to`
:ref:`alt_msg`, :ref:`append_to`, :ref:`cd`, :ref:`dot_dump`, :ref:`dot_read`, :ref:`echo`, :ref:`echo`, :ref:`echoln`, :ref:`eval`, :ref:`export_session_to`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`rebuild`, :ref:`redirect_to`, :ref:`redirect_to`, :ref:`sh`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_view_to`, :ref:`write_view_to`
----
@ -1723,7 +1723,7 @@
:write-view-to /tmp/table.txt
**See Also**
:ref:`alt_msg`, :ref:`append_to`, :ref:`cd`, :ref:`create_logline_table`, :ref:`create_search_table`, :ref:`echo`, :ref:`echo`, :ref:`echoln`, :ref:`eval`, :ref:`export_session_to`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`rebuild`, :ref:`redirect_to`, :ref:`redirect_to`, :ref:`sh`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_to`
:ref:`alt_msg`, :ref:`append_to`, :ref:`cd`, :ref:`create_logline_table`, :ref:`create_search_table`, :ref:`dot_dump`, :ref:`dot_read`, :ref:`echo`, :ref:`echo`, :ref:`echoln`, :ref:`eval`, :ref:`export_session_to`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`rebuild`, :ref:`redirect_to`, :ref:`redirect_to`, :ref:`sh`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_to`
----

@ -182,6 +182,10 @@ DETACH DATABASE *schema-name*
.. code-block:: custsqlite
;DETACH DATABASE customers
✘ error: SQL statement failed
reason: no such database: customers
--> command:1
----
@ -1119,7 +1123,7 @@ echoln(*value*)
* **value\*** --- The value to write to the current output file
**See Also**
:ref:`append_to`, :ref:`echo`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`redirect_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_view_to`
:ref:`append_to`, :ref:`dot_dump`, :ref:`dot_read`, :ref:`echo`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`redirect_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_view_to`
----
@ -4470,3 +4474,460 @@ zeroblob(*N*)
----
.. _dot_dump:
;.dump *path*
^^^^^^^^^^^^^
Dump the contents of the database
**Parameters**
* **path\*** --- The path to the file to write
**See Also**
:ref:`append_to`, :ref:`dot_read`, :ref:`echo`, :ref:`echoln`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`redirect_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_view_to`
----
.. _dot_msgformats:
;.msgformats
^^^^^^^^^^^^
Executes a query that will summarize the different message formats found in the logs
----
.. _dot_read:
;.read *path*
^^^^^^^^^^^^^
Execute the SQLite statements in the given file
**Parameters**
* **path\*** --- The path to the file to write
**See Also**
:ref:`append_to`, :ref:`dot_dump`, :ref:`echo`, :ref:`echoln`, :ref:`export_session_to`, :ref:`pipe_line_to`, :ref:`pipe_to`, :ref:`redirect_to`, :ref:`write_csv_to`, :ref:`write_json_to`, :ref:`write_jsonlines_to`, :ref:`write_raw_to`, :ref:`write_screen_to`, :ref:`write_table_to`, :ref:`write_to`, :ref:`write_view_to`
----
.. _dot_schema:
;.schema
^^^^^^^^
Switch to the SCHEMA view that contains a dump of the current database schema
----
.. _prql_aggregate:
aggregate *expr*
^^^^^^^^^^^^^^^^
PRQL transform to summarize many rows into one
**Parameters**
* **expr\*** --- The aggregate expression(s)
**Examples**
To group values into a JSON array:
.. code-block:: custsqlite
;from [{a=1}, {a=2}] | aggregate { arr = json.group_array a }
[1,2]
**See Also**
:ref:`prql_append`, :ref:`prql_derive`, :ref:`prql_filter`, :ref:`prql_from`, :ref:`prql_group`, :ref:`prql_join`, :ref:`prql_select`, :ref:`prql_sort`, :ref:`prql_take`, :ref:`stats_average_of`, :ref:`stats_by`, :ref:`stats_count_by`, :ref:`stats_sum_of`, :ref:`utils_distinct`
----
.. _prql_append:
append *table*
^^^^^^^^^^^^^^
PRQL transform to concatenate tables together
**Parameters**
* **table\*** --- The table to use as a source
**See Also**
:ref:`prql_aggregate`, :ref:`prql_derive`, :ref:`prql_filter`, :ref:`prql_from`, :ref:`prql_group`, :ref:`prql_join`, :ref:`prql_select`, :ref:`prql_sort`, :ref:`prql_take`, :ref:`stats_average_of`, :ref:`stats_by`, :ref:`stats_count_by`, :ref:`stats_sum_of`, :ref:`utils_distinct`
----
.. _prql_derive:
derive *column*
^^^^^^^^^^^^^^^
PRQL transform to derive one or more columns
**Parameters**
* **column\*** --- The new column
**Examples**
To add a column that is a multiplication of another:
.. code-block:: custsqlite
;from [{a=1}, {a=2}] | derive b = a * 2
a b
1 2
2 4
**See Also**
:ref:`prql_aggregate`, :ref:`prql_append`, :ref:`prql_filter`, :ref:`prql_from`, :ref:`prql_group`, :ref:`prql_join`, :ref:`prql_select`, :ref:`prql_sort`, :ref:`prql_take`, :ref:`stats_average_of`, :ref:`stats_by`, :ref:`stats_count_by`, :ref:`stats_sum_of`, :ref:`utils_distinct`
----
.. _prql_filter:
filter *expr*
^^^^^^^^^^^^^
PRQL transform to pick rows based on their values
**Parameters**
* **expr\*** --- The expression to evaluate over each row
**Examples**
To pick rows where 'a' is greater than one:
.. code-block:: custsqlite
;from [{a=1}, {a=2}] | filter a > 1
2
**See Also**
:ref:`prql_aggregate`, :ref:`prql_append`, :ref:`prql_derive`, :ref:`prql_from`, :ref:`prql_group`, :ref:`prql_join`, :ref:`prql_select`, :ref:`prql_sort`, :ref:`prql_take`, :ref:`stats_average_of`, :ref:`stats_by`, :ref:`stats_count_by`, :ref:`stats_sum_of`, :ref:`utils_distinct`
----
.. _prql_from:
from *table*
^^^^^^^^^^^^
PRQL command to specify a data source
**Parameters**
* **table\*** --- The table to use as a source
**Examples**
To pull data from the 'http_status_codes' database table:
.. code-block:: custsqlite
;from db.http_status_codes | take 3
✘ error: failed to compile SQL statement
reason: no such table: db.http_status_codes
--> command:1
| SELECT
| *
| FROM
| db.http_status_codes
| LIMIT
| 3
|
| -- Generated by PRQL compiler version:0.11.5 target:sql.sqlite (https://prql-lang.org)
|
To use an array literal as a source:
.. code-block:: custsqlite
;from [{ col1=1, col2='abc' }, { col1=2, col2='def' }]
col1 col2
1 abc
2 def
**See Also**
:ref:`prql_aggregate`, :ref:`prql_append`, :ref:`prql_derive`, :ref:`prql_filter`, :ref:`prql_group`, :ref:`prql_join`, :ref:`prql_select`, :ref:`prql_sort`, :ref:`prql_take`, :ref:`stats_average_of`, :ref:`stats_by`, :ref:`stats_count_by`, :ref:`stats_sum_of`, :ref:`utils_distinct`
----
.. _prql_group:
group *key_columns* *pipeline*
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PRQL transform to partition rows into groups
**Parameters**
* **key_columns\*** --- The columns that define the group
* **pipeline\*** --- The pipeline to execute over a group
**Examples**
To group by log_level and count the rows in each partition:
.. code-block:: custsqlite
;from lnav_example_log | group { log_level } (aggregate { count this })
log_level COUNT(*)
debug 1
info 1
warn 1
error 1
**See Also**
:ref:`prql_aggregate`, :ref:`prql_append`, :ref:`prql_derive`, :ref:`prql_filter`, :ref:`prql_from`, :ref:`prql_join`, :ref:`prql_select`, :ref:`prql_sort`, :ref:`prql_take`, :ref:`stats_average_of`, :ref:`stats_by`, :ref:`stats_count_by`, :ref:`stats_sum_of`, :ref:`utils_distinct`
----
.. _prql_join:
join *\[side:inner\]* *table* *condition*
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PRQL transform to add columns from another table
**Parameters**
* **side** --- Specifies which rows to include
* **table\*** --- The other table to join with the current rows
* **condition\*** --- The condition used to join rows
**See Also**
:ref:`prql_aggregate`, :ref:`prql_append`, :ref:`prql_derive`, :ref:`prql_filter`, :ref:`prql_from`, :ref:`prql_group`, :ref:`prql_select`, :ref:`prql_sort`, :ref:`prql_take`, :ref:`stats_average_of`, :ref:`stats_by`, :ref:`stats_count_by`, :ref:`stats_sum_of`, :ref:`utils_distinct`
----
.. _prql_select:
select *expr*
^^^^^^^^^^^^^
PRQL transform to pick and compute columns
**Parameters**
* **expr\*** --- The columns to include in the result set
**Examples**
To pick the 'b' column from the rows:
.. code-block:: custsqlite
;from [{a=1, b='abc'}, {a=2, b='def'}] | select b
b
abc
def
To compute a new column from an input:
.. code-block:: custsqlite
;from [{a=1}, {a=2}] | select b = a * 2
b
2
4
**See Also**
:ref:`prql_aggregate`, :ref:`prql_append`, :ref:`prql_derive`, :ref:`prql_filter`, :ref:`prql_from`, :ref:`prql_group`, :ref:`prql_join`, :ref:`prql_sort`, :ref:`prql_take`, :ref:`stats_average_of`, :ref:`stats_by`, :ref:`stats_count_by`, :ref:`stats_sum_of`, :ref:`utils_distinct`
----
.. _prql_sort:
sort *expr*
^^^^^^^^^^^
PRQL transform to sort rows
**Parameters**
* **expr\*** --- The values to use when ordering the result set
**Examples**
To sort the rows in descending order:
.. code-block:: custsqlite
;from [{a=1}, {a=2}] | sort {-a}
a
2
1
**See Also**
:ref:`prql_aggregate`, :ref:`prql_append`, :ref:`prql_derive`, :ref:`prql_filter`, :ref:`prql_from`, :ref:`prql_group`, :ref:`prql_join`, :ref:`prql_select`, :ref:`prql_take`, :ref:`stats_average_of`, :ref:`stats_by`, :ref:`stats_count_by`, :ref:`stats_sum_of`, :ref:`utils_distinct`
----
.. _stats_average_of:
stats.average_of *col*
^^^^^^^^^^^^^^^^^^^^^^
Compute the average of col
**Parameters**
* **col\*** --- The column to average
**Examples**
To get the average of a:
.. code-block:: custsqlite
;from [{a=1}, {a=1}, {a=2}] | stats.average_of a
1.3333333333333333
**See Also**
:ref:`prql_aggregate`, :ref:`prql_append`, :ref:`prql_derive`, :ref:`prql_filter`, :ref:`prql_from`, :ref:`prql_group`, :ref:`prql_join`, :ref:`prql_select`, :ref:`prql_sort`, :ref:`prql_take`, :ref:`stats_by`, :ref:`stats_count_by`, :ref:`stats_sum_of`, :ref:`utils_distinct`
----
.. _stats_by:
stats.by *col* *values*
^^^^^^^^^^^^^^^^^^^^^^^
A shorthand for grouping and aggregating
**Parameters**
* **col\*** --- The column to sum
* **values\*** --- The aggregations to perform
**Examples**
To partition by a and get the sum of b:
.. code-block:: custsqlite
;from [{a=1, b=1}, {a=1, b=1}, {a=2, b=1}] | stats.by a {sum b}
a COALESC⋯(b), 0)
1 2
2 1
**See Also**
:ref:`prql_aggregate`, :ref:`prql_append`, :ref:`prql_derive`, :ref:`prql_filter`, :ref:`prql_from`, :ref:`prql_group`, :ref:`prql_join`, :ref:`prql_select`, :ref:`prql_sort`, :ref:`prql_take`, :ref:`stats_average_of`, :ref:`stats_count_by`, :ref:`stats_sum_of`, :ref:`utils_distinct`
----
.. _stats_count_by:
stats.count_by *column*
^^^^^^^^^^^^^^^^^^^^^^^
Partition rows and count the number of rows in each partition
**Parameters**
* **column** --- The columns to group by
**Examples**
To count rows for a particular value of column 'a':
.. code-block:: custsqlite
;from [{a=1}, {a=1}, {a=2}] | stats.count_by a
a total
1 2
2 1
**See Also**
:ref:`prql_aggregate`, :ref:`prql_append`, :ref:`prql_derive`, :ref:`prql_filter`, :ref:`prql_from`, :ref:`prql_group`, :ref:`prql_join`, :ref:`prql_select`, :ref:`prql_sort`, :ref:`prql_take`, :ref:`stats_average_of`, :ref:`stats_by`, :ref:`stats_sum_of`, :ref:`utils_distinct`
----
.. _stats_sum_of:
stats.sum_of *col*
^^^^^^^^^^^^^^^^^^
Compute the sum of col
**Parameters**
* **col\*** --- The column to sum
**Examples**
To get the sum of a:
.. code-block:: custsqlite
;from [{a=1}, {a=1}, {a=2}] | stats.sum_of a
4
**See Also**
:ref:`prql_aggregate`, :ref:`prql_append`, :ref:`prql_derive`, :ref:`prql_filter`, :ref:`prql_from`, :ref:`prql_group`, :ref:`prql_join`, :ref:`prql_select`, :ref:`prql_sort`, :ref:`prql_take`, :ref:`stats_average_of`, :ref:`stats_by`, :ref:`stats_count_by`, :ref:`utils_distinct`
----
.. _prql_take:
take *n_or_range*
^^^^^^^^^^^^^^^^^
PRQL command to pick rows based on their position
**Parameters**
* **n_or_range\*** --- The number of rows or range
**Examples**
To pick the first row:
.. code-block:: custsqlite
;from [{a=1}, {a=2}, {a=3}] | take 1
1
To pick the second and third rows:
.. code-block:: custsqlite
;from [{a=1}, {a=2}, {a=3}] | take 2..3
a
2
3
**See Also**
:ref:`prql_aggregate`, :ref:`prql_append`, :ref:`prql_derive`, :ref:`prql_filter`, :ref:`prql_from`, :ref:`prql_group`, :ref:`prql_join`, :ref:`prql_select`, :ref:`prql_sort`, :ref:`stats_average_of`, :ref:`stats_by`, :ref:`stats_count_by`, :ref:`stats_sum_of`, :ref:`utils_distinct`
----
.. _utils_distinct:
utils.distinct *col*
^^^^^^^^^^^^^^^^^^^^
A shorthand for getting distinct values of col
**Parameters**
* **col\*** --- The column to sum
**Examples**
To get the distinct values of a:
.. code-block:: custsqlite
;from [{a=1}, {a=1}, {a=2}] | utils.distinct a
a
1
2
**See Also**
:ref:`prql_aggregate`, :ref:`prql_append`, :ref:`prql_derive`, :ref:`prql_filter`, :ref:`prql_from`, :ref:`prql_group`, :ref:`prql_join`, :ref:`prql_select`, :ref:`prql_sort`, :ref:`prql_take`, :ref:`stats_average_of`, :ref:`stats_by`, :ref:`stats_count_by`, :ref:`stats_sum_of`
----

@ -2740,7 +2740,7 @@ SELECT tbl_name FROM sqlite_master WHERE sql LIKE 'CREATE VIRTUAL TABLE%'
}
/* If we statically linked against an ncurses library that had a
* non- standard path to the terminfo database, we need to set this
* non-standard path to the terminfo database, we need to set this
* variable so that it will try the default path.
*/
setenv("TERMINFO_DIRS",

@ -238,7 +238,8 @@ md2attr_line::leave_block(const md4cpp::event_handler::block& bl)
}
} else if (lang_sf == "lnav") {
readline_lnav_highlighter(block_text, block_text.length());
} else if (lang_sf == "sql" || lang_sf == "sqlite") {
} else if (lang_sf == "sql" || lang_sf == "sqlite" || lang_sf == "prql")
{
readline_sqlite_highlighter(block_text, block_text.length());
} else if (lang_sf == "shell" || lang_sf == "bash") {
readline_shlex_highlighter(block_text, block_text.length());

@ -480,6 +480,7 @@ static readline_context::command_t sql_commands[] = {
prql_cmd_from,
help_text("from")
.prql_transform()
.with_tags({"prql"})
.with_summary("PRQL command to specify a data source")
.with_parameter({"table", "The table to use as a source"})
.with_example({
@ -500,6 +501,7 @@ static readline_context::command_t sql_commands[] = {
prql_cmd_aggregate,
help_text("aggregate")
.prql_transform()
.with_tags({"prql"})
.with_summary("PRQL transform to summarize many rows into one")
.with_parameter(
help_text{"expr", "The aggregate expression(s)"}.with_grouping(
@ -518,6 +520,7 @@ static readline_context::command_t sql_commands[] = {
prql_cmd_append,
help_text("append")
.prql_transform()
.with_tags({"prql"})
.with_summary("PRQL transform to concatenate tables together")
.with_parameter({"table", "The table to use as a source"}),
nullptr,
@ -529,6 +532,7 @@ static readline_context::command_t sql_commands[] = {
prql_cmd_derive,
help_text("derive")
.prql_transform()
.with_tags({"prql"})
.with_summary("PRQL transform to derive one or more columns")
.with_parameter(
help_text{"column", "The new column"}.with_grouping("{", "}"))
@ -546,6 +550,7 @@ static readline_context::command_t sql_commands[] = {
prql_cmd_filter,
help_text("filter")
.prql_transform()
.with_tags({"prql"})
.with_summary("PRQL transform to pick rows based on their values")
.with_parameter(
{"expr", "The expression to evaluate over each row"})
@ -563,6 +568,7 @@ static readline_context::command_t sql_commands[] = {
prql_cmd_group,
help_text("group")
.prql_transform()
.with_tags({"prql"})
.with_summary("PRQL transform to partition rows into groups")
.with_parameter(
help_text{"key_columns", "The columns that define the group"}
@ -585,6 +591,7 @@ static readline_context::command_t sql_commands[] = {
prql_cmd_join,
help_text("join")
.prql_transform()
.with_tags({"prql"})
.with_summary("PRQL transform to add columns from another table")
.with_parameter(
help_text{"side", "Specifies which rows to include"}
@ -605,6 +612,7 @@ static readline_context::command_t sql_commands[] = {
prql_cmd_select,
help_text("select")
.prql_transform()
.with_tags({"prql"})
.with_summary("PRQL transform to pick and compute columns")
.with_parameter(
help_text{"expr", "The columns to include in the result set"}
@ -628,6 +636,7 @@ static readline_context::command_t sql_commands[] = {
prql_cmd_sort,
help_text("stats.average_of", "Compute the average of col")
.prql_function()
.with_tags({"prql"})
.with_parameter(help_text{"col", "The column to average"})
.with_example({
"To get the average of a",
@ -645,6 +654,7 @@ static readline_context::command_t sql_commands[] = {
"stats.count_by",
"Partition rows and count the number of rows in each partition")
.prql_function()
.with_tags({"prql"})
.with_parameter(help_text{"column", "The columns to group by"}
.one_or_more()
.with_grouping("{", "}"))
@ -662,6 +672,7 @@ static readline_context::command_t sql_commands[] = {
prql_cmd_sort,
help_text("stats.sum_of", "Compute the sum of col")
.prql_function()
.with_tags({"prql"})
.with_parameter(help_text{"col", "The column to sum"})
.with_example({
"To get the sum of a",
@ -677,6 +688,7 @@ static readline_context::command_t sql_commands[] = {
prql_cmd_sort,
help_text("stats.by", "A shorthand for grouping and aggregating")
.prql_function()
.with_tags({"prql"})
.with_parameter(help_text{"col", "The column to sum"})
.with_parameter(help_text{"values", "The aggregations to perform"})
.with_example({
@ -694,6 +706,7 @@ static readline_context::command_t sql_commands[] = {
prql_cmd_sort,
help_text("sort")
.prql_transform()
.with_tags({"prql"})
.with_summary("PRQL transform to sort rows")
.with_parameter(help_text{
"expr", "The values to use when ordering the result set"}
@ -712,6 +725,7 @@ static readline_context::command_t sql_commands[] = {
prql_cmd_take,
help_text("take")
.prql_transform()
.with_tags({"prql"})
.with_summary("PRQL command to pick rows based on their position")
.with_parameter({"n_or_range", "The number of rows or range"})
.with_example({
@ -734,6 +748,7 @@ static readline_context::command_t sql_commands[] = {
help_text("utils.distinct",
"A shorthand for getting distinct values of col")
.prql_function()
.with_tags({"prql"})
.with_parameter(help_text{"col", "The column to sum"})
.with_example({
"To get the distinct values of a",
@ -753,6 +768,9 @@ static auto bound_sql_cmd_map
sql_cmd_map_tag>::to_instance(+[]() {
for (auto& cmd : sql_commands) {
sql_cmd_map[cmd.c_name] = &cmd;
if (cmd.c_help.ht_name) {
cmd.c_help.index_tags();
}
}
return &sql_cmd_map;

@ -931,6 +931,12 @@ static std::unordered_map<std::string, attr_line_t> EXAMPLE_RESULTS;
static void
execute_example(const help_text& ht)
{
static const std::set<std::string> IGNORED_NAMES = {"ATTACH"};
if (IGNORED_NAMES.count(ht.ht_name)) {
return;
}
auto& dls = lnav_data.ld_db_row_source;
auto& dos = lnav_data.ld_db_overlay;
auto& db_tc = lnav_data.ld_views[LNV_DB];

@ -59,7 +59,7 @@ been loaded, you can use the following options:
•  -c cmd  A command, query, or file to execute. The
first character determines the type of operation: a colon
( : ) is used for the built-in commands; a semi-colon (
 ; ) for SQL queries; and a pipe symbol ( | ) for
 ; ) for SQL/PRQL queries; and a pipe symbol ( | ) for
executing a file containing other commands. For example,
to open the file "foo.log" and go to the tenth line in
the file, you can do:
@ -469,7 +469,7 @@ scrollbar.
NOTE: You need to manually enable this feature by setting the LNAV_EXP
environment variable to "mouse". F2 toggles mouse support.
SQL Queries
Log Analysis
Lnav has support for performing SQL queries on log files using the
SQLite3 "virtual" table feature. For all supported log file types,
@ -575,6 +575,37 @@ example of a top ten query into the "/tmp/topten.db" file, you can do:
 FROM access_log GROUP BY cs_uri_stem ORDER BY total DESC 
 LIMIT 10; 
PRQL Support
The Pipelined Relational Query Language ]8;;https://prql-lang.org\(PRQL)]8;;\[1] is an alternative
database query language that compiles to SQL. The main advantage of
PRQL, in the context of lnav, is that it is easier to work with
interactively compared to SQL. For example, lnav can provide previews
of different stages of the pipeline and provide more accurate
tab-completions for the columns in the result set.
▌[1] - https://prql-lang.org
You can execute a PRQL query in the SQL prompt. A PRQL query starts
with the  from  keyword that specifies the table to use as a data
source. The next stage of a pipeline is started by entering a pipe
symbol ( | ) followed by a ]8;;https://prql-lang.org/book/reference/stdlib/transforms/index.html\PRQL transform]8;;\[1]. As you build the query
in the prompt, lnav will display any relevant help and preview for the
current and previous stages of the pipeline.
▌[1] - https://prql-lang.org/book/reference/stdlib/transforms/index.html
Using the top ten URLs query from earlier as an example, the PRQL
version would be as follows:
;from access_log | stats.count_by cs_uri_stem | take 10 
The first stage selects the data source, the web  access_log  table in
this case. The  stats.count_by  transform is a convenience provided
by lnav that groups by the given column, counts the rows in each
group, and sorts by count in descending order. The  take 10 turns
into the  LIMIT 10 .
Dynamic logline Table (experimental)
(NOTE: This feature is still very new and not completely reliable yet,
@ -719,9 +750,10 @@ For support questions, email:
Parameter
path The path to the file to append to
See Also
:echo, :export-session-to, :pipe-line-to, :pipe-to, :redirect-to,
:write-csv-to, :write-json-to, :write-jsonlines-to, :write-raw-to,
:write-screen-to, :write-table-to, :write-to, :write-view-to, echoln()
.dump, .read, :echo, :export-session-to, :pipe-line-to, :pipe-to,
:redirect-to, :write-csv-to, :write-json-to, :write-jsonlines-to,
:write-raw-to, :write-screen-to, :write-table-to, :write-to,
:write-view-to, echoln()
Example
#1 To append marked lines to the file /tmp/interesting-lines.txt:
:append-to /tmp/interesting-lines.txt 
@ -968,7 +1000,7 @@ For support questions, email:
-n Do not print a line-feed at the end of the output
msg The message to display
See Also
:alt-msg, :append-to, :cd, :eval, :export-session-to,
.dump, .read, :alt-msg, :append-to, :cd, :eval, :export-session-to,
:export-session-to, :pipe-line-to, :pipe-to, :rebuild, :redirect-to,
:redirect-to, :sh, :write-csv-to, :write-csv-to, :write-json-to,
:write-json-to, :write-jsonlines-to, :write-jsonlines-to,
@ -1026,12 +1058,12 @@ For support questions, email:
Parameter
path The path to the file to write
See Also
:alt-msg, :append-to, :cd, :echo, :echo, :eval, :pipe-line-to,
:pipe-to, :rebuild, :redirect-to, :redirect-to, :sh, :write-csv-to,
:write-csv-to, :write-json-to, :write-json-to, :write-jsonlines-to,
:write-jsonlines-to, :write-raw-to, :write-raw-to, :write-screen-to,
:write-screen-to, :write-table-to, :write-table-to, :write-to,
:write-to, :write-view-to, :write-view-to, echoln()
.dump, .read, :alt-msg, :append-to, :cd, :echo, :echo, :eval,
:pipe-line-to, :pipe-to, :rebuild, :redirect-to, :redirect-to, :sh,
:write-csv-to, :write-csv-to, :write-json-to, :write-json-to,
:write-jsonlines-to, :write-jsonlines-to, :write-raw-to, :write-raw-to,
:write-screen-to, :write-screen-to, :write-table-to, :write-table-to,
:write-to, :write-to, :write-view-to, :write-view-to, echoln()
:filter-expr expr
══════════════════════════════════════════════════════════════════════
@ -1296,9 +1328,10 @@ For support questions, email:
Parameter
shell-cmd The shell command-line to execute
See Also
:append-to, :echo, :export-session-to, :pipe-to, :redirect-to,
:write-csv-to, :write-json-to, :write-jsonlines-to, :write-raw-to,
:write-screen-to, :write-table-to, :write-to, :write-view-to, echoln()
.dump, .read, :append-to, :echo, :export-session-to, :pipe-to,
:redirect-to, :write-csv-to, :write-json-to, :write-jsonlines-to,
:write-raw-to, :write-screen-to, :write-table-to, :write-to,
:write-view-to, echoln()
Example
#1 To write the top line to 'sed' for processing:
:pipe-line-to sed -e 's/foo/bar/g' 
@ -1311,9 +1344,10 @@ For support questions, email:
Parameter
shell-cmd The shell command-line to execute
See Also
:append-to, :echo, :export-session-to, :pipe-line-to, :redirect-to,
:write-csv-to, :write-json-to, :write-jsonlines-to, :write-raw-to,
:write-screen-to, :write-table-to, :write-to, :write-view-to, echoln()
.dump, .read, :append-to, :echo, :export-session-to, :pipe-line-to,
:redirect-to, :write-csv-to, :write-json-to, :write-jsonlines-to,
:write-raw-to, :write-screen-to, :write-table-to, :write-to,
:write-view-to, echoln()
Example
#1 To write marked lines to 'sed' for processing:
:pipe-to sed -e s/foo/bar/g 
@ -1404,12 +1438,13 @@ For support questions, email:
path The path to the file to write. If not specified,
the current redirect will be cleared
See Also
:alt-msg, :append-to, :cd, :echo, :echo, :eval, :export-session-to,
:export-session-to, :pipe-line-to, :pipe-to, :rebuild, :sh,
:write-csv-to, :write-csv-to, :write-json-to, :write-json-to,
:write-jsonlines-to, :write-jsonlines-to, :write-raw-to, :write-raw-to,
:write-screen-to, :write-screen-to, :write-table-to, :write-table-to,
:write-to, :write-to, :write-view-to, :write-view-to, echoln()
.dump, .read, :alt-msg, :append-to, :cd, :echo, :echo, :eval,
:export-session-to, :export-session-to, :pipe-line-to, :pipe-to,
:rebuild, :sh, :write-csv-to, :write-csv-to, :write-json-to,
:write-json-to, :write-jsonlines-to, :write-jsonlines-to,
:write-raw-to, :write-raw-to, :write-screen-to, :write-screen-to,
:write-table-to, :write-table-to, :write-to, :write-to, :write-view-to,
:write-view-to, echoln()
Example
#1 To write the output of lnav commands to the file /tmp/script-output.txt:
:redirect-to /tmp/script-output.txt 
@ -1659,11 +1694,11 @@ For support questions, email:
--anonymize Anonymize the table contents
path The path to the file to write
See Also
:alt-msg, :append-to, :cd, :create-logline-table, :create-search-table,
:echo, :echo, :eval, :export-session-to, :export-session-to,
:pipe-line-to, :pipe-to, :rebuild, :redirect-to, :redirect-to, :sh,
:write-csv-to, :write-csv-to, :write-csv-to, :write-json-to,
:write-json-to, :write-json-to, :write-jsonlines-to,
.dump, .read, :alt-msg, :append-to, :cd, :create-logline-table,
:create-search-table, :echo, :echo, :eval, :export-session-to,
:export-session-to, :pipe-line-to, :pipe-to, :rebuild, :redirect-to,
:redirect-to, :sh, :write-csv-to, :write-csv-to, :write-csv-to,
:write-json-to, :write-json-to, :write-json-to, :write-jsonlines-to,
:write-jsonlines-to, :write-jsonlines-to, :write-raw-to, :write-raw-to,
:write-raw-to, :write-screen-to, :write-screen-to, :write-screen-to,
:write-to, :write-to, :write-view-to, :write-view-to, :write-view-to,
@ -1681,14 +1716,15 @@ For support questions, email:
--anonymize Anonymize the row contents
path The path to the file to write
See Also
:alt-msg, :append-to, :cd, :create-logline-table, :create-search-table,
:echo, :echo, :eval, :export-session-to, :export-session-to,
:pipe-line-to, :pipe-to, :rebuild, :redirect-to, :redirect-to, :sh,
:write-json-to, :write-json-to, :write-json-to, :write-jsonlines-to,
:write-jsonlines-to, :write-jsonlines-to, :write-raw-to, :write-raw-to,
:write-raw-to, :write-screen-to, :write-screen-to, :write-screen-to,
:write-table-to, :write-table-to, :write-table-to, :write-to,
:write-to, :write-view-to, :write-view-to, :write-view-to, echoln()
.dump, .read, :alt-msg, :append-to, :cd, :create-logline-table,
:create-search-table, :echo, :echo, :eval, :export-session-to,
:export-session-to, :pipe-line-to, :pipe-to, :rebuild, :redirect-to,
:redirect-to, :sh, :write-json-to, :write-json-to, :write-json-to,
:write-jsonlines-to, :write-jsonlines-to, :write-jsonlines-to,
:write-raw-to, :write-raw-to, :write-raw-to, :write-screen-to,
:write-screen-to, :write-screen-to, :write-table-to, :write-table-to,
:write-table-to, :write-to, :write-to, :write-view-to, :write-view-to,
:write-view-to, echoln()
Example
#1 To write SQL results as CSV to /tmp/table.csv:
:write-csv-to /tmp/table.csv 
@ -1702,14 +1738,15 @@ For support questions, email:
--anonymize Anonymize the JSON values
path The path to the file to write
See Also
:alt-msg, :append-to, :cd, :create-logline-table, :create-search-table,
:echo, :echo, :eval, :export-session-to, :export-session-to,
:pipe-line-to, :pipe-to, :rebuild, :redirect-to, :redirect-to, :sh,
:write-csv-to, :write-csv-to, :write-csv-to, :write-jsonlines-to,
:write-jsonlines-to, :write-jsonlines-to, :write-raw-to, :write-raw-to,
:write-raw-to, :write-screen-to, :write-screen-to, :write-screen-to,
:write-table-to, :write-table-to, :write-table-to, :write-to,
:write-to, :write-view-to, :write-view-to, :write-view-to, echoln()
.dump, .read, :alt-msg, :append-to, :cd, :create-logline-table,
:create-search-table, :echo, :echo, :eval, :export-session-to,
:export-session-to, :pipe-line-to, :pipe-to, :rebuild, :redirect-to,
:redirect-to, :sh, :write-csv-to, :write-csv-to, :write-csv-to,
:write-jsonlines-to, :write-jsonlines-to, :write-jsonlines-to,
:write-raw-to, :write-raw-to, :write-raw-to, :write-screen-to,
:write-screen-to, :write-screen-to, :write-table-to, :write-table-to,
:write-table-to, :write-to, :write-to, :write-view-to, :write-view-to,
:write-view-to, echoln()
Example
#1 To write SQL results as JSON to /tmp/table.json:
:write-json-to /tmp/table.json 
@ -1723,14 +1760,15 @@ For support questions, email:
--anonymize Anonymize the JSON values
path The path to the file to write
See Also
:alt-msg, :append-to, :cd, :create-logline-table, :create-search-table,
:echo, :echo, :eval, :export-session-to, :export-session-to,
:pipe-line-to, :pipe-to, :rebuild, :redirect-to, :redirect-to, :sh,
:write-csv-to, :write-csv-to, :write-csv-to, :write-json-to,
:write-json-to, :write-json-to, :write-raw-to, :write-raw-to,
:write-raw-to, :write-screen-to, :write-screen-to, :write-screen-to,
:write-table-to, :write-table-to, :write-table-to, :write-to,
:write-to, :write-view-to, :write-view-to, :write-view-to, echoln()
.dump, .read, :alt-msg, :append-to, :cd, :create-logline-table,
:create-search-table, :echo, :echo, :eval, :export-session-to,
:export-session-to, :pipe-line-to, :pipe-to, :rebuild, :redirect-to,
:redirect-to, :sh, :write-csv-to, :write-csv-to, :write-csv-to,
:write-json-to, :write-json-to, :write-json-to, :write-raw-to,
:write-raw-to, :write-raw-to, :write-screen-to, :write-screen-to,
:write-screen-to, :write-table-to, :write-table-to, :write-table-to,
:write-to, :write-to, :write-view-to, :write-view-to, :write-view-to,
echoln()
Example
#1 To write SQL results as JSON Lines to /tmp/table.json:
:write-jsonlines-to /tmp/table.json 
@ -1748,11 +1786,11 @@ For support questions, email:
--anonymize Anonymize the lines
path The path to the file to write
See Also
:alt-msg, :append-to, :cd, :create-logline-table, :create-search-table,
:echo, :echo, :eval, :export-session-to, :export-session-to,
:pipe-line-to, :pipe-to, :rebuild, :redirect-to, :redirect-to, :sh,
:write-csv-to, :write-csv-to, :write-csv-to, :write-json-to,
:write-json-to, :write-json-to, :write-jsonlines-to,
.dump, .read, :alt-msg, :append-to, :cd, :create-logline-table,
:create-search-table, :echo, :echo, :eval, :export-session-to,
:export-session-to, :pipe-line-to, :pipe-to, :rebuild, :redirect-to,
:redirect-to, :sh, :write-csv-to, :write-csv-to, :write-csv-to,
:write-json-to, :write-json-to, :write-json-to, :write-jsonlines-to,
:write-jsonlines-to, :write-jsonlines-to, :write-screen-to,
:write-screen-to, :write-screen-to, :write-table-to, :write-table-to,
:write-table-to, :write-to, :write-to, :write-view-to, :write-view-to,
@ -1771,11 +1809,11 @@ For support questions, email:
--anonymize Anonymize the lines
path The path to the file to write
See Also
:alt-msg, :append-to, :cd, :create-logline-table, :create-search-table,
:echo, :echo, :eval, :export-session-to, :export-session-to,
:pipe-line-to, :pipe-to, :rebuild, :redirect-to, :redirect-to, :sh,
:write-csv-to, :write-csv-to, :write-csv-to, :write-json-to,
:write-json-to, :write-json-to, :write-jsonlines-to,
.dump, .read, :alt-msg, :append-to, :cd, :create-logline-table,
:create-search-table, :echo, :echo, :eval, :export-session-to,
:export-session-to, :pipe-line-to, :pipe-to, :rebuild, :redirect-to,
:redirect-to, :sh, :write-csv-to, :write-csv-to, :write-csv-to,
:write-json-to, :write-json-to, :write-json-to, :write-jsonlines-to,
:write-jsonlines-to, :write-jsonlines-to, :write-raw-to, :write-raw-to,
:write-raw-to, :write-table-to, :write-table-to, :write-table-to,
:write-to, :write-to, :write-view-to, :write-view-to, :write-view-to,
@ -1793,11 +1831,11 @@ For support questions, email:
--anonymize Anonymize the table contents
path The path to the file to write
See Also
:alt-msg, :append-to, :cd, :create-logline-table, :create-search-table,
:echo, :echo, :eval, :export-session-to, :export-session-to,
:pipe-line-to, :pipe-to, :rebuild, :redirect-to, :redirect-to, :sh,
:write-csv-to, :write-csv-to, :write-csv-to, :write-json-to,
:write-json-to, :write-json-to, :write-jsonlines-to,
.dump, .read, :alt-msg, :append-to, :cd, :create-logline-table,
:create-search-table, :echo, :echo, :eval, :export-session-to,
:export-session-to, :pipe-line-to, :pipe-to, :rebuild, :redirect-to,
:redirect-to, :sh, :write-csv-to, :write-csv-to, :write-csv-to,
:write-json-to, :write-json-to, :write-json-to, :write-jsonlines-to,
:write-jsonlines-to, :write-jsonlines-to, :write-raw-to, :write-raw-to,
:write-raw-to, :write-screen-to, :write-screen-to, :write-screen-to,
:write-to, :write-to, :write-view-to, :write-view-to, :write-view-to,
@ -1815,13 +1853,13 @@ For support questions, email:
--anonymize Anonymize the lines
path The path to the file to write
See Also
:alt-msg, :append-to, :cd, :echo, :echo, :eval, :export-session-to,
:export-session-to, :pipe-line-to, :pipe-to, :rebuild, :redirect-to,
:redirect-to, :sh, :write-csv-to, :write-csv-to, :write-json-to,
:write-json-to, :write-jsonlines-to, :write-jsonlines-to,
:write-raw-to, :write-raw-to, :write-screen-to, :write-screen-to,
:write-table-to, :write-table-to, :write-view-to, :write-view-to,
echoln()
.dump, .read, :alt-msg, :append-to, :cd, :echo, :echo, :eval,
:export-session-to, :export-session-to, :pipe-line-to, :pipe-to,
:rebuild, :redirect-to, :redirect-to, :sh, :write-csv-to,
:write-csv-to, :write-json-to, :write-json-to, :write-jsonlines-to,
:write-jsonlines-to, :write-raw-to, :write-raw-to, :write-screen-to,
:write-screen-to, :write-table-to, :write-table-to, :write-view-to,
:write-view-to, echoln()
Example
#1 To write marked lines to the file /tmp/interesting-lines.txt:
:write-to /tmp/interesting-lines.txt 
@ -1836,11 +1874,11 @@ For support questions, email:
--anonymize Anonymize the lines
path The path to the file to write
See Also
:alt-msg, :append-to, :cd, :create-logline-table, :create-search-table,
:echo, :echo, :eval, :export-session-to, :export-session-to,
:pipe-line-to, :pipe-to, :rebuild, :redirect-to, :redirect-to, :sh,
:write-csv-to, :write-csv-to, :write-csv-to, :write-json-to,
:write-json-to, :write-json-to, :write-jsonlines-to,
.dump, .read, :alt-msg, :append-to, :cd, :create-logline-table,
:create-search-table, :echo, :echo, :eval, :export-session-to,
:export-session-to, :pipe-line-to, :pipe-to, :rebuild, :redirect-to,
:redirect-to, :sh, :write-csv-to, :write-csv-to, :write-csv-to,
:write-json-to, :write-json-to, :write-json-to, :write-jsonlines-to,
:write-jsonlines-to, :write-jsonlines-to, :write-raw-to, :write-raw-to,
:write-raw-to, :write-screen-to, :write-screen-to, :write-screen-to,
:write-table-to, :write-table-to, :write-table-to, :write-to,
@ -2349,10 +2387,10 @@ For support questions, email:
Parameter
value The value to write to the current output file
See Also
:append-to, :echo, :export-session-to, :pipe-line-to, :pipe-to,
:redirect-to, :write-csv-to, :write-json-to, :write-jsonlines-to,
:write-raw-to, :write-screen-to, :write-table-to, :write-to,
:write-view-to
.dump, .read, :append-to, :echo, :export-session-to, :pipe-line-to,
:pipe-to, :redirect-to, :write-csv-to, :write-json-to,
:write-jsonlines-to, :write-raw-to, :write-screen-to, :write-table-to,
:write-to, :write-view-to
encode(value, algorithm)
══════════════════════════════════════════════════════════════════════

Loading…
Cancel
Save