Commit Graph

739 Commits

Author SHA1 Message Date
Timothy Stack
b184272edb [file_collection] fix recursion
Fixes #877
2021-07-03 21:57:50 -07:00
Timothy Stack
dbcfdda363 [reltime] some more relative time fixes 2021-06-18 21:58:30 -07:00
Timothy Stack
aa7ac37cdc [log_source] try to report sql filter errors 2021-06-18 09:04:53 -07:00
Timothy Stack
a760a4240f [file_collection] handle dirs with colons
Fixes #863
2021-06-08 10:57:11 -07:00
Timothy Stack
b2d296b5d3 [open] handle file names with colons
Fixes #863
2021-06-07 16:04:41 -07:00
Timothy Stack
408a2ea38e [tests] fix for busybox 2021-06-07 08:59:24 -07:00
Timothy Stack
d5d44e63b3 [build] fix cmake build 2021-06-06 21:52:06 -07:00
Timothy Stack
d3d527d290 [tests] adding some more coverage 2021-06-02 21:09:50 -07:00
Timothy Stack
b5406f2cee [tests] some more coverage 2021-06-02 08:59:53 -07:00
Timothy Stack
75a01d0f50 [test] tweak test_remote.sh config 2021-06-01 23:17:09 -07:00
Timothy Stack
9c417afb2d [logfile] remove error exception and some optimizations 2021-05-28 22:53:50 -07:00
Timothy Stack
0dca44f487 [rebuild] add a deadline to make the UI more responsive 2021-05-26 22:09:47 -07:00
Timothy Stack
b80e322162 [test] fix test_logfile.sh 2021-05-25 23:16:00 -07:00
Timothy Stack
1737964df6 [remote] add some tailer tests 2021-05-25 22:27:09 -07:00
Timothy Stack
edfd835866 [remote] use the APE version of the tailer 2021-05-21 21:39:36 -07:00
Timothy Stack
b78c914d6f [remote] a path is synced on an error 2021-05-20 15:48:05 -07:00
Timothy Stack
9a91cdffbb [test] scrub error message 2021-05-20 09:27:26 -07:00
Timothy Stack
12cf00f4dc [tests] set config to use for remote test 2021-05-20 09:12:51 -07:00
Timothy Stack
6ebd9f16a2 [remote] add a test 2021-05-19 22:44:32 -07:00
Timothy Stack
c3dc668b69 [remote] add some config options and remove the copied tailer binary
Also fix time offset issue
2021-05-19 22:05:21 -07:00
Timothy Stack
be51a4e3de [arc/remote] use a single work directory for archive/remote files 2021-05-15 22:39:18 -07:00
Timothy Stack
e13816a8e3 [cmds] change :write-raw-to to copy the original log content and add :write-view-to command
Also added log_raw_text column to the log tables.

Fixes #859
2021-05-14 13:31:19 -07:00
Timothy Stack
b64342ca47 [log_format] display log_body for the body field instead of the format name and add :mark-expr command 2021-05-13 22:00:26 -07:00
Timothy Stack
d2b5690e0c [readline] improve filename completion 2021-05-13 19:50:04 -07:00
Timothy Stack
cc1e79d1cc [init] fix order of init for views so they can be manipulated by SQL files in format dirs
Fixes #853
2021-04-17 13:58:53 -07:00
Timothy Stack
3741985ee3 [format] allow leading slash in field names
Fixes #852
2021-04-16 14:47:47 -07:00
Timothy Stack
70c07da5c5 [sql] fix help for sparkline() 2021-04-04 21:54:54 -07:00
Timothy Stack
b7817d4bb9 [build] fix musl test build 2021-04-03 22:30:45 -07:00
Timothy Stack
d31aa845f5 [base] refactor duration2str 2021-04-03 14:24:24 -07:00
Timothy Stack
0658b9ef57 [tests] move some test code around 2021-04-01 21:11:20 -07:00
Timothy Stack
e593bde875 [build] fix some issues on xenial32 2021-03-30 22:50:10 -07:00
Timothy Stack
2e4106b80a [build] more test/dist fixes 2021-03-29 09:35:11 -07:00
Timothy Stack
bfa1877f4e [tests] try to fix test build 2021-03-28 22:42:51 -07:00
Timothy Stack
def50ee6cf [build] try fix 2021-03-28 14:53:18 -07:00
Timothy Stack
c0ed59e61e [sqlite] implement .dump SQL command 2021-03-28 13:50:26 -07:00
Timothy Stack
6f4791ec35 [sqlite] include generate_series() extension 2021-03-23 22:23:08 -07:00
Timothy Stack
92e20ffd51 [reltime] add support for weekdays and having timeslice() return NULL when a time is out-of-range 2021-03-21 08:53:21 -07:00
Timothy Stack
4f4d90a65d [tests] include more information in error message 2021-03-20 13:39:35 -07:00
Timothy Stack
5d4f08b1b4 [scripty] add a timeout 2021-03-19 23:12:33 -07:00
Timothy Stack
1b16376076 [build] other build fix stuff 2021-03-19 21:35:53 -07:00
Timothy Stack
3c10b1f34b [build] try to fix some tests 2021-03-19 21:29:18 -07:00
Timothy Stack
749fdacfa8 [build] fixes 2021-03-19 15:43:03 -07:00
Timothy Stack
7b0e1c4f54 [filters] fix previews when created in the editor 2021-03-18 15:59:39 -07:00
Timothy Stack
def2fe4bcb [test] try to fix test 2021-03-07 14:59:22 -08:00
Timothy Stack
f6128240ab [sql] add lnav_top_file() SQL function 2021-03-07 14:29:22 -08:00
Timothy Stack
68759ada2d [data_scanner] fix quoted string regex
Fixes #842
2021-03-02 15:19:08 -08:00
Timothy Stack
c177a321fb [leak] try to fix leak in a test 2021-02-25 22:22:16 -08:00
Timothy Stack
d45d7d6c58 [docs] add report generation to cookbook 2021-02-25 15:47:36 -08:00
Timothy Stack
f5e88b7158 [docs] add ARCHITECTURE.md and various other things 2021-02-13 12:41:48 -08:00
Timothy Stack
a4368223ea some docs and test fixup 2021-02-06 23:08:01 -08:00
Timothy Stack
351ad7a8fc [xml/xpath] suggest xpath() calls for xml fields 2021-02-06 22:13:08 -08:00
Timothy Stack
6deb7032a6 [tests] add more CSIs to scripty 2021-02-05 16:04:34 -08:00
Tim Stack
0269aa1d47
Delete tui_echo.0 2021-02-04 23:28:27 -08:00
Timothy Stack
2750d23b07 [tests] add more TUI tests 2021-02-04 23:10:44 -08:00
Timothy Stack
58f1c9df1c [tests] fix tui test 2021-02-02 22:12:06 -08:00
Timothy Stack
0a701394fe [tests] add an initial TUI test 2021-02-02 21:58:42 -08:00
Timothy Stack
37523fe7d0 [test] test SQL state funcs 2021-02-01 22:51:02 -08:00
Timothy Stack
3b5a5b448a [tests] test old date 2021-02-01 22:36:31 -08:00
Timothy Stack
7e6b3dcc23 [scripty] fix handling of 'C' escape sequence 2021-02-01 21:59:46 -08:00
Timothy Stack
b847ae7164 [tests] more coverage 2021-02-01 15:56:37 -08:00
Timothy Stack
7821d07b7c [tests] coverage for relative_time.hh 2021-01-31 23:20:38 -08:00
Timothy Stack
85b9329332 [tests] more coverage stuff 2021-01-31 22:43:19 -08:00
Timothy Stack
f35c0c397e [tests] increasing coverage a bit 2021-01-31 21:59:48 -08:00
Timothy Stack
caa5cc9003 [tests] add realpath() test 2021-01-31 15:01:11 -08:00
Timothy Stack
5c02766485 [tests] add test for realpath() 2021-01-31 14:41:24 -08:00
Timothy Stack
47fb48eaf3 [tests] add SQL readlink() test 2021-01-31 14:19:14 -08:00
Timothy Stack
7778042847 [tests] set TZ for test 2021-01-30 22:50:57 -08:00
Timothy Stack
c5ab97352d [cover] add some tests for :unix-time 2021-01-30 22:24:11 -08:00
Timothy Stack
efea67e61c [tests] move test_pcrepp 2021-01-29 22:20:42 -08:00
Timothy Stack
00890970cd [tests] disable a couple of tests 2021-01-29 21:03:42 -08:00
Timothy Stack
49ebde5953 [build] remove pcrecpp and other stuff
Linking against a binary c++ lib suuuucckkkkkssss...
2021-01-29 11:12:02 -08:00
Timothy Stack
cac1175973 [services] initial pass at injecting services and a bunch of other stuff 2021-01-23 13:00:51 -08:00
Timothy Stack
56bee6f4c9 [archive] remove old unpacked archives and a bunch of other stuff 2021-01-16 22:23:20 -08:00
Timothy Stack
8851c4492f [tests] add a test for decompression 2021-01-10 22:04:55 -08:00
Timothy Stack
6be36eda19 [build] isdigit() instead of isnumber() 2021-01-08 20:48:54 -08:00
Timothy Stack
8235f9af5d [tests] try to make scripty output readable 2021-01-08 14:08:52 -08:00
Timothy Stack
4d936f7bba [build] add more tests to cmake build 2021-01-01 20:40:52 -08:00
Timothy Stack
cc072d29ea [tests] update another date-sensitive test 2020-12-31 23:26:57 -08:00
Timothy Stack
60dde499ac [tests] fix test to make it insensitive to the current date 2020-12-31 23:23:56 -08:00
Timothy Stack
1cefe583fa [debt] fix a leak in lnav_view_filters 2020-12-31 22:55:34 -08:00
Timothy Stack
2996a90afa [format] use the module level for a message 2020-12-31 13:55:53 -08:00
Timothy Stack
63c31838eb [debt] fixing some more debt 2020-12-28 22:31:11 -08:00
Timothy Stack
7476dd5bb9 [misc] misc bugs 2020-12-24 22:29:38 -08:00
Timothy Stack
4ca6fd7bfd [sqlite] add an xml/xpath extension 2020-12-23 15:01:21 -08:00
Timothy Stack
4ff2b710d5 [log_format] change w3c_log to put extra columns into a JSON column 2020-12-18 22:35:23 -08:00
Timothy Stack
941504a4cb [help] fix word wrap for example desc 2020-12-13 21:11:07 -08:00
Timothy Stack
e6df0fba0a [humanize] something seems wrong 2020-12-12 22:33:58 -08:00
Timothy Stack
04da95eb95 [installer] fix file type check for formats
Fixes #801
2020-12-10 21:18:56 -08:00
Timothy Stack
8a2fb80888 [build] fix distcheck 2020-12-08 09:29:52 -08:00
Timothy Stack
267e32d479 [tests] update expected output 2020-12-08 09:11:39 -08:00
Timothy Stack
e3f28eaa43 [logfile] a captured file cannot be overwritten
Fixes #741
2020-12-07 15:21:33 -08:00
Timothy Stack
623da1fe33 [build] add w3c logfiles to dist 2020-12-05 23:33:17 -08:00
Timothy Stack
32f112b8de [formats] add w3c extended log format impl
Fiexes #798
2020-12-05 21:51:46 -08:00
Timothy Stack
19e0624ab6 [debt] even more mem leaks
Defect Number:
    Reviewed By:
   Testing Done:
2020-12-01 11:27:03 -08:00
Timothy Stack
22c2e95df0 [filters] sql filter
Related to #568
2020-11-29 13:20:07 -08:00
Timothy Stack
6d0054d3b6 [logfile] refactor visibility 2020-11-25 14:47:39 -08:00
Timothy Stack
2aad7985a6 [build] opt build
Defect Number:
    Reviewed By:
   Testing Done:
2020-11-25 14:46:39 -08:00
Timothy Stack
8e629b166a [build] some more compiler profiling
Defect Number:
    Reviewed By:
   Testing Done:
2020-11-19 21:36:51 -08:00
Timothy Stack
51d1204a96 [arc] add content hash to archive temp dir and tech debt 2020-11-17 10:04:23 -08:00
Timothy Stack
0765c07010 [view] migrate view_action stuff to function objects 2020-11-14 21:39:12 -08:00
Timothy Stack
f192cb7c3e [input_dispatcher] fix handling of unicode input
Fixes #791
2020-11-09 22:18:17 -08:00
Timothy Stack
db8a3c4d38 [debt] doing some cleanup
Defect Number:
    Reviewed By:
   Testing Done:
2020-11-09 22:17:17 -08:00
Timothy Stack
e4ea9ca584 [arc] report errors when opening archives 2020-10-31 22:19:41 -07:00
Timothy Stack
8b38bb0970 [session] store file visibility 2020-10-28 21:23:25 -07:00
Timothy Stack
c22c78220c [text] hide text files 2020-10-28 21:23:16 -07:00
Timothy Stack
7b77a612c2 [files] make file scanning async 2020-10-28 21:22:56 -07:00
Timothy Stack
dfd18a4be5 [view] add files view 2020-10-28 21:21:57 -07:00
Timothy Stack
3b0168025d [cmds] add support for hiding files 2020-10-28 21:18:57 -07:00
Timothy Stack
579e4274f9 [files] initial impl for opening archives 2020-10-28 21:17:57 -07:00
Timothy Stack
9f524136bd [fs] remove old filesystem impl 2020-10-20 22:55:46 -07:00
Timothy Stack
d8a083f0fe [filters] add :toggle-filtering command 2020-10-02 21:17:52 -07:00
Timothy Stack
74bfa21050 [build] missed a file 2020-09-24 13:22:42 -07:00
Timothy Stack
65c991919b [loader] fix format loader and add $schema refs to formats 2020-09-24 13:12:43 -07:00
Suresh Sundriyal
63ed3ee677 [dist] Prevent distributing built sources 2020-09-22 13:50:37 -07:00
Timothy Stack
a5171058c7 [build] some built files were being included in the dist 2020-09-22 12:01:13 -07:00
Suresh Sundriyal
f35f855458 [tests] Remove use of deprecated std::random_shuffle 2020-09-19 15:10:28 -07:00
Timothy Stack
ce0383dee1 [build] fix some warnings 2020-09-11 23:10:11 -07:00
Timothy Stack
349991a695 [shlex] fix handling of tilde 2020-09-10 21:19:23 -07:00
Timothy Stack
815a05577e [keymap] suggest the config option to set for an unknown keystroke
Also, add a unicode green-check/red-X mark for the success/error
message for commands.
2020-09-05 14:41:05 -07:00
Timothy Stack
ba1bc28209 [build] fix issues with test files 2020-09-03 22:13:21 -07:00
Timothy Stack
82b34bb02a [config] remove :save-config 2020-09-03 14:09:35 -07:00
Timothy Stack
84a7079e3f [main] capture stderr only after initscr() and fix curl configure detection 2020-09-02 15:39:27 -07:00
Timothy Stack
a2b3427502 [tests] add tests for SQL annotation functionality 2020-08-26 22:44:12 -07:00
Timothy Stack
d695febe14 [cmd] add :write-jsonlines-to command 2020-08-21 23:18:51 -07:00
Timothy Stack
d040e75d42 [build] fix a test 2020-06-28 22:56:24 -07:00
Timothy Stack
07d5aba7b5 [build] remove test files 2020-05-19 22:23:16 -07:00
Timothy Stack
ebd1038057 [pretty] include linefeeds in the pretty output
Defect Number:
    Reviewed By:
   Testing Done:
2020-05-19 22:13:52 -07:00
Timothy Stack
e85fff0b88 [sql] add json_concat() function and misc fixes 2020-05-17 22:23:30 -07:00
Timothy Stack
9cbe4d73a9 [docs] start to flesh out command and sql docs 2020-05-15 07:38:40 -07:00
Timothy Stack
defb00303f [docs] some changes 2020-05-13 09:38:41 -07:00
Timothy Stack
6073464722 [build] disable test for now 2020-05-09 21:48:37 -07:00
Timothy Stack
a8e97a180b [build] remove stale dep 2020-05-09 13:48:06 -07:00
Timothy Stack
8413dd13d8 [config] add configs directory 2020-05-07 07:08:59 -07:00
Timothy Stack
99a8eb9850 [build] try again 2020-04-25 17:22:59 -07:00
Timothy Stack
7e538c85f9 [tests] try to fix breakage 2020-04-25 13:30:28 -07:00
Timothy Stack
3da3ec799a [session] allow the name 2020-04-25 07:32:05 -07:00
Timothy Stack
138a506b1a [perf] a few performance tweaks 2020-04-23 06:42:48 -07:00
Timothy Stack
0744a9b6de [docs] mention the view tables
Defect Number:
    Reviewed By:
   Testing Done:
2020-04-21 21:04:03 -07:00
Timothy Stack
07f4b56646 [tests] update some expected results
so lazy...
2020-01-17 05:43:15 -08:00
Timothy Stack
e525d247ae [json-log] some more fixes for handling invalid json logs 2019-10-19 07:11:47 -07:00
Timothy Stack
8b750c83f1 update expected output 2019-09-11 08:12:56 -07:00
Timothy Stack
98519b045a [help] mention hide/show-unmarked-lines
Fixes #631
2019-09-10 05:50:10 -07:00
Timothy Stack
b7255419e6 [test] fix sed usage
Fixes #572
2019-09-10 05:39:34 -07:00
Timothy Stack
9d6e6e4638 [build] fix a typo 2019-09-09 22:10:50 -07:00
Timothy Stack
27587ee917 [test] get test working on OS X 2019-09-09 21:47:41 -07:00
Tim Stack
b2463e44a3
Merge pull request #677 from phord/gzip-index
Add a gzip indexing class for faster gz navigation
2019-09-09 21:35:44 -07:00
Timothy Stack
f11364660d [build] missed file in makefile 2019-09-03 07:27:24 -07:00
Timothy Stack
73e902f7d8 [json-log] handle invalid json log lines
Header from folded patch 'foldme1.patch':

[local] foldme
2019-09-03 06:19:57 -07:00
Phil Hord
5513deeade Add a gzip indexing class for faster gz navigation
The gzread function is slow.  Every time you seek to a new location, the
whole file up to that position has to be decompressed again.  This causes
massive lags when trying to do simple things in lnav on a large .gz file.

Use the zlib inflate* functions instead and record the dictionary
periodically while processing the file the first time.  Then use
inflateSetDictionary to restore the dictionary to a convenient
location when trying to seek into the file again in the future.

Use a default period of 1MB of compressed data for syncpoints.
Each syncpoint uses 32KB. This is a ratio of 3.2%.  For example,
a 1GB .gz file (compressed size) will require us to keep 32MB
of index data in memory. A better method may be to use a fixed
number of syncpoints and divide the file appropriately. This
would keep the memory bounded at the cost of slower file
navigation on large .gz files.

Use pread to read the data for the stream decompressor and remove
the lock_hack previously employed.

NB. The documentation on these zlib functions is sparse. I followed
the example in zlib/examples/zran.c, but I used the z_stream total_in
and total_out variables instead of keeping my own separately as zran.c
does.  Maybe this is incompatible with some very old zlib versions.
I haven't looked.
2019-08-17 18:40:55 -07:00
Timothy Stack
585986057d [build] missed cleaning a dir up 2019-08-10 21:59:43 -07:00