97 Commits (42056760d00d45a3b0cfe39ebf93a98c1d211630)

Author	SHA1	Message	Date
Tim Stack	5787f47767	[piper] replace piper_proc with a thread ... Related to #1029	1 year ago
Tim Stack	e0ff4434a9	[logfile] report utf error location ... Related to #1156	1 year ago
Tim Stack	5db31300c9	[logfile] avoid reading full log messages that are bigger than the line_buffer max size	1 year ago
Tim Stack	6f0f66a418	[json] fix json log handling with multiple lines in a field	1 year ago
Tim Stack	dd21bdfd82	[text_filter] scrub ansi when testing a filter ... Related to #1163	1 year ago
Tim Stack	faeaf477ab	[yajlpp] flesh things out a bit more	2 years ago
Tim Stack	7f513b2d1f	[tidy] fixing some uninit issues	2 years ago
Tim Stack	d940d9d5eb	[logfile] plumb gzip header through file meta	2 years ago
Tim Stack	e0ecbdff10	[logfile] improve handling of ansi escapes ... I think this covers the rest of the functionality that needs to deal with ansi escapes. Related to #1057	2 years ago
Tim Stack	740b827901	[logfile] handle logs with ANSI escapes ... This is a partial fix for handling ANSI escapes in parts of log messages that would prevent regexes from matching. Still more work to do. Related to #1057	2 years ago
Tim Stack	66ec8e553e	[build] fix static var issue	2 years ago
Tim Stack	6ca01cd934	[line_buffer] fix some long-line handling	2 years ago
Tim Stack	3df07139b3	[line_buffer] fix long line issue ... Fixes #980	2 years ago
Tim Stack	7077495389	[crash] missed expanding the alt buffer in line_buffer	2 years ago
Tim Stack	7b32ea9d39	[logfile] fix handling of partial lines	2 years ago
Tim Stack	1ae1600f7b	[line_buffer] cache compressed files	2 years ago
Timothy Stack	8ef581177c	[line_buffer] add some performance counters for tracking SQL perf ... Also, check for keyboard input during SQL execution so we can cancel. Fixes #894	2 years ago
Timothy Stack	b3ee092dc5	[cleanup] remove unused stuff and add a test	2 years ago
Timothy Stack	4c20b0039c	[sql] some more error improvements	2 years ago
Timothy Stack	39763b2d31	[vtab] add indexes for text columns	2 years ago
Timothy Stack	f3934c742d	[error] leverage sqlite3_error_offset	2 years ago
Timothy Stack	8b3b8a6e3c	[perf] fix some text file loading issues	2 years ago
Timothy Stack	01031a9fa3	[build] add missing include	2 years ago
Timothy Stack	36e9433891	[perf] try to overlap line buffer i/o	2 years ago
Timothy Stack	f9f797fc9d	[logfile_sub_source] add log message watch expressions ... Fixes #539	2 years ago
Timothy Stack	9d590836f3	[perf] tweak initial line buffer request size	2 years ago
Timothy Stack	a27198e8ca	[regex101] add an integration with regex101 ... ... and a pile of other changes	2 years ago
Timothy Stack	81e63784e7	[modernize] some more cleanups	3 years ago
Timothy Stack	b856cd9657	[build] silence some warnings	3 years ago
Timothy Stack	66ef5fdae1	[clang-format] init	3 years ago
Timothy Stack	f5e88b7158	[docs] add ARCHITECTURE.md and various other things	4 years ago
Timothy Stack	9a073e80e7	[result] fix Result.map() issue with copying the value when the result was in error	4 years ago
Phil Hord	7ad40f7263	Improved fix for multi-stream gz files ... Instead of throwing an error when we unexpectedly reach a stream EOS, treat it as regular stream end. This allows for streams that might different encodings for different sections. Even though we don't recognize the encoding and continue on with the data, at least we don't fail when we reach this situation. This allows us safely to try and continue parsing the next catted gz stream, knowing that if it fails, we will handle it gracefully.	4 years ago
Phil Hord	f10a303f54	Revert "Don't try to read concatenated gzip streams" ... This reverts commit 1d127053e9.	4 years ago
Timothy Stack	8e629b166a	[build] some more compiler profiling ... Defect Number: Reviewed By: Testing Done:	4 years ago
Timothy Stack	db8a3c4d38	[debt] doing some cleanup ... Defect Number: Reviewed By: Testing Done:	4 years ago
Phil Hord	1d127053e9	Don't try to read concatenated gzip streams ... Don't try to continue reading the next stream of a concatenated gzip file. The next stream may be CRC noise or other garbage. Maybe in the future we should look for a gzip header in the following bytes of the stream and try to decode from there. But it's not clear that anyone ever uses this supposed gzip feature anyway. Let's just end the stream when we reach EOS. Also, if the stream fails to init, let's leave it closed instead of throwing an error no one is likely to catch. Log the error msg from zlib if one is provided.	4 years ago
Suresh Sundriyal	0dedb3d455	[simd] Allow toggling of simd related functionality	4 years ago
Timothy Stack	ce0383dee1	[build] fix some warnings	4 years ago
Timothy Stack	16c5fdccb4	[build] more win build	4 years ago
Timothy Stack	ead1d3eb36	[build] fix some more warnings	4 years ago
Timothy Stack	138a506b1a	[perf] a few performance tweaks	4 years ago
Phil Hord	5513deeade	Add a gzip indexing class for faster gz navigation ... The gzread function is slow. Every time you seek to a new location, the whole file up to that position has to be decompressed again. This causes massive lags when trying to do simple things in lnav on a large .gz file. Use the zlib inflate* functions instead and record the dictionary periodically while processing the file the first time. Then use inflateSetDictionary to restore the dictionary to a convenient location when trying to seek into the file again in the future. Use a default period of 1MB of compressed data for syncpoints. Each syncpoint uses 32KB. This is a ratio of 3.2%. For example, a 1GB .gz file (compressed size) will require us to keep 32MB of index data in memory. A better method may be to use a fixed number of syncpoints and divide the file appropriately. This would keep the memory bounded at the cost of slower file navigation on large .gz files. Use pread to read the data for the stream decompressor and remove the lock_hack previously employed. NB. The documentation on these zlib functions is sparse. I followed the example in zlib/examples/zran.c, but I used the z_stream total_in and total_out variables instead of keeping my own separately as zran.c does. Maybe this is incompatible with some very old zlib versions. I haven't looked.	5 years ago
Timothy Stack	76edd2a9d1	[line_buffer] refactor left a bug with non-blocking files ... Fixes #662	5 years ago
Timothy Stack	02c53976b4	Some refactoring for the line buffer and other cleanups ... Fixes #649	5 years ago
Timothy Stack	84135c44e4	[build] disable simd stuff for now	6 years ago
Timothy Stack	2589345e5c	[perf] improve initial indexing times	6 years ago
Timothy Stack	2e10ca09d0	[line_buffer] fix crlf handling in log files	6 years ago
Timothy Stack	3e457e30ca	[json-format] fix json file recognition for crlf line-endings ... Fixes #528	6 years ago
Timothy Stack	d805dee15f	[line_buffer] avoid a crash with binary files, but still not handled well	6 years ago