Don't try to continue reading the next stream of a concatenated
gzip file. The next stream may be CRC noise or other garbage.
Maybe in the future we should look for a gzip header in the
following bytes of the stream and try to decode from there.
But it's not clear that anyone ever uses this supposed gzip
feature anyway.
Let's just end the stream when we reach EOS. Also, if the
stream fails to init, let's leave it closed instead of throwing
an error no one is likely to catch. Log the error msg from
zlib if one is provided.
The gzread function is slow. Every time you seek to a new location, the
whole file up to that position has to be decompressed again. This causes
massive lags when trying to do simple things in lnav on a large .gz file.
Use the zlib inflate* functions instead and record the dictionary
periodically while processing the file the first time. Then use
inflateSetDictionary to restore the dictionary to a convenient
location when trying to seek into the file again in the future.
Use a default period of 1MB of compressed data for syncpoints.
Each syncpoint uses 32KB. This is a ratio of 3.2%. For example,
a 1GB .gz file (compressed size) will require us to keep 32MB
of index data in memory. A better method may be to use a fixed
number of syncpoints and divide the file appropriately. This
would keep the memory bounded at the cost of slower file
navigation on large .gz files.
Use pread to read the data for the stream decompressor and remove
the lock_hack previously employed.
NB. The documentation on these zlib functions is sparse. I followed
the example in zlib/examples/zran.c, but I used the z_stream total_in
and total_out variables instead of keeping my own separately as zran.c
does. Maybe this is incompatible with some very old zlib versions.
I haven't looked.
Remove exception specifications to appease modern compilers and reduce
warning counts.
I sort of hate to remove all these exception-decls, but they are in fact
deprecated, and my compiler loves to complain.
* grep_proc.cc: When a request is queued with the start line
== -1, we need to start searching from the highest line
ever seen and not the last line processed.
* line_buffer.cc: If a partial line was read, we need to
avoid returning another line if more data is appended
to the file.
* lnav.cc: Accept file name patterns on the command-line that
don't match any files yet. Initialize the screen before
redirecting stderr to the log file or /dev/null.
* log_format.hh: Add some comments. Start to add back support
for scrubbing.
* logfile_sub_source.cc: Move scrubbing to the format impl.
* textview_curses.hh: Add comments.
The cygwin build is pretty straightforward, but there's
still some glitches that need to be fixed.
* line_buffer.hh, line_buffer.cc: If the bzip library is not available, don't
try to open bzip files. Also, ignore ENODATA errors since they seem to
show up when pread reaches the end of the file.
* lnav.cc: The abs() function is ambiguous on windows, use std::abs instead.
