- This component was not published to the rpm registry anyway
- We depended on the repo’s master branch which can break things for everyone
- Its repo was not getting much attention which meant
* no tests, no coverage report
* no linting
* no package linting (e.g. had 2 dependencies that weren’t actually used)
It will be simpler to manage this way
fix(index): Continue on errors during find
This ensures the gallery is created even when some files are
inaccessible (for example due to invalid names or file permissions).
Requires running the test in Docker as non-root to allow file access test
1. Move from a JSON index to a SQLite database.
- This allows the indexing to be interrupted & resumed
- Updating the index consumes less RAM than loading / saving an entire JSON object
- Loading the index consumes less RAM since it can be streamed, only exacting the properties we need every time (instead of loading all EXIF data in memory, only to discard most of it later)
- These make a big difference when processing 10,000+ photos
2. Switch from <glob> to a manual <readdir>
- Glob would take several hundred or GB of RAM when asked to find several thousand files
- Manual approach with <micromatch> library does the same thing in a fraction of the time / memory usage
3. Exiftool optimisations
- Run 1 exiftool process per CPU, still in batch mode (divide all files to be read into 1 bucket per CPU)
- Stream the exiftool output instead of buffering it in memory
