Commit Graph

62 Commits (master)

Author SHA1 Message Date
Romain 812a063b3f fix: cleanup step shows what would be deleted in dry-run 6 months ago
Romain 8cb7a3596b chore: upgrade to latest version of StandardJS 1 year ago
Romain 4d8dfcb7b7 fix(cli): can explicitly set the log level to “default” for the Listr renderer
Fixes #154
5 years ago
Romain 9c581bbd0b feat(albums): optionally generate a ZIP file for each album
Fixes #125
5 years ago
Romain e9beb830a2 feat(core): skip files that can’t be processed + show a summary at the end
Thumbsup used to stop at the first problem processing a file.
This was a problem on large galleries where you’d need to run it again and again, fixing files as you went.
This change:
- skips problematic files and shows a summary at the end
- logs all warnings/errors to <thumbsup.log> when running the default output

Also refactor and cleanup of the logging logic.
6 years ago
Romain cd3d699b34 feat(cli): new --dry-run option to skip resizing the files / creating the website
The index is still updated, and the media folder is still scanned to log how many tasks are required.
This is useful to know what thumbsup is about to do, without actually running the expensive tasks.
7 years ago
Romain a2042f3128 Fix logging option for non-TTY + highlight URL in welcome / error messages 7 years ago
Romain 47a924883e New --log option to print a full text log with different verbosity levels 7 years ago
Romain 06ecd2edad Add usage reporting with Google Analytics + greeting/error messages
This will help understand usage patterns to know what to focus on, e.g.
- are many people using thumbsup on Windows?
- are there many galleries with > 10,000 photos?
7 years ago
Romain 24b2f9bd7c Major optimisations: SQLite index + faster disk glob + new exiftool streaming
1. Move from a JSON index to a SQLite database.
  - This allows the indexing to be interrupted & resumed
  - Updating the index consumes less RAM than loading / saving an entire JSON object
  - Loading the index consumes less RAM since it can be streamed, only exacting the properties we need every time (instead of loading all EXIF data in memory, only to discard most of it later)
  - These make a big difference when processing 10,000+ photos

2. Switch from <glob> to a manual <readdir>
  - Glob would take several hundred or GB of RAM when asked to find several thousand files
  - Manual approach with <micromatch> library does the same thing in a fraction of the time / memory usage

3. Exiftool optimisations
  - Run 1 exiftool process per CPU, still in batch mode (divide all files to be read into 1 bucket per CPU)
  - Stream the exiftool output instead of buffering it in memory
7 years ago
Romain 30f203af4b Render progress using Listr + split the main process into "steps" which are easier to test 7 years ago
Romain 0b27d37881 Read metadata from Picasa.ini if present 7 years ago
Romain 3e64d2ab38 Create a Metadata model attached to input files
- for easier unit testing
- to enable input filtering (e.g. only include photos with this keyword)
7 years ago
Romain ac67a08da9 Use new “downsize” package that was split out (to make testing edge cases easier) 7 years ago
Romain 6fde22e9d1 Fix #62: crash prevented the error message from showing when migrating from V1 to V2 7 years ago
Romain fb77791c80 Fix #33: new --cleanup option to remove unused generated thumbnails/media 7 years ago
Romain 7e73e6a10b Refactor album mapping code to be much simpler based on (media) => string 7 years ago
Romain 07ec7702cd Fix #f7451db when counting available CPUs (missing array length) 7 years ago
Romain f7451db36e Run as many photo/video tasks as CPUs 7 years ago
Romain 59b9bdbbd2 Use <standard> for code style, which removed lots of unused variables/requires 7 years ago
Romain e71e5d4207 Replace custom "make" DSL with the output task list (#60) 7 years ago
Romain e719bf15b1 Single source of truth for input/outputs + view model for display (#58) 7 years ago
Romain 0c125e90df Thumbnails are back to matching the source extension for now, for compat with V1
Before changing it back, need to list the rules for extension changes. For example:

- should GIF thumbnail should be JPG, to avoid animations on the album page?
- what about transparent GIFs, will they look weird in JPG?
- maybe GIFs should stay as GIFs, but kept to a single frame only for thumbnails
- same thing for pngs, which might be better kept as PNG for transparency
- all other non-browser-friendy formats should become JPG

These rules will be a lot easier to implement when the new input data structure is in place
7 years ago
Romain 8dccb88f25 Use [exiftool-json-db] to maintain the JSON database of media files 7 years ago
Romain e930af7f49 Add GIF support, fixes #22.
One major change here is that thumbnails will always be generated as ".jpg".
This is potentially a breaking change, in the sense that all "png" or "jpeg" thumbnails
would be to re-calculated and re-uploaded.
7 years ago
Romain 58c37bd429 Fix model paths 8 years ago
Romain d91fdbf525 Cleanup output + group photo/video processing 8 years ago
Romain fd9bf181fd Extract model from website generation 8 years ago
Romain b023b72cc0 Extract album creation as separate step + consolidate progress bar logic 8 years ago
Romain Prieto c2f290bbd6 thumbsup v2: rewrite the website around 'albums' 8 years ago
Romain Prieto 1ca3cfe7df Support for "m2ts" format (and smaller video example) 10 years ago
Romain Prieto 82c560d96f Big refactor / cleanup to support multiple pages on the website 10 years ago
Romain Prieto cb835d0ed2 Part of #20, auto-rotates photos based on the EXIF data 10 years ago
Romain Prieto ef294ad513 Fix #18: don't upload original media by default, add new command line options 10 years ago
Romain Prieto c72f46e463 Fix #19: support for MTS files, with full-frame mp4 export 10 years ago
rprieto 3c8ecce018 Use <yargs> to parse command line args, and support JSON config 10 years ago
rprieto 436d126050 Remove <regen> dependency (faster builds)
Since we read all the file metadata for EXIF dates, and we need it as well for the view model,
we should use it to generate the thumbnails and save many calls to glob() and fs.stat()
10 years ago
rprieto 63ee950684 Use the pre-built metadata to create the galleries view model
Fixes #4 : sorts the files by either EXIF date (photos) or file date (videos)
10 years ago
rprieto 11575c65d4 Read and cache EXIF data into <metadata.json> 10 years ago
rprieto 1687999976 Fix #1 : report progress of operations in the console 10 years ago
rprieto d7d9a9e3a9 Fix #3. Display web-friendly videos (smaller size, and keeps original) 10 years ago
rprieto 88ad9d6d05 Fix over-eager copy of original media 10 years ago
rprieto 8b763ef509 Upgrade to regen 0.0.3 10 years ago
rprieto 3c2c3aae96 Extract reusable "regen" module to process source / dest files 10 years ago
rprieto eaa8bd2831 support for custom title and CSS 10 years ago
rprieto 7970913ec7 Fix thumbnail sizes in HTML 10 years ago
rprieto 1608b3cfac Generated website contains original media too
- now generate thumbs + large (original is too big for web download)
- simpler build system
10 years ago
rprieto 3a388181ee Fix thumbnail size in HTML to avoid jumping content while loading 10 years ago
rprieto 288ee7f87c Only create thumbs / copy files if they have changed 10 years ago
rprieto 1268f5557e Remove wrench: using glob() everywhere 10 years ago