2
0
mirror of https://github.com/urbanguacamole/torrent-paradise synced 2024-10-31 21:20:13 +00:00
torrent-paradise/update-index.sh
Urban Guacamole 330f426b81 Redo tracker scraping, index generation
Seed/leech counts are fetched from all trackers for every torrent and
data from the tracker with most seeds for a given torrent is used.

I also got rid of SQLite which was previously used for generating
the index. It was replaced by a simple CSV file.  KISS

Some minor bugfixes and tweaks also included (sorry for not breaking
them up into more commits).
2019-03-02 21:26:57 +01:00

18 lines
712 B
Bash
Executable File

# This script updates the index. Testing and uploading to server/IPFS is done manually.
echo "Scraping trackers for seed/leech data"
mosh nextgen@server "~/tracker-scraper" # you can use ssh instead of mosh aswell
ssh nextgen@server "psql -c 'REFRESH MATERIALIZED VIEW fresh'"
echo "Generating index dump"
rm index-generator/dump.csv
ssh nextgen@server "psql -c '\copy (select * from fresh) to stdout with format csv'" > index-generator/dump.csv
(cd index-generator; node --max-old-space-size=10000 main.js)
python3 index-generator/fix-metajson.py website/generated/inx
echo "Uploading website"
cd website
scp -r . root@server:/www/torrent-paradise.ml #todo use rsync https://superuser.com/a/1405020/373473