This is a collection of shell scripts to capture, preserve, and replay York University and Government of Canada websites using Browsertrix Crawler and pywb.
Add to cron.
Ex:
05 09 * * 1 bash -c 'yulWA --name "yfile" --crawl-config "/crawl-configs/yu-yfile.yaml" --crawl-dir "/browsertrix" --dedup-dir "/dedup" --import-dir "/import" --workers 8 --version 1.6.0 > /dev/null 2>&1'
Public Domain
