I saw this post and I was curious what was out there.
https://neuromatch.social/@jonny/113444325077647843
Id like to put my lab servers to work archiving US federal data thats likely to get pulled - climate and biomed data seems mostly likely. The most obvious strategy to me seems like setting up mirror torrents on academictorrents. Anyone compiling a list of at-risk data yet?
Thank you for the warning. You are correct. It’s prune to command injection. I will validate the URL before executing it. This shoud suffice until archivebox’s rest API is available in stable.