A personal web archive, and the tools built to read it
M4cgyvers Archives is a personal web-preservation project. It crawls sites that are small, old, or likely to disappear â forums, personal homepages, webring-era communities â and stores exactly what the servers sent, as WARC files.
Everything captured is indexed so it can be searched by URL and content type, and replayed as it was captured. The same archive is the reason the WARC viewer exists: a tool for reading a WARC had to be built to check the crawls, and there was no reason not to let it read yours too.
Forums and message boards, personal and hobbyist sites, small community hosts such as Neocities and Nekoweb, and a handful of sites reachable only over I2P. The bias is toward places that are not being archived by anyone else and are one hosting bill away from vanishing.
Crawling is done with wget --warc-file, which records every request and response rather than a rendered copy. Nothing is rewritten on the way in.
WARC (ISO 28500) is the format national libraries and the Internet Archive use. It stores the raw HTTP exchange â headers, status line and body â so a capture can be replayed later, and inspected rather than merely read.
A directory of saved HTML cannot tell you what the server said; a WARC can. See how to open a WARC file for what that means in practice.
A parser written for this archive reads the WARC records and loads them into Postgres, which backs the search index. A small server serves the API; the site itself is a Next.js application.
The interesting part is that the parser is one codebase. The same code that indexes the collection on the server is compiled for the browser and runs in the viewer's Web Workers, so the two cannot disagree about what a WARC record means.
One person, as a hobby project. It is not a company, it is not funded, and it has no users to sell â which is most of the reason the viewer parses your files locally instead of asking you to upload them.
Captures are stored as they were received. Inclusion in this archive is not an endorsement of anything captured, and the archive holds no rights over third-party content â the copyright in a captured page belongs to whoever wrote it.