Announcing Documentation for the Internet Archive S3 API
Back in 2011 I was hired at Internet Archive to develop a digital archive service for memory institutions. Unfortunately, after six months the project was scrapped (along with my position).
In the last month of the project, while waiting for the go-ahead to keep moving forward, I undertook documenting the Archive’s S3-like API. The project was going to need this API and, to be entirely frank, the existing documentation is laughable (sorry, IA team, but it’s a spade). My API doc was nearing completion about the same time the project was axed, but it was never published and ended up collecting dust in my Dropbox.
Fast forward a couple of years. Now, as a co-organizer for San Francisco Perl Mongers, I stream and record as many of our events as possible. Afterward, I upload them to our SF.pm Collection on Internet Archive because I believe in their mission of “Universal access to all knowledge.”
In the process of that uploading, I found myself referring frequently to that old in-progress API doc. Finally it dawned on me that I should probably share the damn thing so others could benefit as well.
It took a lot of cleanup and editing, but now I present to you:
The Internet Archive S3 API Documentation.
This API will allows for the creation and maintenance of items on Internet Archive. It also allows uploading of files to the item and, if the item has the appropriate metadata values, Internet Archive provides online viewers for this item content. For more information, have a look at the API Summary & FAQ.
It’s my hope that this documentation will allow many more user groups, individuals, and institutions to preserve and share their content via Internet Archive (for free, might I add, but donations are always welcome). I think of it as a grassroots continuation of the stillborn Digital Archive Service I once worked to produce.
NOTA BENE #1: If you have a lot of content to upload to the Archive, please be a good citizen and contact Internet Archive to coordinate with them. The crack IA Collections department will help the process be as smooth as possible.
NOTA BENE #2: This is not an Internet Archive document. They are not responsible for any shortcomings it may have. Please see the support page for more information about that.
If you use this document (and I do hope you will) and do find any shortcomings, please let me know! This doc is in Github specifically because it makes it so easy to collaborate on this sort of thing.