Preserving the History of Cobble Hill

by | Mar 26, 2020

The Cobble Hill Historical Society is a non-profit group dedicated to preserving the history of Cobble Hill and the surrounding areas (Duncan, Shawnigan and Cowichan). They are embarking on an ambitious project to digitize and catalogue their historical assets.

We’re talking about things like land survey reports, personal and business correspondence, advertisements, photographs, newspaper articles – all from an era where everything was handwritten or typewritten, one-off physical copies.

The society is a team of volunteers with limited technical abilities. They need to upload scans, edit metadata and comb through existing metadata to make corrections. Since the scans make for very large files, and they are limited to about 10gb of data by their hosting plan, they also need local thumbnail copies of each image and a full sized version saved to a cloud server, for download as needed.

Here’s what we built.

Cloud storage

DreamHost has a nice front-end for AWS, using almost the same API. For something like $4/month, we were able to totally fulfil this client’s storage needs. Being unfamiliar with the AWS API, I expected this to be the trickiest part of the job, but thanks to the excellent documentation, it was the easiest. I included the SDK, found here, in a plugin that contains all the archive functionality. From there I wrote a class with only three methods – authenticate, upload, and download.

Front end editor

We started with a pile of existing data – 40 or so gigabytes of scanned images on a portable hard drive, and an Open Access database. I built a viewer for quickly clicking through thumbnails and updating descriptions. The client’s priorities were:

  • all the input fields on one screen
  • disable inputs depending on user permission level
  • visual indication when an edit occurs, so user knows they need to save
  • one-click download of full-sized image
  • fast text search
  • dense index listing

And other small tweaks toward the goal of finding skipping through records as quickly as possible.

Uploader

The front end editor and cloud connection were built without a framework, using hand-rolled PHP. When we got to the uploader phase of the job, I realized that it would be simpler to make this a WordPress plugin, so as to take advantage of WordPress’s file handling, database connection and validation.

So I converted the whole thing into a plugin, separating my test application from the custom tool, which was already operational and in use by volunteers.

The simplest way to make a form was to build it in Gravity Forms. Then I hooked into an action that fires after submitting the form, and caught all the data the was submitted. The submission processor was the most complex part of this whole job.

  • convert tiffs, pngs, and jpgs into thumbnail jpgs
  • validate and format submitted data for the database.
  • maintain the file naming scheme established by the CHHS

Fairly simple goals, but accomplishing them was more difficult and took longer than I expected.

What I would have done differently

Maintaining the database as designed by the client was difficult. If I had the chance, I would have pushed back and imported his records into WordPress in the first place. I learned that lesson with my next project, Clayoquot Sound Grey Whales. 

WordPress comes with a ton of features that clients take for granted in web projects – filtering by category and tag, full text search, pagination, image thumbnails, autosave, database locks, and the list goes on. Almost every time I try building a custom project, I learn that I would be better off if I learned about, mastered, and used existing tools. But you can only learn by doing, and this project had a lot of learning in it.