Rainfall Rescue Project – bringing archived data back to life

At the Met Office National Meteorological Archive in Exeter we look after the ‘national memory of the weather’, writes Catherine Ross, the Met Office’s archivist.

This archive consists of the millions of meteorological observations produced by the Met Office since its incarnation in 1859 and a large collection of supporting data and information from across the globe.

Catherine Ross – Met Office archivist – with part of the vast Met Office archive: the national memory of weather. Picture: Grahame Madge, Met Office

Among this treasure trove of climatological data are the Ten Year rainfall books, which are not very well named, and are actually folders of loose paper sheets! They hold a vast and hitherto largely untapped supply of monthly rainfall data from 1677 – 1960.

The earliest data comes from a range of different published and unpublished sources. From 1860 data was sent into the British Rainfall Organization, and later the Rainfall Branch of the Met Office, by a network of observers from all around the British and Irish Isles.

Rainfall observation proved popular! When George Symons established the British Rainfall Organization in 1860 there were already several hundred rain gauges and this number increased to several thousand over the following 30 years. All of them diligently cared for and monitored by volunteer observers, some of whom dedicated many decades of their lives to the careful collection of rainfall data.

One challenge in using the data is that each folder contains ten years of data for every station, so you have to go through every single folder to find all the data for one place. The data series was therefore a perfect candidate for scanning in the archive. One year and 66000 sheets of paper later we had a lot of digital images loaded to our Digital Library and Archive, but to make best use of them, the data still needed transcription.

That’s where Ed Hawkins, the Zooniverse platform and the RainfallRescue.org site came in. The national lockdown in March 2020 also provided a sudden supply of people with spare time on their hands and quite incredibly 16000 volunteers transcribed all 66000 pages (containing 5.28 million numbers) within 16 days.

In fact each number was transcribed four times for quality control, so the 16,000 volunteers actually transcribed 21 million observations and with the additional collection of location information the total keystrokes are estimated at 100 million!

It was great to see the work of the volunteers of 2020 bringing the records of those volunteer observers of the 18th, 19th and early 20th centuries back to life. After a rigorous process of checking by a team from Reading, the Met Office and eight dedicated volunteers from the original project, a large tranche of that data has been added to the official Met Office national rainfall climate series. Rainfall Rescue now accounts for 84% of the monthly observations in the HadUK-Grid for 1836 – 1960!

UK annual rainfall statistics

UK annual rainfall statistics.

From this work we now have a record of UK-wide rainfall variations and trends back to 1836. To understand the risks from climate change and weather extremes we need to understand the past. Rainfall in the UK is notoriously variable, we are an island nation prone to both flood and drought. Therefore to capture past extremes we need long historical climate records. The figure below shows annual rainfall averaged across the whole of the UK back to 1836. A general increase in rainfall is evident, but the driest year in the series is 1855 and the wettest 1872 with the new data allowing us to map these extremes in more detail than ever before.

Driest and wettest years in the UK rainfall record.

Not only has all this work been achieved incredibly quickly, it has also broken the definition of what makes a record archival. In its lifecycle a document moves from being a record – in everyday use – to an archive where it is kept as part of a memory – in our case the memory of the weather. The 66,000 sheets of numbers certainly count as that, but in their new transcribed form the same numbers are ‘living a new life’ as part of the datasets available for daily use by scientists at the Met Office and around the world. It’s a little like a parallel universe for archives! We think this is great, but we’re also loving the fact that this goldmine of data is now back in the hands of the scientists who need it.

You can read more about the project in our press release here.

This entry was posted in Met Office News and tagged , , , , , . Bookmark the permalink.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.