1

Crowdsourcing is an increasingly popular and attractive option for archivists, librarians, scientists, and scholarly editors working with large collections in need of tagging, annotating, editing, or transcribing. These tasks, it has been argued, can be accomplished more quickly and more cheaply by outsourcing them to enthusiastic members of the public who volunteer their time and effort for free [Holley 2010].[1] Crowdsourcing also benefits the public by making available and engaging volunteers with material hitherto only accessible to diligent researchers, or with sources previously considered too complex for a non-expert to understand. A project like Galaxy Zoo, for example, has successfully built up a community of more than 200,000 users who have classified over 100 million galaxies, thus supporting a great deal of academic research [Raddick et al 2010]. Crowdsourcing aims to raise the profile of academic research, by allowing volunteers to play a part in its generation and dissemination.

2

The Bentham Project at University College London (UCL) sought to harness the power of crowdsourcing to facilitate the transcription of the manuscript papers of Jeremy Bentham (1748-1832), the great philosopher and reformer. The purpose of the Bentham Project is to produce the new authoritative scholarly edition of The Collected Works of Jeremy Bentham, which is based in large part on transcripts of the vast collection – around 60,000 folios – of Bentham manuscripts held by UCL Special Collections.[2] The Bentham Project was founded in 1958, and since then 20,000 folios have been transcribed and twenty-nine volumes have been published. The Project estimates that the edition will run to around seventy volumes; before the commencement of Transcribe Bentham around 40,000 folios remained untranscribed.

3

This new edition of Bentham’s Collected Works will replace the poorly-edited, inadequate and incomplete eleven-volume edition published between 1838 and 1843 by Bentham’s literary executor, John Bowring [Schofield 2009, 14–15, 20–22]. The Bowring edition omitted a number of works published in Bentham’s lifetime, as well as many substantial works which had not been published but which have survived in manuscript; a forthcoming Collected Works volume entitled Not Paul, but Jesus – only a part of which was previously published by Bentham, and was left out of the Bowring edition altogether – will recover Bentham’s thinking on religion and sexual morality. This material has significant implications for our understanding of utilitarian thought, the history of sexual morality, atheism, and agnosticism. Bentham’s writings on his panopticon prison scheme still require transcription, as do large swathes of important material on civil, penal, and constitutional law, on economics, and on legal and political philosophy. In short, while Bentham’s manuscripts comprise material of potentially great significance for a wide range of disciplines, much of the collection – far from being even adequately studied – is virtually unknown. A great deal of work, both in exploring the manuscripts and producing the Collected Works, clearly remains to be done.

4

The Bentham Papers Transcription Initiative – Transcribe Bentham – was established to quicken the pace of transcription, speed up publication of the Collected Works, create a freely-available and searchable digital Bentham Papers repository, and engage the community with Bentham’s ideas at a time when they are of increasing contemporary relevance.[3] Transcribe Bentham crowdsources manuscript transcription, a task usually performed by skilled researchers, via the web to members of the public who require no specialist training or background knowledge in order to participate. The project team developed the "Transcription Desk", a website, tool and interface to facilitate web-based transcription and encoding of common features of the manuscripts in Text Encoding Initiative-compliant XML. Transcripts submitted by volunteers are subsequently uploaded to UCL’s digital repository, linked to the relevant manuscript image and made searchable, while the transcripts will also eventually form the basis of printed editions of Bentham’s works.[4] The products of this crowdsourcing will thus be utilised for both scholarly and general access purposes. Transcribe Bentham was established and funded under a twelve-month Arts and Humanities Research Council Digital Equipment and Database Enhancement for Impact (DEDEFI) grant. The funding period was divided into six months of development work from April 2010, and the Transcription Desk went live for a six-month testing period in September of that year.[5]

5

Crowdsourcing is becoming more widespread, and thus, it is important to understand exactly how, and if, it works. It is a viable and cost-effective strategy only if the task is well facilitated, and the institution or project leaders are able to build up a cohort of willing volunteers. Participant motivation in crowdsourcing projects is, therefore, attracting more focused attention. The teams behind the Zooniverse projects have analysed the motivations and demographic characteristics of their volunteers in an attempt to understand what drives people to participate in online citizen-science projects, while the North American Bird Phenology Programme, established to track climate change by crowdsourcing the transcription of birdwatchers’ cards, also assessed its participants’ opinions. Rose Holley has also offered several invaluable general insights on user motivation, drawing on the experience of crowdsourcing the correction of OCR software-generated text of historic newspapers at the National Library of Australia (NLA), while Peter Organisciak has provided a useful analysis of user motivations in crowdsourcing projects. [Raddick et al 2010] [Romeo and Blaser 2011] [Phenology Survey 2010] [Holley 2009] [Holley 2010] [Organisciak 2010].

6

However, Transcribe Bentham differs from previous crowdsourcing and community collection schemes, in that its source material is a huge collection of complex manuscripts. Though several projects have crowdsourced manuscript transcription, the material they have made available is generally formulaic, or at least reasonably straightforward to decipher and understand [Old Weather Project] [Family Search Indexing] [War Department Papers]. Transcribing the difficult handwriting, idiosyncratic style, and dense and challenging ideas of an eighteenth and nineteenth-century philosopher is more complex, esoteric, and of less immediate appeal than contributing to a genealogical or community collection.

7

This paper describes how the Transcribe Bentham team sought to attract volunteer transcribers and build an online community. It outlines which strategies worked and which did not, and, drawing on qualitative and quantitative data, analyses the complexion of our volunteer base, comparing its demographic and other characteristics with those of other crowdsourcing projects. This evidence will shed more light on the nature of user participation, crowdsourced manuscript transcription, and provide guidance for future initiatives. Section one will describe our attempts to recruit a crowd and build a community of users; section two will analyse the make-up of this user base, and assess site statistics, user contributions, and motivations; and section three will consider the success of our community building with reference to this analysis.

8

Caroline Haythornthwaite has discerned two overlapping patterns of engagement in online "peer production" initiatives like Transcribe Bentham, distinguishing between a "crowd" and a "community". Contributions made by a crowd, which Haythornthwaite describes as "lightweight peer production", tend to be anonymous, sporadic, and straightforward, whereas the engagement of a community, or "heavyweight peer production", is far more involved. A community of volunteers engaged in the latter requires, Haythornthwaite suggests, qualitative recognition, feedback, and a peer support system. Contributors tend to be smaller in number, to be less anonymous, and to respond to more complex tasks and detailed guidelines. Heavyweight peer production might also involve a multi-tiered progress system to sustain motivation; a crowd on the other hand, is satisfied with quantitative recognition, perhaps in the form of progress statistics, and a two-tiered hierarchy such as that of contributor and moderator. These two patterns, Haythornthwaite contends, are often discernable within one project [Haythornthwaite 2009].

9

Transcribe Bentham blends both heavyweight and lightweight peer production. We attracted an anonymous crowd of one-time or irregular volunteers, along with a smaller cohort of mutually supportive and loyal transcribers. We aimed to cast our net wide by opening the Transcription Desk to all, by creating as user-friendly an interface as possible, and by simplifying the transcription process as much as we could (Figure 1). But, as transcribing Bentham’s handwriting is a complex and time-consuming task which requires considerable concentration and commitment, we also tried to build a dedicated user community to enable sustained participation by, for example, implementing a qualitative and quantitative feedback and reward system. The following section will describe the strategies we devised first to recruit the crowd, and then to foster the community.

Recruiting the Crowd: The Publicity Campaign

Building the Community

27

Our six-month testing period lasted from 8 September 2010 to 8 March 2011, and during this time 1,207 people registered an account (discounting project staff, and seven blocked spam accounts).[18] Between them these volunteers transcribed 1,009 manuscripts, 569 (56%) of which were deemed to be complete and locked to prevent further editing. Though the fully-supported testing period has ceased, the Transcription Desk will remain available dependent on funding, and Transcribe Bentham has become embedded into the Bentham Project’s activities. As of 3 August 2012, the project now has 1,726 registered users. 4,014 manuscripts have been transcribed, of which 3,728 (94%) are complete and locked to prevent further editing. However, unless otherwise stated, the analysis below pertains to the six-month testing period.[19]

28

In this section, we will assess site statistics, user demographics, behaviour, and motivations. Our findings are derived from quantitative data provided by a Google Analytics account,[20] analysis of statistics collated from the Transcription Desk, qualitative findings from a user survey, and comparisons with other studies of crowdsourcing volunteer behaviour.[21] The survey received 101 responses – about 8% of all registered users – 78 of which were fully completed. While it is, therefore, not necessarily representative of the entire user base, the survey contains a great deal of revealing information about those who did respond.

29

Before reviewing the data, it is worth taking note of the following milestones in the project’s life during the testing period:

30

As will be seen, the publication of the article in the New York Times (NYT) had a vital and enduring impact upon Transcribe Bentham, and it is thus helpful to consider the testing period as having had two distinct parts: Period One, or the pre-NYT period, covers 8 September to 26 December 2010 (110 days); and Period Two, the post-NYT period, encompasses 27 December 2010 to 8 March 2011 (72 days).

Site Visits