Evidence and Scarcity

As dig­i­tal tech­nol­ogy expands, our whole rela­tion­ship to pre­sent­ing evi­dence will have to change.

Other dis­ci­plines roll their eyes at his­to­ri­ans for hav­ing  too many spacer exam­ples. A lit scholar might read a his­tor­i­cal arti­cle and think “he had me at five: the forty five exam­ples that fol­lowed  didn’t accom­plish a damn thing.” And even his­tory grad stu­dents learn to skim the long train of exam­ples, unless they’re directly related to their research. It’s a legacy of his­tory as a sci­ence: enough exam­ples proves repeata­bil­ity and the­o­ret­i­cally estab­lishes truth.

Set that aside for a moment, though, and think about the other rea­son why we needed to present the exam­ples: scarcity. Just as the struc­ture of aca­d­e­mic paper pub­lish­ing grew from the scarcity of print, so too the struc­ture of evi­dence pre­sen­ta­tion grew from the scarcity of time and access.

Print­ing a jour­nal or book was/is expen­sive. Even today paper is expen­sive, print­ing is expen­sive, bind­ing and ship­ping are expen­sive; they were even more so in 1870, when the pro­fes­sion got started. Peer review helped con­serve this scarce media, by mak­ing sure that resources–physical resources,  but also resources of time and attention–only went to vet­ted work. Print­ing was a way to dis­ci­pline scarcity.

The historian’s march of exam­ples also rep­re­sented a rela­tion­ship to scarcity–scarcity of time to do the research, scarcity of mate­r­ial, and scarcity of access. Good evi­dence was/is often hard to find, some­times located only in an archive, and some­times buried deep within other sources. The his­to­rian needed pro­fes­sional cre­den­tials to get into the archive, and he or she needed the time required to do the work. Archives had lim­ited work­ing hours. Both were rel­a­tively scarce, and the parade of exam­ples demon­strated the historian’s tri­umph over scarcity.

That scarcity is less and less of a prob­lem. When I wrote my dis­ser­ta­tion, on the inven­tion of stan­dard time, I spent about 9 months look­ing at every Amer­i­can news­pa­per extant for the time period Sep­tem­ber through Decem­ber 1883. Sit­ting in front of a micro­film reader for long hours every day, turn­ing the han­dle, feel­ing slightly sick, stop­ping to make a note­card when I found some­thing use­ful. The New York Times index helped, but oth­er­wise, I was hunt­ing for ref­er­ences to “time,” “stan­dard time,” “rail­roads,” etc. by skim­ming the newspapers.

spacer Today, most of those news­pa­pers are dig­i­tized and word search­able. A search that took months takes ten min­utes. The scarcity is gone. Over a decade ago, at the begin­ning of my research for Face Value, I looked for texts about money, and searched them for metaphors that used racial or genetic terms. Today, most of those same texts are word searcheable: I can find the same results almost instantly across a broad range of texts.

Google has dig­i­tized I believe more than ten mil­lion books. In my field, 19th cen­tury US, that means that nearly every pub­lished book is avail­able instantly on  line, fully searcheable 24/7. Scarcity is gone.

So one of the things the pro­fes­sion rec­og­nized and rewarded–the sheer effort required to plod through scarce resources–is now out­dated. And while of course not all archives are online, and some may never be, who can doubt that arc points toward ever increased digitization?

spacer If the profession’s byways and prac­tices emerged in an era of tex­tual and evi­den­tiary scarcity, how will we remake it to fit an era of evi­den­tiary abun­dance? Should we keep act­ing as if  access to old books is lim­ited to peo­ple with Uni­ver­sity library cards? Do we really need to keep parad­ing the fruit of hours in the library, when any casual user can find the same results in a few seconds?

Clearly, an aca­d­e­mic his­to­rian still has things that can’t be casu­ally acquired–context, depth of knowl­edge, a com­par­a­tive frame­work. These things are still essen­tial. And just as clearly, some kinds of research, some kinds of ques­tions, fore­ground the end of scarcity more than others.

I’m not sure exactly what this change in the axis of scarcity means. It’s par­tic­u­larly acute in the kind of his­tory I do, cul­tural his­tory inflected by ‘post­mod­ernism.” Per­haps in  the future, rather than trot­ting out 40 exam­ples, we will sim­ply describe the “infor­ma­tion archi­tec­ture” of our searches. Thought­ful prac­ti­tion­ers of his­tory quickly real­ize how the kinds of ques­tions they ask deter­mine the range of pos­si­ble out­comes. All his­tor­i­cal inves­ti­ga­tions depend on a scaf­fold­ing of assump­tions and pre0questions, “met­ta­nar­ra­tives” that we assume in order to make our inves­ti­ga­tions cohere.

Per­haps future his­tor­i­cal inquiries will spend more time describ­ing the search terms used, and less fur­nish­ing results. Describ­ing the terms one searches for: “gold” and “negro” and “amal­ga­ma­tion,” within 50 words of each other; or “time,” “rail­road” and “stan­dard­ized” on the same page, not only pro­duces cer­tain results: it reveals the struc­ture of historian’s inquiry, and invites oth­ers to use it or mod­ify it.

The Cen­ter for His­tory and New Media at GMU is work­ing on new ways of “min­ing” text. We are all more and more used to “dumb” word searches that pro­duce huge results. spacer How might we refine and rethink the  dig­i­tal search process, to make it more use­ful for historians?

Imag­ine search­ing a range of documents–the col­lected writ­ings of Char­lotte Perkins Gilman, say–and being able to get a list of all the adjec­tives she used, bro­ken out by fre­quency. Or search­ing for a term like “equal­ity” and see­ing the results ranked by the words most fre­quently pared with “equal­ity?” Or hav­ing the results returned as a tag cloud, with syn­onyms and antonyms dis­played in dif­fer­ent col­ors and sizes, by fre­quency. The limit here is our imag­i­na­tion, not tech­no­log­i­cal possibility.

So it seems to me that it’s now essen­tial to rethink how his­tory will look. The things that drove the forms and modes of schol­arly work–the scarcity of time, mate­ri­als and access–are less and less present. Elec­tric­ity was once a scarce resource. When it became cheap and abun­dant, it changed every­thing about the way we lived. Now it’s threat­en­ing to become more dear, and so we are hav­ing to rethink what it does for us and how we use it. It’s time to rethink how we use evi­dence in an age of evi­den­tiary abundance.

Update

My col­league Sean Takats made an excel­lent response to this post, point­ing out that the abun­dance of evi­dence might only cause his­to­ri­ans to “dou­ble down” on quan­tity. I fear he might be right. It increases my sense that we have an oblig­a­tion to think more about the “infor­ma­tion archi­tec­ture” of our ques­tions. No amount of evi­dence can con­ceal the banal­ity of a weak premise.

October 2, 2010 – 6:37 pm | By mike | Posted in History, Top Posts | Tagged academic publishing, evidence, searching | Comments (18)

Comments are disabled for this post