How to Read 200,000 Publications: VIVO and the intelligent evaluation of scholarship
VIVO and the data that it manages enable a variety of new mechanisms for analyzing and evaluating the scholarship of individuals and organizations. These mechanisms provide both opportunities and dangers.
As critics of overuse of impact factors, magazine rankings, and student survey scores know, it can be tempting to rely on short-cuts. Aggregation and automated analysis of readily available data, used wisely, can reveal important information that can help assess and improve scholarly impact. Used unwisely, however, it can replace the thoughtful assessment of scholarly quality with quick comparisons of mechanically-generated scores that may have little to do with the actual value and impact of scholarly work.
As the Penn Libraries have implemented our VIVO service, which now has data on over 200,000 publications in the healthcare fields alone, we have considered what kinds of questions about research and scholarship can be usefully answered with algorithmic and aggregated "distant reading" of the data we collect in the VIVO service, and what kinds of questions are best answered with "close reading" of this data.
We find that, with appropriate data collection and analysis, VIVO can be a powerful tool to support both kinds of readings. In this talk, we share our thinking about appropriate distant and close reading mechanisms for answering important questions about the scholarship we track in VIVO, and invite implementations and further ideas to help answer these questions.
John Mark Ockerbloom and Anne Seymour. "How to Read 200,000 Publications: VIVO and the intelligent evaluation of scholarship" VIVO 2013 conference. St. Louis, MO. Aug. 2013.
Available at: http://works.bepress.com/john_mark_ockerbloom/15