Existing large-scale scanned book collections have many short- comings for data-driven research, from OCR of variable quality to the lack of accurate descriptive and structural meta-data. We argue that complementary research in inferring relational metadata is important in its own right to support use of these collections and that it can help to mitigate other problems with scanned book collections.