Large-scale text corpora constitute a great opportunity and challenge for natural language processing including machine learning. One state-of-the-art approach, Unsupervised Semantic Parsing technique (USP), clusters synonymic relations on a sentence level and has been shown to answer a broad range of questions by exploiting these semantic clusters. In this paper, we propose Distributed USP (DUSP), which improves USP’s ability to handle large text corpora by distributing several of USP’s key algorithmic steps over a cluster of commodity computers. In experiments with DUSP, we processed a corpus that was over 13 times larger than the largest corpus we were able to handle using USP. In addition, DUSP’s processing speed was 284 documents per minute, versus 69.2 for USP.