Interphase chromosomes are organized into topologically associated domains in order to establish and maintain integrity of transcriptional programs that remain poorly understood. Here, we show that condensin II and TFIIIC are recruited to bidirectionally transcribed promoters by a mechanism that is dependent on the retinoblastoma (RB) protein. Long-range chromosome contacts are disrupted by loss of condensin II loading, which leads to altered expression at bidirectional gene pairs. This study demonstrates that mammalian condensin II functions to organize long-range chromosome contacts and regulate transcription at specific genes. In addition, RB dependence of condensin II suggests that widespread misregulation of chromosome contacts and transcriptional alterations are a consequence of RB mutation.