This review covers a spectrum of optoelectronic properties of and uses for semi-insulating semiconductor heterostructures and thin films, including epilayers and quantum wells. Compensation by doping, implantation, and nonstoichiometric growth are described in terms of the properties of point defects and Fermi level stabilization and pinning. The principal optical and optoelectronic properties of semi-insulating epilayers and heterostructures, such as excitonic electroabsorption of quantum-confined excitons, are described, in addition to optical absorption by metallic or semimetallic precipitates in these layers. Low-temperature grown quantum wells that have an arsenic-rich nonstoichiometry and a supersaturated concentration of grown-in vacancies are discussed. These heterostructures experience transient enhanced diffusion and superlattice disordering. The review discusses the performance of optoelectronic heterostructures and microcavities that contain semi-insulating layers, such as buried heterostructure stripe lasers, vertical cavity surface emitting lasers, and optical electroabsorption modulators. Short time-scale applications arise from the ultrashort carrier lifetimes in semi-insulating materials, such as in photoconductors for terahertz generation, and in saturable absorbers for mode-locking solid state lasers. This review also comprehensively describes the properties and applications of photorefractive heterostructures. The low dark-carrier concentrations of semi-insulating heterostructures make these materials highly sensitive as dynamic holographic thin films that are useful for adaptive optics applications. The high mobilities of free carriers in photorefractive heterostructures produce fast dielectric relaxation rates that allow light-induced space-charge gratings to adapt to rapidly varying optical fringe patterns, canceling out environmental noise during interferometric detection in laser-based ultrasound, and in optical coherence tomography. They are also the functional layers in high-sensitivity dynamic holographic materials that replace static holograms in Fourier imaging systems and in experimental Tbit/s optical systems. Semi-insulating heterostructures and their applications have attained a degree of maturity, but many critical materials science issues remain unexplored.