As the state of Mars Climate modeling advances, there will be a growing need to provide observational results to act as comparisons or even as assimilation products to the next generation of models. This is especially true for atmospheric aerosols that are hard to model as they either have unspecified and/or ubiquitous sources (dust clouds) or form faster than the model time-steps can efficiently use (ice clouds). To that end, we present here our technique to recover ice and dust cloud optical depth maps from Mars Reconnaissance Orbiter (MRO) Compact Reconnaissance Imaging Spectrometer for Mars (CRISM) multi-spectral mapping mode data. We use a multi-step, boot-strapping technique that does not make any a priori assumptions about the aerosols or surface reflectance at any point other than assuming the surface, at any time, can be constructed from a linear combination of a finite number of spectral endmembers. It uses singular value decomposition (SVD) and target transformation (TT) to define a candidate endmember space then uses N-FINDR and Hyperplane-based Craig Simplex Identification (HyperCSI) to locate the spectral endmembers. Radiative transfer models are then used to recover the ice and dust optical depth and surface reflectance that best fits the CRISM spectrum. Our resulting optical depth maps are shown to be in gross agreement with ultraviolet (UV), visible wavelength (VIS), and thermal infrared (TIR) results covering the same time periods. Retrieved optical depth values are, on average, within about 50% of those previously reported (Wolff et al., 2019; Montabone et al., 2015), although specific locations can vary by about a factor of 2 indicating some refinements to our retrievals will be needed before we can go into production mode on the remaining CRISM data.