Evaluation of T lymphocyte frequency provides prognostic information for patients with oral squamous cell cancer (OSCC). However, the effect of simultaneously evaluating T cell frequency and assessing suppressive elements and defects in antigen-processing machinery (APM) has not been clarified. Simultaneous characterization of CD3+, CD8+, FoxP3+, CD163+, and PD-L1+ cells using multispectral imaging was performed on sections from 119 patients with HPV- OSCC. Expression of β2-microglobulin, MHC class I heavy chain, and large multifunctional peptidase 10 was quantified, and all data were correlated with patient outcome. We found that, consistent with previous reports, high numbers of CD8+ T cells at the invasive margin correlated significantly with prolonged overall survival (OS), while the number of FoxP3+ or PD-L1+ cells did not. Compiling the number of FoxP3+ or PD-L1+ cells within 30 μm of CD8+ T cells identified a significant association with a high number of suppressive elements close to CD8+ T cells and reduced OS. Integrating this information into a cumulative suppression index (CSI) increased correlation with OS. Incorporating tumor expression levels of APM components with CSI further improved prognostic power. This multiparametric immune profiling may be useful for stratifying patients with OSCC for clinical trials.