After age 65, the incidence of episodic memory decline in males is greater than in females. We explored the influence of anxiety and depression on objective and subjective memory performance in a diverse sample of community-residing older adults. The study was a secondary analysis of data on three samples of adults from two states, Ohio and Texas: a community sample (n = 177); a retirement community sample (n = 97); and the SeniorWISE Study (n = 265). The sample of 529 adults was 74% female, the average age was 76.58 years (range = 59–100 years), and educational attainment was 13.12 years (±3.68); 68% were Caucasian, and 17% had depressive symptoms. We found no memory performance differences by gender. Males and females were similarly classified into the four memory performance groups, with almost half of each gender in the poor memory category. Even though males had greater years of education, they used fewer compensatory memory strategies. The observed gender differences in memory were subjective evaluations, specifically metamemory. Age was not a significant predictor of cognition or memory performance, nor did males have greater memory impairment than females.