Objective: To assess the effect of prognostic factors and their impact on survival in male and female breast cancer. Methods: Medical records for men and women diagnosed with breast cancer referred to the cancer center for treatment were reviewed. Patients with distant metastatic diseases were excluded. Data on prognostic factors including age, nodal status, resection margin, use of hormonal therapy, chemotherapy with and without hormone and radiation therapy (RT), survival, and recurrence were analyzed. Survival estimates were obtained using Kaplan-Meier methodology. The Cox regression interaction was used to compare male and female differences in prognostic factors. Male breast cancer (MBC) and female breast cancer (FBC) were matched according to propensity scores and survival compared using Cox regression. Results: From 1963-2006, there were 75 MBC and 1,313 FBC totaling 1,388 breast cancers. The median age of the cohort was 53 (range: 23-90) years. Median follow-up was 90 (range: 0.4-339) months. Prognostic factors of patients were balanced among the groups after adjusting for propensity scores. A Cox model adjusting for propensity scores showed that overall survival (OS) (HR= 2.52 (1.65, 3.86), P<0.001) and distant disease recurrence-free survival (DDRFS) (HR= 2.39 (0.75, 3.04), P=0.003) were significantly different for MBC and FBC. Analyses that stratified by propensity score quintiles had similar findings: OS HR=2.41 (1.67, 3.47), P<0.001); DDRFS HR=2.89 (1.81, 4.60), P<0.001). When MBC and FBC were matched (1:3) by propensity scores, differences between MBC and FBC were again observed in OS (HR=1.94, 95%CI:1.18-3.19, P=0.009) and DDRFS (HR=2.79, 95%CI:1.36-5.75, P=0.005) with MBC at a higher risk of death and disease recurrence compared to FBC . Conclusion: This large series showed that MBC and FBC survivals are not similar, with MBC having a worse outcome. The finding of this study needs confirmation from a complete prospective database.