We propose tractable symmetric exponential families of distributions for multivariate vectors of 0's and 1's in dimensions, or what are referred to in this paper as binary vectors, that allow for nontrivial amounts of variation around some central value . We note that more or less standard asymptotics provides likelihood-based inference in the one-sample problem. We then consider mixture models where component distributions are of this form. Bayes analysis based on Dirichlet processes and Jeffreys priors for the exponential family parameters prove tractable and informative in problems where relevant distributions for a vector of binary variables are clearly not symmetric. We also extend our proposed Bayesian mixture model analysis to datasets with missing entries. Performance is illustrated through simulation studies and application to real datasets.