Residential energy consumption is skyrocketing, as residential customers in the U.S. alone used 1.4 trillion kilowatt-hours in 2014 and the consumption is expected to increase in the next years. Previous efforts to limit such consumption have included demand response and smart residential environments. However, recent research has shown that such approaches can actually increase the overall energy consumption due to the numerous complex human psychological processes that take place when interacting with electrical appliances. In this paper we propose a social-behavioral aware framework for energy management in smart residential environments. We envision a smart home where appliances are interconnected using the paradigm of the Internet of Things and where users have a maximum energy budget, for example to reduce their energy bills. Using an experimental and interdisciplinary approach, we define social behavioral models to understand how users perceive different appliances, and how the use of some appliances are contingent on the use of others. We make use of large scale online surveys involving 1500 users to gather data and quantify such models. Based on these models we define a social behavioral aware user utility that is adopted as the objective function of a Mixed Integer Linear Programming problem. The problem looks for a set of appliances that maximizes the user utility while ensuring that the energy budget constraint is met. We show that the problem is NP-Hard and provide a heuristic method to efficiently find a solution. Results on synthetic and real data show that our approach outperforms previously proposed solutions that do not consider the social-behavioral implications, and it requires few iterations to converge towards a final solution.