Thanks to the collective action of participating smartphone users, mobile crowdsensing allows data collection at a scale and pace that was once impossible. The biggest challenge to overcome in mobile crowdsensing is that participants may exhibit malicious or unreliable behavior, thus compromising the accuracy of the data collection process. Therefore, it becomes imperative to design algorithms to accurately classify between reliable and unreliable sensing reports. To address this crucial issue, we propose a novel Framework for optimizing Information Reliability in Smartphone-based participaTory sensing (FIRST) that leverages mobile trusted participants (MTPs) to securely assess the reliability of sensing reports. FIRST models and solves the challenging problem of determining before deployment the minimum number of MTPs to be used to achieve desired classification accuracy. After a rigorous mathematical study of its performance, we extensively evaluate FIRST through an implementation in iOS and Android of a room occupancy monitoring system and through simulations with real-world mobility traces. Experimental results demonstrate that FIRST reduces significantly the impact of three security attacks (i.e., corruption, on/off, and collusion) by achieving a classification accuracy of almost 80% in the considered scenarios. Finally, we discuss our ongoing research efforts to test the performance of FIRST as part of the National Map Corps project.