Evidence gained through assessment data informs us about what a student is ready to learn and how teachers can then scaffold their learning. Games are gaining momentum in the field of assessment for evaluating student performance and other complex 21st century skills which are often difficult to measure using traditional assessment design. Changes in organizational infrastructure have stressed the importance on enhancing critical-thinking skills through the use of collaborative learning environments; creation of which is now possible due to the advances in technology. These environments can be tailored to provide learning activities and the opportunity to collaborate, problem solve and apply critical thinking skills. With the addition of networks, they become multi-user environment, allowing multiple perspectives, argumentation and collaborative decision making behaviours. The Assessment and Teaching of 21st Century Skills (ATC21STM) project designed digital game-like collaborative problem solving assessments for measuring cognitive and non-cognitive 21st century skills. Construction of such complex and interactive game-based assessment tasks can instigate logistical and pedagogical challenges for both the developers and the potential users of such technology. Development of games is not only challenging, but also expensive, time-consuming, resource-demanding process. Issues in game construction are manifold and can arise anytime during the period of conception to implementation – specifically games intended for learning or assessments. This is partly attributed to the specialised design and technical skills required for such development undertakings. Given the pitfalls associated with the development of games, appropriate awareness needs to be created among individuals involved in such effort. This paper will highlight the experiences and reflect recommendations of the development team as observed during the deployment of the collaborative problem solving assessment design and delivery endeavour.