Article
Analyzing Data from a Pretest-Posttest Control Group Design: The Importance of Statistical Assumptions.
European Journal of Training and Development
(2016)
Abstract
Purpose: Among the gold standards in human resource development (HRD) research are studies that test theoretically developed hypotheses and use experimental designs. A somewhat typical experimental design would involve collecting pretest and posttest data on individuals assigned to a control or experimental group. Data from such a design that considered if training made a difference in knowledge, skills or attitudes, for example, could help advance practice. Using simulated datasets, situated in the example of a scenario-planning intervention, this paper aims to show that choosing a data analysis path that does not consider the associated assumptions can misrepresent findings and resulting conclusions. A review of HRD articles in a select set of journals indicated that some researchers reporting on pretest-posttest designs with two groups were not reporting associated statistical assumptions and reported results from repeated-measures analysis of variance that are considered of minimal utility. Design/methodology/approach: Using heuristic datasets, situated in the example of a scenario-planning intervention, this paper will show that choosing a data analysis path that does not consider the associated assumptions can misrepresent findings and resulting conclusions. Journals in the HRD field that conducted pretest-posttest control group designs were coded. Findings: The authors' illustrations provide evidence for the importance of testing assumptions and the need for researchers to consider alternate analyses when assumptions fail, particularly the homogeneity of regression slopes assumption. Originality/value: This paper provides guidance to researchers faced with analyzing data from a pretest-posttest control group experimental design, so that they may select the most parsimonious solution that honors the ecological validity of the data.
Keywords
- Data Analysis,
- Pretests Posttests,
- Control Groups,
- Labor Force Development,
- Research Design,
- Experimental Groups,
- Statistical Analysis,
- Regression (Statistics),
- Quasiexperimental Design,
- Multiple Regression Analysis
Disciplines
Publication Date
September 6, 2016
DOI
10.1108/EJTD-08-2015-0066
Citation Information
Linda Zientek, Kim Nimon and Bryn Hammack-Brown. "Analyzing Data from a Pretest-Posttest Control Group Design: The Importance of Statistical Assumptions." European Journal of Training and Development Vol. 40 (2016) p. 638 - 659 Available at: http://works.bepress.com/kim-nimon/26/