Using synthetic designs, a new form of true experimental design: can the length of university exams be shortened without losing the integrity of the test?

Abstract

Educational researchers seldom resort to using experimental methods when evaluating teaching innovations in higher education, most likely due to the shortcomings of traditional experimental methods (between-subjects, within-subjects, and matchedsubjects). We introduce a new form of experimental designs, synthetic designs, which can offer substantial reductions in sample sizes, cost, time and effort expended, increased statistical power, and fewer threats to validity (internal, external, and statistical conclusion). This new design is a variation of within-subjects design in which each subject serves in only a single treatment condition. Performance scores for all other treatment conditions are derived synthetically without repeated empirical testing of each subject. We use this new design to explore the potential for reducing exam length in four university business statistics classes. We justify synthetic designs here on three grounds: this design has been used successfully in exploring the potential for shortening the length of final exams in university courses, showing marked advantages over traditional experimental designs; a detailed comparison with raditional designs showing their advantages on all but one of the 18 criteria considered; and a description of a method that many university professors could with little effort use to address empirically the same issue of the potential for reducing their exam marking workload in their own course(s). We find conservatively that at least a one-third reduction in exam length can be achieved, which should result in reduced student fatigue and stress, increased perceived fairness by students with diverse learning styles, and reduced educators’ workload.