Zeitschrift für Evaluation: Issue 2/2013, 12. Volume Normally Not Random: The Randomization Test as an Alternative to Classic Significance Testing in Experimental Impact Evaluation

In impact evaluations, experimental designs allow to eliminate and to control for various sources of non-random bias. Besides, evaluators have to deal with sources of random error, which is done by the use of significance tests. In frequently used tests, inferences are usually made from samples about populations assuming that samples were drawn randomly. Because random samples are difficult to achieve in evaluation practice and results are frequently not reflected critically, inferences made on basis of non-random samples are often not appropriate. Therefore, we present the randomization test as an alternative that does not rely on random samples and demonstrate its empirical application.

Keywords
Experiment, Significance, Randomization, Resampling