Abstract

Permutation tests or randomization tests, are nonparametric tests that date back to Fisher(1935). This family of tests is one of the important technique which is distribution-free and suitable to hypothesis testing. Permutation tests are categorized in two groups: Exact permutation tests and approximate permutation tests. In an exact test, only the exchangeable units are allowed to be permuted, while the permuted units can be arbitrary permuted in an approximate test. This test can be applied in experimental designs when the parametric assumptions such as Gaussian error terms is not satisfied. In the literature, different strategies have been proposed for factorial ANOVA (i.e. ANOVA with a single error term), but no general method that can be applied for any ANOVA and testing any factor was proposed. We introduced an exact permutation test for fixed effect ANOVA to test any factor of interest, for balanced and unbalanced designs. We will compare the level and power of our proposed method with other exsiting methods. Results of simulations show that our proposed method is the only one that keeps a correct level over all settings. We extend the permutation methods in the repeated measure ANOVA and mixed-model designs, by introducing an approximate permutation tests with a general formula which is applicable in unbalanced desgin as well. We also introduced an exact permutation test for balanced designs. We will also mention one of the application of our introduced methods in the analysis of real EEG (Electroencephalography) signals.