The Q-Cochran analysis of variance, based on the Q-Cochran test, is described by Cochran (1950). This test is an extended McNemar test for several dependent groups. It is used in hypothesis verification about symmetry between several measurements for the X feature. The analysed feature can have only 2 values - for the analysis, there are ascribed to them the numbers: 1 and 0.

Basic assumptions:

measurement on a nominal scale (dichotomous variables– it means the variables of two categories),

a dependent model.

Hypotheses:
H_{0}: all the ”incompatible” observed frequencies are equal,
H_{1}: not all the ”incompatible” observed frequencies are equal,

where:

”incompatible” observed frequencies – the observed frequencies calculated when the value of the analysed feature is different in several measurements.

The POST-HOC tests available in the Q-Cochran ANOVA:

We want to compare the difficulty of 3 test questions. To do this, we select a sample of 20 people from the analysed population. Every person from the sample answers 3 test questions. Next, we check the correctness of answers (an answer can be correct or wrong). In the table, there are following scores:

Hypotheses:
H0 : The individual questions received the same number of correct answers, in the analysed population,
H1 : There are different numbers of correct and wrong answers in individual test questions, in the analysed population.

Comparing the p value p = 0.007699 with the significance level α = 0.05 we conclude that individual test questions have different difficulty levels. We resume the analysis to perform POST-HOC test by clicking , and in the test option window, we select POST-HOC Dunn.

The carried out POST-HOC analysis indicates that there are differences between the 2-nd and 1-st question and between questions 2-nd and 3-th. The difference is because the second question is easier than the first and the third ones (the number of correct answers the first question is higher).