All a matter of state? A multilevel perspective on the limited comparability of school grades

School grades have always been a controversial topic of discussion, especially with regard to their meaning and comparability. Their controversy is evident from various publications, some dating back to the 19th century. Recently, there have been debates in education policy, highlighting contradictions between the unclear meaning of school grades on the one hand and their great importance for selection and allocation processes on the other. A particular focus in recent discussions involves criticism of the lack of comparability of school grades from different federal states, which, many argue, could be overcome by a national education strategy and the dissolution of federal structures. This article uses these discussions as a starting point from which to examine the comparability of school grades in more detail. For this purpose, we provide a systematic breakdown of the contributions of different levels (within schools, between schools, between federal states) to the lack of comparability of school grades. The study is based on representative data from N = 55,002 students from the IQB Trends in Student Achievement 2015 and 2018 (Grade 9). Results suggest that students with the same school grades acquired substantially different competencies (and vice versa). Further, it was found that these differences can be explained substantially by differences between schools within states and much less by differences between federal states. The results illustrate that the debate on the limited comparability of school grades should focus not only on between-state comparability but also on mechanisms of grading and causes of grade differences between schools within states.