From an educational effectiveness perspective, research based on international large scale assessments has been limited as it neglects to take contextual conditions of educational systems into account. Further, methodological challenges of cross-sectional studies have yet prevented investigations from a longitudinal effectiveness perspective. The paper investigates how effectively educational systems grow, i.e. change, in their performance by applying a methodological approach known from school effectiveness research that captures changes at the country level within repeated cross-sectional data designs. Data from the Progress in International Reading Literacy Study (PIRLS) 2001 to 2006 trend systems is analyzed with hierarchical linear modeling. Effectiveness measures of achievement status in 2006 and of change from 2001 to 2006 are investigated and compared. Results suggest that there are systems which exceed their expected outcomes (status and change) as well as systems which stay below what could have been expected, changing the picture of “high” and “low” performing systems, when contextual conditions and prior performances are taken into account. The study contributes to methodological developments of educational effectiveness research in cross-national assessments. Its results provide complementary information for policymakers to further look at policies, practices, and structures that have favored effectiveness.

Keywords
Cross-national comparisons; Educational effectiveness; Repeated cross-sectional design