New PIAAC results: care is needed in reading reports of international surveys

Results from the Survey of Adult Skills, also known as PIAAC (Programme for the International Assessment of Adult Competencies), were recently made available for 24 participating countries. PIAAC involves several developments in relation to the earlier international “adult skills” surveys (IALS in the 1990s and ALL in the 2000s), notably the use of computer administration of the survey. In this paper, I focus on understanding these studies, by considering conceptual issues, methodological validity of research design and execution, and presentation of results. I consider several of the sample items for numeracy published by OECD (2012). And I discuss illustrative results from Australia made available in February 2013, by the Australian Bureau of Statistics. The paper shows when and how to be sceptical when reading international survey reports. It also opens up questions concerning the relevance of the results, and the other types of research that may be needed, in different national and local contexts.