The Production Cycle of Pisa Data in Brazil

The History of Data Beyond the Numbers

Autores/as

  • Marialuisa Villani Independent Researcher, France

DOI:

https://doi.org/10.25749/sis.15100

Palabras clave:

PISA, INEP, ILSA, data production, reference model

Resumen

We analyse the process of the production cycle of the OECD’s Program for International Student Assessment (PISA) data in Brazil for 2012 and 2015 editions.  We present the qualitative data collected in 2016 through semi-structured interviews conducted with members of the Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira [National Institute for Educational Studies and Research] (INEP) who worked for PISA in Brazil for 2012 and 2015 editions. We made an ethnography of Brazilian PISA data to show the level of social embeddedness of statistics. In this paper, we propose a mindful approach to statistical data, particularly when results of international large-scale assessments (ILSA) are used to describe an educational system as complex as the Brazilian one. It is possible identify several levels of governance in PISA data production. That produces gap and misalignments that have an impact on quality of information disseminated by PISA data in Brazil.

Descargas

Los datos de descargas todavía no están disponibles.

Biografía del autor/a

Marialuisa Villani, Independent Researcher, France

Phd in Social Systems, Organization and Policy Analysis at Sapienza University in Rome. Her thesis focused on Italian and French student university experience in Bologna reform context. In 2014/15, she decided to specialise in Statistic for Social Sciences at EHESS and ENS in Paris.  In 2016 she received Refeb fellowship from French Embassy in Brazil. Her ongoing research is about the relationship between national and international assessment in Southern-American countries, the production of statistical data and educational policy design implementation. In addition she is working on the self-academic construction of European Earlier career researchers. 

Citas

Bottani, N., & Vrignaud, P. (2005). La France et les évaluations internationales. Rapport pour l'haute Conseil de l'évaluation de l'école. Paris : Ministère de l’Éducation Nationale.

Carvalho, L. M. (2012). The Fabrications and Travels of a Knowledge-Policy Instrument. European Educational Research Journal, 11(2), 172-188. doi: 10.2304/eerj.2012.11.2.172

Davies, H. T. O., Nutley, S. M., & Smith, P. C. (2000). What works?: evidence-based policy and practice in public services. Bristol: Policy Press. doi: https://doi.org/10.1111/1467-9302.00144

Desrosières, A. (2014). Prouver et Gouverner. Paris : La Découverte.

El Masri Y. H., Baird J., & Graesser A. (2016). Language effects in international testing: the case of PISA 2006 science items, Assessment. Education: Principles, Policy & Practice, 23(4), 427-455. doi: 10.1080/0969594X.2016.121832

Felouzis, G., & Charmillot S. (2012). Les Enquêtes PISA. Paris : Puf.

Foucault, M. (2004). Sécurité, territoire, population. Cours au Collège de France (1977-1978). Paris : Seuil.

Granovetter, M. (1973). The Strength of Weak Ties. The American Journal of Sociology, 78(6), 1360-1380. doi: https://doi.org/10.1086/225469

Grenet, J. (2008, February 8). PISA : une enquête bancale? . La vie des idées. Retrieved from http://www.laviedesidees.fr/PISA-une-enquete-bancale.html

Head, B. W. (2008). Three lenses of evidence-based policy. Australian Journal of Public Administration, 67(1), 1-11. doi: https://doi.org/10.1111/j.1467-8500.2007.00564.x

Jerrim, J. (2016). PISA 2012: How do results for the paper and computer tests compare? Department of Quantitative Social Science Working Paper No. 16-02, February 2016, (16), 1-38.

Klein, R. (2011). Uma re-análise dos resultados do PISA: problemas de comparabilidade. Ensaio: Avaliação e Políticas Públicas Em Educação, 19(73), 717-768. doi: https://doi.org/10.1590/S0104-40362011000500002

Komatsu, H., & Rappleye, J. (2017). Did the shift to computer-based testing in PISA 2015 affect reading scores? A View from East Asia. Compare: A Journal of Comparative and International Education. doi: https://doi.org/10.1080/03057925.2017.1309864

Lascoumes, P. (2004). Présentation - La gouvernementalité : de la critique de l ’ État aux technologies du pouvoir. Le Portique, Revue de Philosophie et de Sciences Humaines, 14(13-14), 1-15. doi: https://doi.org/https://leportique.revues.org/625

Le Donné, N. (2015), Inégalités sociales de compétences (scolaires) et organisation du système éducatif. Études comparées à partir des enquêtes PISA de 2000 à 2009. (Unpublished doctoral dissertation). Science Po, Paris, France.

Lindblad, S., Pettersson, D., & Popkewitz, T. S. (2015). International Comparison of School Results. A sistematic review of a research on large scale assessments in education. Stockholm: Swedish Research Council. doi: 10.13140/RG.2.2.23176.01286

Loveless, T. (2014). How well are American students learning. With the section on the PISA-Shanghai controversy, Homework and common core. Washington, DC: Brookings Institution—Brown Center. Retrieved from http://www.brookings.edu/~/media/research/files/reports/2001/9/education/09education.pdf

MEC/INEP. (2013). Relatório nacional PISA 2012: Resultados brasileiros. Brasilia. Doi: https://doi.org/10.1787/9789264207486-11-de

MEC/INEP. (2016). Brasil no PISA 2015: análises e reflexões sobre o desempenho dos estudantes brasileiros. Brasilia. Retrieved from http://download.inep.gov.br/acoes_internacionais/pisa/resultados/2015/pisa2015_completo_final_baixa.pdf

Mons, N. (2007). Les nouvelles politiques éducatives. Paris : Puf.

Morgan, C. (2011). Constructing the OECD Programme for International Student Assessment. In M. Pereya (Ed.), PISA under examination: Changing Knowledge, Changing Tests, and Changing Schools (pp. 47-59). Rotterdam: Sense Publisher. doi: https://doi.org/10.1007/978-94-6091-740-0_4

Normand, R. (2015). L’enquête PISA : un objet-frontière des politiques éducatives. Administration & Éducation, 145(1), 121-126. Retrieved from https://www.cairn.info/revue-administration-et-education-2015-1-page-121.htm.

Nóvoa, A., & Yariv-Mashal, T. (2003). Comparative Research In Education: A mode of governance or a historical journey? Comparative Education, 93(4), 423-438.

OECD (1999). Measuring Student Knowledge and Skills: A New Framework for Assessment. Paris: OECD Edition.

OECD (2013). PISA 2012 Results: Excellence through Equity Giving Every Student the Chance to Succeed Volume II. Paris: OECD Edition.

OECD (2014). PISA 2012 Technical Report. Paris: OECD Edition.

OECD (2016a). PISA 2015 Technical Report. Paris: OECD Edition.

OECD (2016b). PISA 2015 Results (Volume I): Excellence and Equity in Education. Paris: OECD Edition.

Pettersson, D. (2008). International knowledge assessments: an element of national educational steering. (Doctoral dissertation) Uppsala: Acta Universitatis Upsaliensis. Uppsala Studies in Education No 120.

Pettersson, D. (2014). Three Narratives: National Interpretations of PISA. Knowledge Cultures, 2(4), 172-191.

Polanyi, K. (1977). The Economy Embedded in Society. In K. Polany (Ed.), The Livelihood of Man (pp. 47-56). New York: Academic Press.

Prais, S. J. (2003). Cautions on OECD’S Recent Educational Survey (PISA). Oxford Review of Education, 29(2), 139-163. doi: https://doi.org/10.1080/0305498032000080657

Rivas, A. (2015). Latin America after PISA: Lessons Learned about Education in Seven Countries. Buenos Aires: CIPPEC.

Robin, I. (2002). L'enquête PISA sur les compétences en lecture des élèves de 15 ans: Trois biais culturels en question. VEI Enjeux, 129(1), 65-91.

Schütz, A. (1976). Tiresias, or our knowledge of future events. In A. Schütz (Ed.), Tiresias, or our knowledge of future events (pp. 277-293). Den Haag: Martinus Nijhoff.

Schwartzman, S. (2013). Uses and abuses of education assessment in Brazil. Prospects, 43(3), 269-288. doi: https://doi.org/10.1007/s11125-013-9275-9

Scott, C. (2005, January). Measuring Up to the Measurement Problem The role of statistics. Evidence- based policy-making, London School of Economics. Research in Economics, United Kingdom. Retrieved from https://www.paris21.org/sites/default/files/MUMPS-full.pdf

Segone, M., & Pron, N. (2008). Bridging the gap. The role of Monitoring & Evaluation in Evidence-based policy making. UNICEF.

Strassheim, H., & Kettunen, P. (2014). When does evidence-based policy turn into policy-based evidence configurations, contexts and mechanisms. Evidence and Policy, 10(2), 259-277. doi: https://doi.org/10.1332/174426514X13990433991320

Turner, R., & Adams, R. J. (2007). The Programme for International Student Assessment: An Overview. Journal of Applied Measurement, 8(3), 237-248.

Descargas

Publicado

2018-10-31

Número

Vol. 6 Núm. 3 (2018): Temas de Investigación Educativa

Sección

Artículos

Licencia

Copyright (c) es propiedad de Sisyphus – Journal of Education. Sin embargo, alentamos que los artículos publicados en la revista se publiquen en otro lugar, siempre que se solicite el permiso de Sisyphus y los autores incorporen nuestra cita original y un enlace a nuestra página web.

Política de Autoarchivo
Los autores pueden autoarchivar la versión final publicada de sus artículos en repositorios institucionales, temáticos o páginas web personales e institucionales.

Suscriptor de DORA
El Instituto de Educação de la Universidade de Lisboa, editor de Sisyphus, es uno de los suscriptores de la Declaración de San Francisco sobre la Evaluación de la Investigación (DORA).