Web application for the analysis of assessment tests

  • Paula Prata Instituto de Telecomunicações; Departamento de Informática, Universidade da Beira Interior, Covilhã, Portugal
  • Luís Duarte Departamento de Informática, Universidade da Beira Interior, Covilhã, Portugal
  • Maria Eugénia Ferrão Universidade da Beira Interior, CEMAPRE - Centro de Matemática Aplicada à Previsão e Decisão Económica e Departamento de Matemática, Covilhã, Portugal
Keywords: Classical test theory, E-assessment;, Data analysis

Abstract

Introduction: An assessment test enables the evaluation of an individual’s competence or ability. Such tests are important for both teaching and professional training institutions, as well as for the recruiting of human resources in the enterprise.

Objectives: The present paper introduces the “Evaluate” web application, for the analysis of assessment tests.

Methods: The design and implementation of the application is described, which allows the management of assessment items, used to constitute evaluation tests, upon which results the main descriptive statistic values used under the classical test theory in the analysis of assessment tests are calculated. The application was developed in Python, within the Django framework, and tested with real assessment tests.

Results: Scores are assigned to each assessment item, and various statistics — such as difficulty and discrimination index, point-biserial correlation, test internal consistency coefficient — can be obtained upon the answers of the subjects, as well as a graphic analysis of the performance of each subject on each assessment item, as well as on the test as a whole.

Conclusions: The “Evaluate” application makes a meaningful contribution to a better knowledge of assessment tools used in competence evaluation, by allowing the detection of inconsistencies and the consequent improvement in the process.

References

Costa, P. M. & Ferrão, M. E. (2015). On the complementarity of the classical test theory and item response models: Item difficulty estimates. Ensaio: Avaliação e Políticas Públicas em Educação, 23(88), 593–610. http://dx.doi.org/10.1590/S0104-40362015000300003

Django (2018). In Django Docs Model instance reference. Retrieved from: https://docs.djangoproject.com/en/2.0/ref/models/instances/

Django (2014). In Nested SQL queries in Django. Retrieved from: http://www.lexev.org/en/2014/nested-sql-queries-django/

Ferrão, M. E. (2010). E-assessment within the Bologna paradigm: evidence from Portugal. Assessment & Evaluation in Higher Education, 35(7), 819–830. Retrieved from: https://www.tandfonline.com/doi/abs/10.1080/02602930903060990

Ferrão, M.E., & Prata, P. (2014). Item Response Models in Computerized Adaptive Testing: A Simulation Study. In: Murgante, B., et al. (Eds.) Computational Science and Its Applications – ICCSA 2014, Lecture Notes in Computer Science, 8581, 552-565. Cham: Springer Nature. https://doi.org/10.1007/978-3-319-09150-1_40

Ferrão, M. E. & Prata, P. (2015). Invited Paper Session 079 Statistical Methods in Computerized Adaptive Testing: Statistical issues in the item bank development for adaptive testing. In 60th ISI World Statistics Congress.

Guilford, J.P. & Fruchter, B. (1978). Fundamental Statistics in Psychology and Education. 6th ed. New York: McGraw-Hill.

Hambleton, R., Swaminathan, H. & Rogers, H. J. (1991). Fundamentals of Item Response Theory. Newbury Park, California: SAGE Publications.

Kirsch, I., & Lennon, M. L. (2017). PIAAC : a new design for a new era. Large-Scale Assessments Education, 5(11), 1-22. https://doi.org/10.1186/s40536-017-0046-6

Lord, F. M. & Novick, M. R (1968). Statistical Theories of Mental Test Scores. Oxford, England: Addison-Wesley.

Ramsay, C. (2018). In Schreyer Institute Item Analysis and Difficulty. Retrieved from: http://sites.psu.edu/itemanalysis/difficulty-2/

Ratner, B. (2018). The Correlation Coefficient: Definition. DM STAT-1 CONSULTING [Newsletter]. Retrieved from: http://www.dmstat1.com/res/TheCorrelationCoefficientDefined.html

Strang, K. D. (2016). Do the critical success factors from learning analytics predict student outcomes? Journal of Educational Technology, 44(3), 273–299. https://doi.org/10.1177/0047239515615850

Testing Assisté par Ordinateur (2018). Open Source Assessment Platform -TAO Testing. Retrieved from: https://www.taotesting.com/

University of Cambridge.The Psychometrics Centre/Concerto. (2018). The Open-Source Online Adaptive Testing Platform. Retrieved from: https://www.psychometrics.cam.ac.uk/newconcerto

Vieira, S. (2015). Alfa de Cronbach. Retrieved from: http://soniavieira.blogspot.pt/2015/10/alfa-de-cronbach.html

Published
2018-09-27
Section
Engineering, Technology, Management and Tourism