Web application for the analysis of assessment tests
Keywords:Classical test theory, E-assessment;, Data analysis
Introduction: An assessment test enables the evaluation of an individual’s competence or ability. Such tests are important for both teaching and professional training institutions, as well as for the recruiting of human resources in the enterprise.
Objectives: The present paper introduces the “Evaluate” web application, for the analysis of assessment tests.
Methods: The design and implementation of the application is described, which allows the management of assessment items, used to constitute evaluation tests, upon which results the main descriptive statistic values used under the classical test theory in the analysis of assessment tests are calculated. The application was developed in Python, within the Django framework, and tested with real assessment tests.
Results: Scores are assigned to each assessment item, and various statistics — such as difficulty and discrimination index, point-biserial correlation, test internal consistency coefficient — can be obtained upon the answers of the subjects, as well as a graphic analysis of the performance of each subject on each assessment item, as well as on the test as a whole.
Conclusions: The “Evaluate” application makes a meaningful contribution to a better knowledge of assessment tools used in competence evaluation, by allowing the detection of inconsistencies and the consequent improvement in the process.
Costa, P. M. & Ferrão, M. E. (2015). On the complementarity of the classical test theory and item response models: Item difficulty estimates. Ensaio: Avaliação e Políticas Públicas em Educação, 23(88), 593–610. http://dx.doi.org/10.1590/S0104-40362015000300003
Django (2018). In Django Docs Model instance reference. Retrieved from: https://docs.djangoproject.com/en/2.0/ref/models/instances/
Django (2014). In Nested SQL queries in Django. Retrieved from: http://www.lexev.org/en/2014/nested-sql-queries-django/
Ferrão, M. E. (2010). E-assessment within the Bologna paradigm: evidence from Portugal. Assessment & Evaluation in Higher Education, 35(7), 819–830. Retrieved from: https://www.tandfonline.com/doi/abs/10.1080/02602930903060990
Ferrão, M.E., & Prata, P. (2014). Item Response Models in Computerized Adaptive Testing: A Simulation Study. In: Murgante, B., et al. (Eds.) Computational Science and Its Applications – ICCSA 2014, Lecture Notes in Computer Science, 8581, 552-565. Cham: Springer Nature. https://doi.org/10.1007/978-3-319-09150-1_40
Ferrão, M. E. & Prata, P. (2015). Invited Paper Session 079 Statistical Methods in Computerized Adaptive Testing: Statistical issues in the item bank development for adaptive testing. In 60th ISI World Statistics Congress.
Guilford, J.P. & Fruchter, B. (1978). Fundamental Statistics in Psychology and Education. 6th ed. New York: McGraw-Hill.
Hambleton, R., Swaminathan, H. & Rogers, H. J. (1991). Fundamentals of Item Response Theory. Newbury Park, California: SAGE Publications.
Kirsch, I., & Lennon, M. L. (2017). PIAAC : a new design for a new era. Large-Scale Assessments Education, 5(11), 1-22. https://doi.org/10.1186/s40536-017-0046-6
Lord, F. M. & Novick, M. R (1968). Statistical Theories of Mental Test Scores. Oxford, England: Addison-Wesley.
Ramsay, C. (2018). In Schreyer Institute Item Analysis and Difficulty. Retrieved from: http://sites.psu.edu/itemanalysis/difficulty-2/
Ratner, B. (2018). The Correlation Coefficient: Definition. DM STAT-1 CONSULTING [Newsletter]. Retrieved from: http://www.dmstat1.com/res/TheCorrelationCoefficientDefined.html
Strang, K. D. (2016). Do the critical success factors from learning analytics predict student outcomes? Journal of Educational Technology, 44(3), 273–299. https://doi.org/10.1177/0047239515615850
Testing Assisté par Ordinateur (2018). Open Source Assessment Platform -TAO Testing. Retrieved from: https://www.taotesting.com/
University of Cambridge.The Psychometrics Centre/Concerto. (2018). The Open-Source Online Adaptive Testing Platform. Retrieved from: https://www.psychometrics.cam.ac.uk/newconcerto
Vieira, S. (2015). Alfa de Cronbach. Retrieved from: http://soniavieira.blogspot.pt/2015/10/alfa-de-cronbach.html
How to Cite
Authors who submit proposals for this journal agree to the following terms:
a) Articles are published under the Licença Creative Commons (CC BY 4.0), in full open-access, without any cost or fees of any kind to the author or the reader;
b) The authors retain copyright and grant the journal right of first publication, allowing the free sharing of work, provided it is correctly attributed the authorship and initial publication in this journal;
c) The authors are permitted to take on additional contracts separately for non-exclusive distribution of the version of the work published in this journal (eg, post it to an institutional repository or as a book), with an acknowledgment of its initial publication in this journal;
d) Authors are permitted and encouraged to publish and distribute their work online (eg, in institutional repositories or on their website) as it can lead to productive exchanges, as well as increase the impact and citation of published work
Documents required for submission