Psychometric properties of the ActEval questionnaire on university teachers' assessment activity
DOI:
https://doi.org/10.7203/relieve.19.1.2611Keywords:
Skills assessment at University, perception of the teacher, construct validity, internal consistency
Abstract
This article presents the results of the empirical validation of the questionnaire ActEval (Self-Report on the Evaluation Activity of university teachers). ActEval attempts to identify how important teachers consider a variety of assessment tasks, whether they feel competent, and to what extent they use them in their daily practice, in line with the new competences under the European Higher Education Area. Validation is studied using Non-Metric Multidimensional Scaling (PROXSCAL) and the reliability results show high levels of internal consistency. The data suggest that some items in the questionnaire should be reviewed in order to obtain an effective tool for the analysis of evaluation practice.
Author Biographies
Chantal Biencinto, Universidad Complutense de Madrid
Elvira Carpintero, Universidad Complutense de Madrid
Mercedes García-García, Universidad Complutense de Madrid
References
Adam, C. & King, K. (1995). Towards a framework for student self-assessment. Innovations in Education and Training International, 32, 336–343.
http://dx.doi.org/10.1080/1355800950320405
Álvarez-Rojo, Víctor; Asensio-Muñoz Inmaculada; Clares, José; del-Frago, Rakel; García-Lupión, Beatriz; García-Nieto, Narciso; García-García, Mercedes; Gil, Javier; González-González, Daniel; Guardia, Soledad; Ibarra, Marisol; López-Fuentes, Rafael; Rodríguez-Diéguez, Antonio; Rodríguez-Gómez, Gregorio; Rodríguez-Santero, Javier; Romero, Soledad; Salmerón, Purificación (2009). Perfiles docentes para el espacio europeo de educación superior (EEES) en el ámbito universitario español. RELIEVE, v. 15, n. 1. http://www.uv.es/RELIEVE/v15n1/RELIEVEv15n1_1.htm.
Andreu-Andrés, MªA. & García-Casas, M. (2006). Evaluación, coevaluación y autoevaluación del trabajo en grupo en la lectura de mapas topográficos. En: F. Watts & A. García-Carbonell (Eds.): La evaluación compartida: investigación multidisciplinar. Valencia, Universidad Politécnica de Valencia, pp. 69- 90.
Arenas, C. & Cuadras, C.M. (2002). Recent statistical methods based on distances. Contributions to Science, 2 (2), 183-191.
Biggs, J. y Tang, C. (2009). Teaching for Quality Learning at University. Maidenhead: Open University Press, McGraw-Hill.
Bloxham, S. y Boyd, P. (2007). Developing Effective Assesment in Higher Education. A Practical Guide. Londres: Open University Press, McGraw-Hill.
Boud, D. (2006). Foreword. En: C. Bryan & K. Clegg. Innovative Assessment in Higher Education (pp. xvii-xix). New York: Routledge.
Boud, D. (2010). Assessment 2020: Seven propositions for assessment reform in higher education. Sydney: Australian Learning and Teaching Council.
Bozu, Z. & Canto, P.J. (2009). El profesorado universitario en la sociedad del conocimiento: competencias profesionales docentes. Revista de Formación e Innovación Educativa Universitaria, 2, 87-97.
Briggs, C.L., Stark, J.S. & Rowland- Poplawsky, J. (2003). How Do We Know a "Continuous Planning" Academic Program When We See One? The Journal of Higher Education, 74 (4), 361-385.
http://dx.doi.org/10.1353/jhe.2003.0028
Busing, F. (1997). PROXSCAL: A multidimensional scaling program. En: A. Bandilla & F.
Faulbaum. Softstat 97. Advances in Statistical Software. The 9th Conference on the Scientific Use of Statistical Software, 1997, Heidelberg.
Butcher, C., Davies, C. & Highton, M. (2006). Designing learning from module outline to effective teaching. Abindgon: Routledge.
Carballo, J.L., Pérez-Jover, MªV., Espada, J.P., Orgilés, M. & Piqueras, J.A. (2012). Propiedades psicométricas de la escala multidimensional de expresión social para la evaluación de habilidades sociales en el contexto de internet. Psicothema, 24, 121-126.
Carless, D. (2003). Learning-oriented assessment. Comunicación presentada en la Evaluation and Assessment. Conference, University of South Australia, Adelaide, 25 de noviembre de 2003.
Carless, D. (2011). Learning-oriented assessment and the development of student learning capacities. Ponencia presentada en el Congreso Internacional EVALtrends. Cádiz, 9 al 11 de marzo.
Biencinto, Chantal; Carpintero, Elvira & García-García, Mercedes (2013). Propiedades psicométricas del cuestionario ActEval sobre la actividad evaluadora del profesorado universitario. RELIEVE, v. 19, n. 1, art. 2. DOI:10.7203/relieve.19.1.2611.
http://dx.doi.org/10.7203/relieve.19.1.2611
Carless, D., Joughin, G. & Liu, N.F. (2006). How Assessment supports learning: learningoriented assessment in action. Hong Kong University Press.
http://dx.doi.org/10.5790/hongkong/9789622098237.001.0001
Chen, Y.M. (2008). Learning to Self-assess Oral Performance in English: A Longitudinal Case Study. Language Teaching Research, 12 (2), 235-262.
http://dx.doi.org/10.1177/1362168807086293
Cheng, W. & Warren, M. (1997). Having second thoughts: student perceptions before and after a peer assessment exercise. Studies in Higher Education, 22, 233–239.
http://dx.doi.org/10.1080/03075079712331381064
Cuadras, C.M. (1996). Métodos de análisis multivariante. Barcelona: EUB.
De Leeuw, J., & Heiser, W.J. (1980). Multidimensional Scaling with Restrictions on the Configuration. En P.R. Krishnaiah (Ed.), Multivariate Analysis, Vol. 5, Amsterdam, Holland: North Holland Publishing Company.
Dochy, F., Segers. M. & Sluijsmans, D. (1999). The use of self-, peer and coassessment in higher education: a review. Studies in Higher Education, 24 (3), 331–350.
http://dx.doi.org/10.1080/03075079912331379935
Falchikov, N. (2005). Improving Assessment through Student Involvement: Practical solutions for aiding learning in higher and further education. Abindgon: Routledge.
Gibbs, G. (2006). How assessment frames student learning. En Cordelia Bryan & Karen Clegg, Innovative Assessment in higher education (23-36). Abingdon: Routledge.
Gibbs, G. & Simpson, C. (2004). Conditions under which assessment supports students' learning. Learning and Teaching in Higher Education, Issue 1.
Gower, J.C. (1971). A general coefficient of similarity and some of its properties. Biometrics, 27, 857-874.
http://dx.doi.org/10.2307/2528823
Hanrahan, S. & Isaacs, G. (2001). Assessing self- and peer assessment: the students' views. Higher Education Research & Development, 20 (1), 53–70.
http://dx.doi.org/10.1080/07294360123776
Ibarra, M.S. (Dir.) (2008). EvalCOMIX: Evaluación de competencias en un contexto de aprendizaje mixto. Cádiz: Servicio de Publicaciones de la Universidad de Cádiz.
Ibarra, M.S. & Rodríguez, G. (2010). Aproximación al discurso dominante sobre la evaluación del aprendizaje en la universidad. Revista de Educación, 351, 385-407.
Ibarra, M.S., Rodríguez, G. & Gómez, M.A. (2012). La evaluación entre iguales: beneficios y estrategias para su práctica en la universidad. Revista de Educación, 359.
Ibarra Sáiz, María Soledad; Rodríguez-Gómez, Gregorio & Gómez-Ruiz, Miguel-Ángel (2010). La planificación basada en competencias en los másteres oficiales: un reto para el profesorado universitario. RELIEVE, v. 16, n. 1. http://www.uv.es/RELIEVE/v16n1/RELIEVEv16n1_6.htm
Knight, P. (1995). Assessment for Learning in Higher Education. London: Kogan Page.
Kruskal, J.B. (1964). Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis. Psychometrika, 29, 1–28.
http://dx.doi.org/10.1007/BF02289565
Kruskal, J.B. (1964). Nonmetric multidimensional scaling: a numerical method. Psychometrika, 29, 115–129.
http://dx.doi.org/10.1007/BF02289694
Kruskal, J.B. & Wish, M. (1978). Multidimensional Scaling. London: Sage.
Litwin, M. (1995). How to measure survey reliability and validity. London: Sage.
Ljungman, A.G. & Silén, C. (2008). Examination involving students as peer-examiners. Assessment & Evaluation in Higher Education, 33 (3), 289–300.
http://dx.doi.org/10.1080/02602930701293306
López Pastor, V.M. (Coord.) (2009). La evaluación formativa y compartida en docencia universitaria: propuestas, técnicas, instrumentos y experiencias. Madrid: Narcea.
Meulman, J. & Heiser, W.J. (2005). SPSS Categories 14. Chicago: SPSS Inc.
Monereo, C. (2009). La enseñanza de tareas de autorregulación a través de tareas auténticas. En P. Díez (Coord.), Nueva funciones de la evaluación. Madrid: Ministerio de Educación.
Nunnally, J.C. (1978). Psychometric theory. New York: McGraw-Hill Book.
Olivia, P. & Henson, K. (1980). What are the Essential Generic Teaching Competencies?. Theory into Practice, 9 (2), 117-121.
http://dx.doi.org/10.1080/00405848009542884
Osterlind, S.J. (1989). Constructing test items. Boston: Kluwer.
http://dx.doi.org/10.1007/978-94-009-1071-3
Pavoine, S., Vallet, J., Dufour, A.B., Gachet, S. & Daniel, H. (2009). On the challenge of treating various types of variables: application for improving the measurement of functional diversity. Oikos, 118, 391-402.
http://dx.doi.org/10.1111/j.1600-0706.2008.16668.x
Peña, D. (2002). Análisis de datos multivariantes. Madrid: McGraw-Hill.
Pope, N. (2005). The impact of stress in selfand peer assessment. Assessment & Evaluation in Higher Education, 30 (3), 51–63.
http://dx.doi.org/10.1080/0260293042003243896
Quesada, V., Rodríguez, G. & Ibarra, M.S. (2013, en prensa). ActEval: un instrumento para el análisis y reflexión sobre la actividad evaluadora del profesorado universitario. Revista de Educación, 362, Septiembre-diciembre.
Rodríguez, G. (Dir.) (2009). EvalHIDA: Evaluación de Competencias con Herramientas de Interacción Dialógica Asíncronas (foros, blogs y wikis). Cádiz: Servicio de Publicaciones de la Universidad de Cádiz.
Rodríguez, G., Ibarra, M.S. & Gómez, M.A. (2011). e-Autoevaluación en la universidad: Un reto para profesores/as y estudiantes. Revista de Educación, 356, 401-430.
Sánchez, J. (2011). Evaluación de los aprendizajes universitarios: una comparación sobre sus posibilidades y limitaciones en el Espacio Europeo de Educación Superior. Revista de Formación e Innovación Educativa Universitaria, 4 (1), 40-54.
Takane, Y., Young, F.W. & De Leeuw, J. (1977). Nonmetric individual differences multidimensional scaling: an alternating least squares method with optimal scaling features. Psychometrika, 42, 593-600.
http://dx.doi.org/10.1007/BF02293745
Villardón, L. (2006). Evaluación del aprendizaje para promover el desarrollo de competencias. Educación Siglo XXI, 24, 57-76.
Yalcin, I. & Amemiya, Y. (2001). Nonlinear Factor Analysis as a Statistical Method. Statistical Science, 16, 275-294.
Downloads
Published
Issue
Section
License
The authors grant non-exclusive rights of exploitation of works published to RELIEVE and consent to be distributed under the Creative Commons Attribution-Noncommercial Use 4.0 International License (CC-BY-NC 4.0), which allows third parties to use the published material whenever the authorship of the work and the source of publication is mentioned, and it is used for non-commercial purposes.
The authors can reach other additional and independent contractual agreements, for the non-exclusive distribution of the version of the work published in this journal (for example, by including it in an institutional repository or publishing it in a book), as long as it is clearly stated that the Original source of publication is this magazine.
Authors are encouraged to disseminate their work after it has been published, through the internet (for example, in institutional archives online or on its website) which can generate interesting exchanges and increase work appointments.
The fact of sending your paper to RELIEVE implies that you accept these conditions.
Language
Information