An Early Warning System to Detect At-risk Students: the First contribution of the New Goals LIS project

Artificial intelligence (AI) has impacted education in recent years as we can check easily on the web. There are many contributions related to tools, papers, and many promises to change the learning environments. We do not know where we will end. There is a huge investment on educational institutions and many companies are appearing related to EdTech since AI is (or is being) the next lucrative hot technology. 

eLearn Center performed a call in 2018 denoted as New Goals seeking a project to apply AI within the UOC Campus. We are an online university where every day a massive amount of data is produced and currently stored in the institutional data mart waiting for being exploited by our researchers. The call selected one project that started on February 2019 with the name LIS (Learning Intelligent System) [1]. The project aims to assist students in their educational processes and the main objective is to develop an adaptive system to be globally applicable at the UOC campus to help students to succeed in their learning process. The project intends to provide help to the student in terms of automatic feedback in assessment activities, recommendations in terms of learning paths, self-regulation or learning resources, and gamification techniques to improve the students’ engagement.

There is a large list of objectives and a high motivation for tackling all of them. However, the time is limited and the workforce as well. Currently, the core group is composed of four members: M. Elena Rodriguez, Ana Elena Guerrero, Abdulkadir Karadeniz, and me. However, we are happy to announce that the first contribution of the project is ready to be presented: an early warning system. The objective of such tool is to detect at-risk students by using data from the past and present and to warn the student and his or her teacher about the situation. Also, the system provides semi-automatic feedback as an early intervention mechanism in order to amend possible conditions of failure. 

The researchers invested a large effort in order to offer a solution applicable to the whole UOC campus. First, the backend system has been developed to extract the data from the data mart and apply different predictive models to predict the likelihood to pass a course. Note that, currently, the system works individually for each course. Currently, the predictive model uses profiling information of the students and the grades of the activities to perform the prediction. The backend system is capable of testing the models in the whole set of UOC courses (more than 1500 courses) and shows individual and aggregated results.  

Flow diagram of the Prediction Service based on the definition of a Predictive Model, the Data Mart information and students’ grades (CAR – Continuous Assessment Registry).

Second, the system offers an interface for piloting the system on individual courses. The different stakeholders have been considered: students, teachers and collaborators.  Different features are currently tested and analyzed: dashboards, feedback system based on prediction, types of feedback, expectations of the stakeholders, among others.  

Students‘ dashboard showing at-risk situation for Assessment Activity 1.

Teachers’ dashboard showing at-risk prediction and the grade for the Assessment Activity 1 and the at-risk color assignment for each student who signed the consent form.

The system has been tested among more than 1000 students during the past two semesters in five UOC courses. Teachers Cristina Pérez, Tona Monjo, Amal Elasri, Dalilis Escobar, Pau Cortadas, and Javier Panadero helped to test the system and we appreciate their help and proactive attitude towards testing the system. Also, students mostly appreciate the system since they are feeling a more personalized learning experience and care from teachers. Although the system may be mostly automated, there is a unique feature that currently the machine cannot replace and it is the expertise of the teachers on giving recommendations and help to students during the long journey until the end of the semester. A summary of the first pilot can be found in JENUI conference paper that can be found in [2] awarded with the best paper recognition or in the Applied Science journal version [3]. 

Video presentation of the JENUI conference paper awarded with the BEST PAPER recognition.

What’s next? Well, the system is alive and continues evolving. Next, a model to predict potential dropout is in the development roadmap. Also, a unique dashboard is ready for teachers to better manage the course. Useful or not. Who knows…? The next pilot will show it. 

And, we end this entry as we started the New Goals LIS project: 

“Welcome LIS at UOC, we expect that you succeed as students do.”

David Bañeres, CO-IP of the New Goals LIS Project

  1. Karadeniz, A.; Baneres, D.; Rodríguez, M.E.; Guerrero-Roldán, A.E. Enhancing ICT Personalized Education through a Learning Intelligent System. In Proceedings of the Online, Open and Flexible Higher Education Conference, Madrid, Spain, 16–18 October 2019; pp. 142–147.
  2. Guerrero-Roldán, A.E.; Rodríguez, M.E.; Baneres, D.; Pérez. C.; Panadero, J. Karadeniz, A. Hacia un sistema de detección temprana de estudiantes en riesgo en entornos de enseñanza-aprendizaje en línea. En Actas de las Jenui, vol. 5, July 2020.  Valencia, 2020; pp. 37-44
  3. Bañeres, D.; Rodríguez, M.E.; Guerrero-Roldán, A.E.; Karadeniz, A. An Early Warning System to Detect At-Risk Students in Online Higher Education. Appl. Sci. 2020, 10, 4427.

Header photo by Wes Hicks on Unsplash

David Bañeres
David Bañeres received a BSc in Computer Science (2001) and a Ph.D. in Computer Science (2008) from the Universitat Politècnica de Catalunya, Spain. He is currently a full-time lecturer at Universitat Oberta de Catalunya in the IT, Multimedia and Telecommunications department. His research interests include innovative e-learning systems such as intelligent tutoring systems, automated assessment, learning analytics and the application of AI in educational contexts. Currently, he is co-coordinating the development of an intelligent tutoring system and an early warning system for UOC Campus.

Leave a Reply