tamara tupper

article

Yılmaz, F. G. K., & Yılmaz, R. (2021). Learning Analytics intervention improves students’ engagement in online learning. Technology, Knowledge, and Learning, 27(2), 449–460. https://doi.org/10.1007/s10758-021-09547-w

article summary and review

summary

Student engagement is one of the main challenges encountered in digital learning. In an eLearning environment, learners may encounter problems with behavioral, cognitive, and emotional engagement. Personalized metacognition feedback can help solve problems associated with student engagement, which can be generated from Learning Analytics (LA). Two key components make up the personalized metacognitive feedback support used in this study: (a) Learning analytics reports created with data obtained from students’ weekly learning management system usage (b) Recommendation messages prepared and personalized for each participant based on learning analytics reports. The students’ engagement scale used as a preliminary and post-test has provided the data of the study. The study results have shown that student involvement was significantly higher in the experimental group than in the control group. The research findings showed that providing students with personalized metacognition feedback using Learning Analytics in eLearning would increase their engagement. Various suggestions have been suggested for teachers, administrators, and researchers based on the findings from the research.

Research question: Is there a statistically significant difference in experimental and control group students’ engagement scores arising from the personalized metacognitive feedback support based on learning analytics? (Yilmaz & Yilmas, 2021)

research method

In this study, the experimental design has been used with a pretest and posttest control group. The experimental and control groups were divided randomly among the participants. The experimental group had 33 students and the control group had 35 students. There were 63% females and 37% males in the first cohort of 68 students. Participants were randomly assigned to the experimental and control groups at the beginning of the research process. Participants were then informed of the procedure and scales had been used as a preliminary measure. The trial was initiated for 12 weeks after the pretest had been performed. Learning Analytics metacognitive feedback was administered to the experimental groups of students. Students in the control group were not provided with this support. The student engagement scale was used as a posttest at the end of the application process. Therefore, the pretest and posttest in the experimental and control groups were compared, and the hypotheses of the study were tested.

The study was conducted on students enrolled in the Computing II course at a university during the spring term. The course was taught through online learning, and the students were introduced to the electronic spreadsheet program as part of their coursework. The research used Moodle as the Learning Management System (LMS). The teacher would add weekly lecture videos, e-books, and exercises related to the subject to Moodle every week. Additionally, a weekly quiz was uploaded for students to test themselves on the subject matter. The LMS environment also had a forum tool that allowed students to communicate and collaborate with each other and the teacher. In the forum environment, students shared information and discussed the course exercises.

The study utilized personalized metacognitive feedback support, comprised of two fundamental components: (a) Learning Analytics reports generated from data collected through students’ weekly LMS usage and (b) personalized recommendation messages based on the learning analytics reports. Learning Analytics reports helped students identify which weekly learning content they were most and least engaged with, and recognize their areas of success and failure. This created awareness and helped students develop metacognition, which involves understanding one’s own learning process. With the Learning Analytics reports, students could see which of the weekly learning content they were more engaged with and which content they were less engaged with, and they could recognize the issues they have succeeded and failed. This lead to awareness for students by means of Learning Analytics reports. Thus, efforts were made to develop a student’s awareness of their own learning process, which is contained in the definition of metacognition.

data collected and analyzed

Metacognition is a valuable strategy for assessing one’s own cognitive process. This strategy involves three stages: planning, monitoring, and evaluation. In the planning phase, a student determines appropriate learning strategies for themselves and plans for their own learning process. In the monitoring phase, they keep track of whether they are acting in accordance with their planned strategies and if things are going as planned. In the evaluation phase, the student evaluates their own learning process, identifies areas of success and failure, and makes new plans to address any learning deficiencies. This approach to learning has been shown to be effective in improving student engagement and can be facilitated using personalized metacognitive feedback generated from learning analytics data.

The personalized metacognitive feedback support, which included recommendation messages based on learning analytics reports, helped increase students’ awareness of their own learning process. This led to improved student engagement and metacognitive awareness. At the end of the 12-week experimental process, the posttest student engagement scale was administered to both the experimental and control groups. In the study, the normality of the scores obtained from the students’ engagement scale was examined, and a Kurtosis and Skewness normality test was performed. The results of the test showed that the data had a normal distribution. Therefore, parametric tests were used for data analysis. In line with the first research question of the study, the engagement scale scores of students were compared.

findings and limitations

In the study conducted by Yılmaz and Yılmaz (2021), the challenge of student engagement in digital learning was addressed. The study found that personalized metacognitive feedback generated from Learning Analytics (LA) can help solve problems related to student engagement. The personalized metacognitive feedback support used in this study was comprised of two key components: (a) Learning analytics reports created using data obtained from students’ weekly Learning Management System usage and (b) personalized recommendation messages prepared for each participant based on learning analytics reports. The study found that providing students with personalized metacognition feedback using LA in eLearning significantly increased their engagement.

The study examined whether the scores obtained from the students’ engagement scale showed normal distribution by performing a Kurtosis and Skewness normality test. The data was found to have a normal distribution, and parametric tests were used to analyze it. The first research question of the study focused on comparing the students’ engagement scale scores. To achieve this, the students’ engagement scale pretest of both experimental and control groups was kept constant, and their posttest scores were compared using ANCOVA. The study aimed to test the effectiveness of providing learning analytics-based metacognitive feedback, which was the independent variable of the research.

Based on the research findings, it is recommended to use similar personalized metacognitive feedback designs in similar eLearning environments and contexts to improve student engagement. It is important to include Learning Analytics reports in the feedback to provide students with a visual representation of their learning performance and process. However, it is also important to note that simply providing Learning Analytics reports may not be effective for students who have not developed self-directed learning skills, as they may not know how to use the information provided. Therefore, it is suggested that teachers and administrators provide additional support and guidance to help students understand and use the feedback effectively.

According to the researchers, students who have not developed self-directed learning skills may not know what to do as a result, even if they examine the learning analytics reports (Karaoglan Yilmaz & Yilmaz, 2020b; Karaoglan Yilmaz, 2021; Schumacher, & Ifenthaler, 2021).

It is crucial to provide personalized recommendations and guidance to learners along with the learning analytics reports in the feedback. However, it is essential to ensure that these recommendations and guidance are not in the form of directly giving information or telling the solution. Instead, it should encourage learners to reflect on their learning process and guide them towards finding a solution on their own. The importance of providing metacognitive support to learners through feedback is highlighted in this study. Feedback messages should guide, motivate, and reinforce an individual’s learning. In this study, personalized feedback was provided to learners in the form of learning analytics reports on their performance. In future research, feedback could be given based on the average performance of the class or the best-performing learners in the class. This approach would provide students with valuable insights into their own learning process and help them develop metacognitive awareness. It is important to design feedback messages that serve as effective guidance for learners, and learning analytics can be a valuable tool in achieving this goal.

It is important to note that the research conducted in this study has some limitations. The study was limited to a sample of 68 university students, and further research is necessary to determine the generalizability of the research findings. Additionally, similar research can be carried out for different courses and for students at different levels of education. It is important to consider that the self-directed learning skills of university students are generally more developed than those of middle and high school students. Therefore, it may be necessary to provide additional external support for secondary and high school students in online learning environments. Further research can help to determine the effectiveness of personalized metacognitive feedback support using Learning Analytics in different educational contexts and for different student populations.

impact and implications for the field

In online learning systems, the teacher may find it challenging to provide personalized feedback to each learner due to the considerable number of students. As a result, learners may encounter issues such as feeling lost or not knowing what to do, which can ultimately lead to decreased engagement over time. These problems can be especially problematic in a digital learning environment, where learners may struggle with cognitive, behavioral, and emotional engagement. If left unaddressed, these difficulties may ultimately lead to student failure in online learning.

In online learning environments, learners may face challenges with engagement and may require external support and guidance. To address this issue, metacognitive feedback can be provided to students. The purpose of metacognitive feedback is to provide learners with information about their current learning process and to offer recommendations and guidance to improve their learning outcomes. However, it can be challenging for teachers to determine the current status of each student, particularly in online learning environments with many students. In such cases, learning analytics can be a useful tool.

Learning Analytics (LA) enables teachers to obtain insights into students’ engagement and learning progress, thus helping them to provide personalized metacognitive feedback to students. This feedback can aid learners in identifying their strengths and weaknesses, as well as areas for improvement. LA reports generated from data collected through students’ weekly learning management system usage, coupled with personalized recommendation messages, can help learners develop metacognitive skills and improve their engagement in online learning.

Overall, the use of LA and personalized metacognitive feedback can benefit both learners and teachers in online learning environments, enabling learners to achieve better learning outcomes and teachers to provide more effective support and guidance.