-
-
Author(s): Kaliisa, Rogers; Jivet, Ioana; Prinsloo, Paul
Title: A checklist to guide the planning, designing, implementation, and evaluation of learning analytics dashboards
In: International Journal of Educational Technology in Higher Education, 20 (2023) , S. 28
DOI: 10.1186/s41239-023-00394-6
URL: https://educationaltechnologyjournal.springeropen.com/articles/10.1186/s41239-023-00394-6
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Benutzeroberfläche; Datenanalyse; Design; E-Learning; Empfehlung; Empirische Forschung; Evaluation; Implementierung; Lehrer; Lernprozess; Planung; Systematic Review; Tool; Visualisierung
Abstract (english): Higher education institutions are moving to design and implement teacher-facing learning analytics (LA) dashboards with the hope that instructors can extract deep insights about student learning and make informed decisions to improve their teaching. While much attention has been paid to developing teacher-facing dashboards, less is known about how they are designed, implemented and evaluated. This paper presents a systematic literature review of existing studies reporting on teacher-facing LA dashboards. Out of the 1968 articles retrieved from several databases, 50 articles were included in the final analysis. Guided by several frameworks, articles were coded based on the following dimensions: purpose, theoretical grounding, stakeholder involvement, ethics and privacy, design, implementation, and evaluation criteria. The findings show that most dashboards are designed to increase teachers' awareness but with limited actionable insights to allow intervention. Moreover, while teachers are involved in the design process, this is mainly at the exploratory/problem definition stage, with little input beyond this stage. Most dashboards were prescriptive, less customisable, and implicit about the theoretical constructs behind their designs. In addition, dashboards are deployed at prototype and pilot stages, and the evaluation is dominated by self-reports and users' reactions with limited focus on changes to teaching and learning. Besides, only one study considered privacy as a design requirement. Based on the findings of the study and synthesis of existing literature, we propose a four-dimensional checklist for planning, designing, implementing and evaluating LA dashboards. (DIPF/Orig.)
DIPF-Departments: Informationszentrum Bildung
-
-
Author(s): Zesch, Torsten; Horbach, Andrea; Zehner, Fabian
Title: To score or not to score. Factors influencing performance and feasibility of automatic content scoring of text responses
In: Educational Measurement: Issues and Practice, 42 (2023) 1, S. 44-58
DOI: 10.1111/emip.12544
URL: https://onlinelibrary.wiley.com/doi/10.1111/emip.12544
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Beitrag in Sonderheft
Language: Englisch
Keywords: Antwort; Automatisierung; Bewertung; Einflussfaktor; Inhalt; Leistung; Text; Tool; Verfahren
Abstract (english): In this article, we systematize the factors influencing performance and feasibility of automatic content scoring methods for short text responses. We argue that performance (i.e., how well an automatic system agrees with human judgments) mainly depends on the linguistic variance seen in the responses and that this variance is indirectly influenced by other factors such as target population or input modality. Extending previous work, we distinguish conceptual, realization, and nonconformity variance, which are differentially impacted by the various factors. While conceptual variance relates to different concepts embedded in the text responses, realization variance refers to their diverse manifestation through natural language. Nonconformity variance is added by aberrant response behavior. Furthermore, besides its performance, the feasibility of using an automatic scoring system depends on external factors, such as ethical or computational constraints, which influence whether a system with a given performance is accepted by stakeholders. Our work provides (i) a framework for assessment practitioners to decide a priori whether automatic content scoring can be successfully applied in a given setup as well as (ii) new empirical findings and the integration of empirical findings from the literature on factors that influence automatic systems' performance. (DIPF/Orig.)
DIPF-Departments: Lehr und Lernqualität in Bildungseinrichtungen
-
-
Author(s): Tilgner, Renate; Massar, Tamara; Schulte, Gwendolyn
Title: Barrierefrei studieren? Linktipps des Deutschen Bildungsservers
In: Voß-Nakkour, Sarah; Rustemeier, Linda; Möhring, Monika Maria; Deitmer, Andreas; Grimminger, Sanja (Hrsg.): Digitale Barrierefreiheit in der Bildung weiterdenken.: Innovative Impulse aus Praxis, Technik und Didaktik, Frankfurt am Main: Universitätsbibliothek Johann Christian Senckenberg, 2023 , S. 282-288
URL: https://publikationen.ub.uni-frankfurt.de/opus4/frontdoor/deliver/index/docId/62773/file/Sammelband_Digitale_Barrierefreiheit.pdf#page=282
Publication Type: 4. Beiträge in Sammelbänden; Tagungsband/Konferenzbeitrag/Proceedings
Language: Deutsch
Keywords: Barrierefreiheit; Behinderung; Digitalisierung; E-Learning; Hochschule; Hochschullehre; Inklusion; Rahmenrichtlinien; Rechtsgrundlage; Studium; Tool
Abstract: Die Hochschulen sind rechtlich dazu verpflichtet, Studierende mit Behinderung nicht zu benachteiligen, wobei insbesondere Prüfungen für Studierenden eine hohe Relevanz haben. Vor allem bei digitalen Prüfungen ist es sinnvoll, diese proaktiv von vornherein barrierefrei zu gestalten und nicht nur im Nachhinein - im Rahmen des Nachteilsausgleichs - individuell anzupassen. Bei digitalen Prüfungen steigt die Bedeutung einer barrierefreien technischen und formalen Gestaltung. Zudem ist dies auch für Präsenzprüfungen relevant, da hier die Möglichkeit besteht, papierbasierte Prüfungen in digitaler Form barrierefrei anzubieten. Zusätzlich können didaktische und organisatorische Aspekte die Barrierefreiheit erhöhen. (DIPF/Autor)
DIPF-Departments: Informationszentrum Bildung
-
-
Author(s): Di Mitri, Daniele; Schneider, Jan; Drachsler, Hendrik
Title: Keep me in the loop. Real-time feedback with multimodal data
In: International Journal of Artificial Intelligence in Education, 32 (2022) 4, S. 1093-1118
DOI: 10.1007/s40593-021-00281-z
URL: https://link.springer.com/article/10.1007/s40593-021-00281-z
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Datenanalyse; Datenverarbeitung; Effektivität; Empirische Untersuchung; Feedback; Fehler; Fertigkeit; Fragebogen; Lernprozess; Medizin; Psychomotorik; Simulation; System; Technologie; Teilnehmer; Tool; Training
Abstract (english): This paper describes the CPR Tutor, a real-time multimodal feedback system for cardiopulmonary resuscitation (CPR) training. The CPR Tutor detects training mistakes using recurrent neural networks. The CPR Tutor automatically recognises and assesses the quality of the chest compressions according to five CPR performance indicators. It detects training mistakes in real-time by analysing a multimodal data stream consisting of kinematic and electromyographic data. Based on this assessment, the CPR Tutor provides audio feedback to correct the most critical mistakes and improve the CPR performance. The mistake detection models of the CPR Tutor were trained using a dataset from 10 experts. Hence, we tested the validity of the CPR Tutor and the impact of its feedback functionality in a user study involving additional 10 participants. The CPR Tutor pushes forward the current state of the art of real-time multimodal tutors by providing: (1) an architecture design, (2) a methodological approach for delivering real-time feedback using multimodal data and (3) a field study on real-time feedback for CPR training. This paper details the results of a field study by quantitatively measuring the impact of the CPR Tutor feedback on the performance indicators and qualitatively analysing the participants' questionnaire answers. (DIPF/Orig.)
DIPF-Departments: Informationszentrum Bildung
-
-
Author(s): Ruipérez-Valiente, José A.; Martínez-Maldonado, Roberto; Di Mitri, Daniele; Schneider, Jan
Title: From sensor data to educational insights
In: Sensors, 22 (2022) 21, S. 8556
DOI: 10.3390/s22218556
URL: https://www.mdpi.com/1424-8220/22/21/8556
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Datenanalyse; Datenerfassung; Künstliche Intelligenz; Lernprozess; Logdatei; Mensch-Maschine-Kommunikation; Messverfahren; Tool; Übersicht; Virtuelles Lernen
Abstract: Technology is gradually becoming an integral part of learning at all levels of educational. This includes the now pervasive presence of virtual learning environments (VLEs) and the inclusion of interactive devices used or worn by learners or that are present in the physical classroom environment. These new technology-rich educational ecosystems have greatly facilitated data capture about learners. Thus, several research areas, such as learning analytics (LA), educational data mining (EDM), and artificial intelligence in education (AIED), have grown exponentially during the last decade, with multiple venues supporting this research [1]. However, the inferences about learning that can be made by solely analyzing trace data from VLEs are limited, since logged data do not commonly provide a complete view of the learning experience [2]. Therefore, research communities are moving beyond the data obtained from VLEs and other online tools by incorporating data from external sources such as sensors, pervasive devices, and computer vision systems. Within the context of education, this subfield is often denominated as multimodal learning analytics (MMLA) [3]; nevertheless, the use of these data sources is also common in broader research areas, such as affective computing (e.g., [4]) and human-computer interaction (HCI) (e.g., [5]). The promise is to augment and improve the extent and quality of the analysis that can be performed with these new data sources [6]. Moreover, many new sensor-based tools, such as sensor-based games [7] or realistic laboratories [8,9], are being built to support the educational process. The challenge is embedding sensors and resulting data representations in authentic educational settings in pedagogically meaningful and ethical ways [10]. This Special Issue (SI) invited publications that include approaches to converting data captured using sensors (e.g., cameras, smartphones, microphones, or temperature sensors), wearables (e.g., smart wristbands, watches, or glasses), or other Internet of Things (IoT) devices (e.g., interactive whiteboards, eBooks, or tablets) into meaningful educational insights. Moreover, it invited papers on tools, architectures, or frameworks to manage the orchestration of these sensors and IoT devices to improve education. The submitted articles had to appropriately explain how the inclusiveness of sensor devices can augment the analyses performed to improve teaching, learning, or the educational context in which the sensing it occurs (e.g., in classrooms, VLEs, or other educational spaces). This SI has focused on empirical case studies that fulfill the aforementioned criteria and experimental architectures, methodologies, frameworks, or survey papers. (DIPF/Orig.)
DIPF-Departments: Informationszentrum Bildung
-
-
Author(s): Cardenas-Hernandez, Fernando Pedro; Schneider, Jan
Title: Considerations in feedback and periodization for the multimodal learning experience of running via wearable devices
In: Mat Sanusi, Khaleel Asyraaf; Limbu, Bilbeg; Schneider, Jan; Di Mitri, Daniele; Klemke, Roland (Hrsg.): Proceedings of the Second International Workshop on Multimodal Immersive Learning Systems (MILeS 2022) at the Seventeenth European Conference on Technology Enhanced Learning (EC-TEL 2022), Toulouse, France, September 12th-16th, 2022, Aachen: RWTH, 2022 (CEUR Workshop Proceedings, 3247), S. 28-33
URL: https://ceur-ws.org/Vol-3247/paper5.pdf
Publication Type: 4. Beiträge in Sammelbänden; Beiträge in Proceedings mit Peer-Review-System
Language: Englisch
Keywords: Feedback; Fertigkeit; Lernerfahrung; Psychomotorik; Technologie; Tool; Training; Virtuelle Realität
Abstract (english): For the integral learning/training of a psychomotor activity such as running, it is necessary to target not only the physical aspects but also the technical and mental aspects that make it up, an alternative to solve this issue is through the understanding and consideration of feedback and periodization, which are elements that constitute and influence transcendentally and differently each of the three aspects involved. That is why, in this paper, some of the definitions and classifications commonly used in the study of psychomotor skills are first cited. Likewise, the use of wearable devices is proposed as a multimodal and main technology to analyze both elements since they have provided positive results in the same field of research. Subsequently, important considerations are discussed which could serve as a starting point or a conceptual reference for the correct application and analysis of feedback and periodization in each aspect of running and possibly other similar psychomotor skills. (DIPF/Orig.)
DIPF-Departments: Informationszentrum Bildung
-
-
Author(s): Jornitz, Sieglinde; Mayer, Ben
Title: "Das ist jammerschade" Vom Unterricht in Abwesenheit wegen pandemiebedingter Schulschließungen
In: Pädagogische Korrespondenz, 34 (2021) 63, S. 66-84
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Deutsch
Keywords: Schule; Pandemie; Unterricht; Distanz; Veränderung; Krise; Pädagogisches Handeln; Lernplattform; Digitale Medien; Tool; Unterrichtsmaterial; Aufgabenstellung; Organisation; Lehrer; Fallbeispiel; Interview; Analyse; Bremen; Frankfurt a.M.; Deutschland
Abstract: Die Autor*innen analysieren zwei Interviews mit zwei Lehrkräften, die sie während der ersten Schulschließungen in Deutschland im Frühjahr 2021 aufgrund der Covid-19-Pandemie veranlasst wurden. Es zeigt sich, dass die Lehrer*innen entgegen der politischen Maßgabe, weiter zu unterrichten, dies nicht aufgrund diverser Gründe, die im Interview dargelegt werden, nicht geschah. (DIPF/Autor)
DIPF-Departments: Informationszentrum Bildung
-
-
Author(s): Ritter, Marcel; Schiffner, Daniel; Harders, Matthias
Title: Visual analysis of point cloud neighborhoods via multi-scale geometric measures
In: Computer Graphics Forum, 5 (2021) 3, S. 1-14
DOI: 10.1016/j.visinf.2021.05.001
URL: https://www.sciencedirect.com/science/article/pii/S2468502X21000206?via%3Dihub
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Geometrie; Algorithmus; Visualisieren; Daten; Datenverarbeitung; FRAMEWORK; System; Software; Softwareentwicklung; Tool
Abstract (english): Point-based geometry representations have become widely used in numerous contexts, ranging from particle-based simulations, over stereo image matching, to depth sensing via light detection and ranging. Our application focus is on the reconstruction of curved line structures in noisy 3D point cloud data. Respective algorithms operating on such point clouds often rely on the notion of a local neighborhood. Regarding the latter, our approach employs multi-scale neighborhoods, for which weighted covariance measures of local points are determined. Curved line structures are reconstructed via vector field tracing, using a bidirectional piecewise streamline integration. We also introduce an automatic selection of optimal starting points via multi-scale geometric measures. The pipeline development and choice of parameters was driven by an extensive, automated initial analysis process on over a million prototype test cases. The behavior of our approach is controlled by several parameters - the majority being set automatically, leaving only three to be controlled by a user. In an extensive, automated final evaluation, we cover over one hundred thousand parameter sets, including 3D test geometries with varying curvature, sharp corners, intersections, data holes, and systematically applied varying types of noise. Further, we analyzed different choices for the point of reference in the co-variance computation; using a weighted mean performed best in most cases. In addition, we compared our method to current, publicly available line reconstruction frameworks. Up to thirty times faster execution times were achieved in some cases, at comparable error measures. Finally, we also demonstrate an exemplary application on four real-world 3D light detection and ranging datasets, extracting power line cables.
DIPF-Departments: Informationszentrum Bildung
-
-
Author(s): Jivet, Ioana; Wong, Jacqueline; Scheffel, Maren; Torre, Manuel Valle; Specht, Marcus; Drachsler, Hendrik
Title: Quantum of choice. How learners' feedback monitoring decisions, goals and self-regulated learning skills are related
In: Association for Computing Machinery (Hrsg.): LAK21 conference proceedings: The impact we make: The contributions of learning analytics to learning, the Eleventh International Conference on Learning Analytics & Knowledge, April 12-16, 2021, online, everywhere, New York; NY: Association for Computing Machinery, 2021 , S. 416-427
DOI: 10.1145/3448139.3448179
URL: https://dl.acm.org/doi/10.1145/3448139.3448179
Publication Type: 4. Beiträge in Sammelbänden; Tagungsband/Konferenzbeitrag/Proceedings
Language: Englisch
Keywords: Chi-Quadrat-Test; Regressionsanalyse; Lernender; Entscheidungsfindung; Feedback; Lernziel; Selbstgesteuertes Lernen; Learning Analytics; Tool; Design; Lernplattform; MOOC <Massive Open Online Course>; E-Learning; Interaktion; Lernverhalten; Datenerfassung; Fragebogenerhebung
Abstract: Learning analytics dashboards (LADs) are designed as feedback tools for learners, but until recently, learners rarely have had a say in how LADs are designed and what information they receive through LADs. To overcome this shortcoming, we have developed a customisable LAD for Coursera MOOCs on which learners can set goals and choose indicators to monitor. Following a mixed-methods approach, we analyse 401 learners' indicator selection behaviour in order to understand the decisions they make on the LAD and whether learner goals and self-regulated learning skills influence these decisions. We found that learners overwhelmingly chose indicators about completed activities. Goals are not associated with indicator selection behaviour, while help-seeking skills predict learners' choice of monitoring their engagement in discussions and time management skills predict learners' interest in procrastination indicators. The findings have implications for our understanding of learners' use of LADs and their design. (DIPF/Orig.)
DIPF-Departments: Informationszentrum Bildung