-
-
Herausgeber*innen: Goldhammer, Frank; Scherer, Ronny; Greiff, Samuel
Titel: Advancements in technology-based assessment. Emerging item formats, test designs, and data sources
Erscheinungsvermerk: Lausanne: Frontiers Media, 2020 (Frontiers in Psychology. Sonderheft)
DOI: 10.3389/fpsyg.2019.03047
URL: https://www.frontiersin.org/research-topics/7841/advancements-in-technology-based-assessment-emerging-item-formats-test-designs-and-data-sources
Dokumenttyp: 2. Herausgeberschaft; Zeitschriftensonderheft
Sprache: Englisch
Schlagwörter: Technologiebasiertes Testen; Item; Test; Design; Auswertung; Automatisierung; Prozessdatenverarbeitung; Lernen; Bewertung
Abstract (english): Technology has become an indispensable tool for educational and psychological assessment in today's world. Researchers and large-scale assessment programs alike are increasingly using digital technology (e.g., laptops, tablets, and smartphones) to collect behavioral data beyond the mere idea of responses as correct. Along these lines, technology innovates and enhances assessments in terms of item and test design, methods of test delivery, data collection and analysis, as well as the reporting of test results.
The aim of this Research Topic is to present recent advancements in technology-based assessment. Our focus is on cognitive assessments, including the measurement of abilities, competencies, knowledge, and skills but may also include non-cognitive aspects of the assessment. In the area of (cognitive) assessments the innovations driven by technology are manifold: Digital assessments facilitate the creation of new types of stimuli and response formats that were out of reach for assessments using paper; for instance, interactive simulations including multimedia elements, as well as virtual or augmented realities which serve as the task environment. Moreover, technology allows the automated generation of items based on specific item models. Such items can be assembled into tests in a more flexible way than that offered by paper-and-pencil tests and could even be created on the fly; for instance, tailoring item difficulty to individual ability (adaptive testing), while assuring that multiple content constraints are met. As a requirement for adaptive testing or to lower the burden of raters coding item responses manually, computers enable the automatic scoring of constructed responses; for instance, text responses can be scored automatically by using natural language processing and text mining. Technology-based assessments provide not only response data (e.g., correct vs. incorrect responses) but also process data (e.g., frequencies and sequences of test-taking strategies, including navigation behavior) which reflects the course of solving a test item. Process data has been used successfully, among others, to evaluate the data quality, to define process-oriented constructs, to improve measurement precision, and to address substantial research questions.
We expect the contributions of this Research Topic to build on this research by considering how technology can further improve, and enhance, educational and psychological assessment. Regarding educational testing, both research papers on the assessment of learning (e.g., summative assessment of learning outcomes) and on the assessment for learning (e.g., formative assessment to support the learning process) are welcome. We expect submissions of empirical papers that present and evaluate innovative technology-based assessment approaches, as well as new applications or illustrations of already existing approaches. We are also interested in papers addressing the validity of test scores and other indicators obtained from innovative assessment procedures.
DIPF-Abteilung: Bildungsqualität und Evaluation
-
-
Herausgeber*innen: Mackowiak, Katja; Beckerle, Christine; Gentrup, Sarah; Titz, Cora
Titel: Forschungsinstrumente im Kontext institutioneller (schrift-)sprachlicher Bildung
Erscheinungsvermerk: Bad Heilbrunn: Klinkhardt, 2020
DOI: doi.org/1035468/5801
URN: urn:nbn:de:0111-pedocs-201399
URL: https://www.pedocs.de/frontdoor.php?source_opus=20139
Dokumenttyp: 2. Herausgeberschaft; Sammelband (keine besondere Kategorie)
Sprache: Deutsch
Schlagwörter: Aktivierung; Anwendung; Auswertung; Baden-Württemberg; Bayern; Beobachtung; Beobachtungsmethode; Berufliche Kompetenz; Bewertung; Bildungsforschung; Bildungspraxis; Datenerfassung; Deutschland; Deutschunterricht; Diagnostik; Dokumentation; Empirische Forschung; Erhebungsinstrument; Erzieher; Erzieherin; Fachdidaktik; Fachsprache; Fachunterricht; Fachwissen; Feed-back; Förderdiagnostik; Förderung; Forschung; Forschungsmethode; Fragebogen; Grundschule; Handlungskompetenz; Instrument; Interview; Kind; Kindergarten; Kindergartenalltag; Kindergartenpädagogik; Kindertagesstätte; Kompetenz; Kompetenzmessung; Korrektur; Längsschnittuntersuchung; Lehramtsstudent; Lehrer; Lehrer-Schüler-Interaktion; Leitfadeninterview; Leseförderung; Lesekompetenz; Mathematikunterricht; Messinstrument; Methode; Nordrhein-Westfalen; Pädagogische Fachkraft; Pädagogisches Handeln; Primarbereich; Profession; Professionalisierung; Protokoll; Psychometrie; Qualitative Forschung; Quantitative Forschung; Reflexion <Phil>; Sachsen-Anhalt; Schriftsprache; Schule; Schüler; Schülermitarbeit; Sekundarstufe I; Software; Sprachbildung; Sprachdiagnostik; Spracherwerb; Sprachförderung; Sprachgebrauch; Sprachkompetenz; Sprachpraktische Übung; Sprachstandsforschung; Test; Testkonstruktion; Testvalidität; Testverfahren; Transfer; Unterricht; Videoanalyse; Videoaufzeichnung; Vignette <Methode>; Wissen; Wissenstest
Abstract: Das Buch stellt sieben Verfahren vor, die unterschiedliche Facetten (schrift-)sprachlicher Bildung erfassen. Die Verfahren fokussieren unterschiedlcihe Aspekte der Sprachbildung sowie Sprach- und Leseförderung in Kindertageseinrichtungen und Schulen und adressieren unterschiedliche Zielgruppen (pädagogische Fachkräfte, Lehrkräfte, Kinder). Die Instrumente wurden im Rahmen der Initiative "Bildung durch Sprache und Schrift" (BiSS) entwickelt und primär für Forschungszwecke, jedoch auch in der Aus- und Weiterbildung von pädagogischen Fach- und Lehrkräften einsetzbar. Ein online-Anhang stellt die Instrumente sowie alle notwendigen Materialien zur Verfügung.
DIPF-Abteilung: Bildung und Entwicklung
-
-
Herausgeber*innen: Wilmers, Annika; Anda, Carolin; Keller, Carolin; Rittberger, Marc
Titel: Bildung im digitalen Wandel. Die Bedeutung für das pädagogische Personal und für die Aus- und Fortbildung
Erscheinungsvermerk: Münster: Waxmann, 2020 (Digitalisierung im Bildungsbereich, 1)
DOI: 10.31244/9783830991991
URN: urn:nbn:de:0111-pedocs-207627
URL: https://www.pedocs.de/frontdoor.php?source_opus=20762
Dokumenttyp: 2. Herausgeberschaft; Sammelband (keine besondere Kategorie)
Sprache: Deutsch
Schlagwörter: Ausbildung; Berufsbildung; Deutschland; Digitale Medien; Digitalisierung; Einstellung <Psy>; Elementarbereich; Erwachsenenbildung; Forschungsstand; Fortbildung; Frühpädagogik; Kompetenz; Kompetenzentwicklung; Lehrerausbildung; Lehrerbildner; Lehrerfortbildung; Literaturauswertung; Literaturrecherche; Literaturübersicht; Medieneinsatz; Methode; Qualität; Schule; Studie; Technologische Entwicklung; Weiterbildung; Wissenschaftliche Literatur
DIPF-Abteilung: Informationszentrum Bildung
-
-
Autor*innen: Aditomo, Anindito; Köhler, Carmen
Titel: Do student ratings provide reliable and valid information about teaching quality at the school level? Evaluating measures of science teaching in PISA 2015
In: Educational Assessment, Evaluation and Accountability, 32 (2020) 3, S. 275-310
DOI: 10.1007/s11092-020-09328-6
URL: https://link.springer.com/article/10.1007/s11092-020-09328-6
Dokumenttyp: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Sprache: Englisch
Schlagwörter: Naturwissenschaftlicher Unterricht; Qualität; Messung; Evaluation; Schüler; Bewertung; Schülerurteil; Validität; Reliabilität; Schulklima; Einflussfaktor; Wirkung; PISA <Programme for International Student Assessment>; Modell; Faktorenanalyse; Empirische Untersuchung; OECD-Länder
Abstract: Large-scale educational surveys, including PISA, often collect student ratings to assess teaching quality. Because of the sampling design in PISA, student ratings must be aggregated at the school level instead of the classroom level. To what extent does school-level aggregation of student ratings yield reliable and valid measures of teaching quality? We investigate this question for six scales measuring classroom management, emotional support, inquiry-based instruction, teacher-directed instruction, adaptive instruction, and feedback provided by PISA 2015. The sample consisted of 503,146 students from 17,678 schools in 69 countries/regions. Multilevel CFA and SEM were conducted for each scale in each country/region to evaluate school-level reliability (intraclass correlations 1 and 2), factorial validity, and predictive validity. In most countries/regions, school-level reliability was found to be adequate for the classroom management scale, but only low to moderate for the other scales. Examination of factorial and predictive validity indicated that the classroom management, emotional support, adaptive instruction, and teacher-directed instruction scales capture meaningful differences in teaching quality between schools. Meanwhile, the inquiry scale exhibited poor validity in almost all countries/regions. These findings suggest the possibility of using student ratings in PISA to investigate some aspects of school-level teaching quality in most countries/regions. (DIPF/Orig.)
DIPF-Abteilung: Bildungsqualität und Evaluation
-
-
Autor*innen: Baier, Franziska; Kunter, Mareike
Titel: Construction and validation of a test to assess (pre-service) teachers' technological pedagogical knowledge (TPK)
In: Studies in Educational Evaluation, 67 (2020) , S. 100936
DOI: 10.1016/j.stueduc.2020.100936
URL: https://www.sciencedirect.com/science/article/abs/pii/S0191491X2030184X?via%3Dihub
Dokumenttyp: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Sprache: Englisch
Schlagwörter: Deutschland; Lehramtsanwärter; Referendariat; Informationstechnologische Bildung; Unterricht; Unterrichtsqualität; Digitale Medien; Pädagogische Kompetenz; Wissen; Selbstbeurteilung; Bildungsforschung; Forscher; Bewertung; Testvalidität; Kompetenzmessung; Kompetenzentwicklung;
Abstract: As society becomes increasingly digital, teachers must be trained to integrate technology effectively into their classrooms. Teachers' technological pedagogical knowledge (TPK), as defined in the TPACK framework, is considered an important prerequisite for effectively integrating technology. The TPACK framework has received a great deal of attention, yet few knowledge tests have been developed that directly assess TPK. However, those tests are crucial for evaluating the effectiveness of teacher education courses on technology integration. We have developed a 17-item test that covers teacher knowledge about various digital technologies as employed in teaching. Experts rated the items to represent the construct adequately. Data obtained from 245 pre-service teachers supports the test's internal structure. Concerning convergent and discriminant validity, the pre-service teachers' test scores were not related to their self-reported TPK, but to their self-reported technological knowledge. The test was sensitive to changes in pre-service teachers' TPK through teacher education courses. (DIPF/Orig.)
DIPF-Abteilung: Bildungsqualität und Evaluation
-
-
Autor*innen: Beißert, Hanna; Bonefeld, Meike
Titel: German pre-service teachers' evaluations of and reactions to interethnic social exclusion scenarios
In: Frontiers in Education, (2020) , S. 5:586962
DOI: 10.3389/feduc.2020.586962
URN: urn:nbn:de:0111-pedocs-231031
URL: https://nbn-resolving.org/urn:nbn:de:0111-pedocs-231031
Dokumenttyp: 3a. Beiträge in begutachteten Zeitschriften; Beitrag in Sonderheft
Sprache: Englisch
Schlagwörter: Schüler; Soziale Ausgrenzung; Ethnischer Aspekt; Lehramtsstudent; Bewertung; Reaktion; Intervention; Befragung; Szenarium; Deutschland
Abstract (english): This study investigated pre-service teachers' evaluations, reactions, and interventions with regard to interethnic exclusion scenarios in Germany. More specifically, we focused on pre-service teachers (N = 145, 99 female, Mage= 21.34) in the role of observers of exclusion among students. Using hypothetical scenarios in which either a German or a Turkish boy was excluded by other children of his class, we assessed teachers' evaluations of this exclusion behavior. This included evaluating how likely teachers were to intervene in the situation and what they would specifically do. The aim of this research was to examine whether the origin of an excluded student represents a relevant category for teachers' evaluations of and reactions to social exclusion. In addition, we aimed to determine whether teachers include aspects related to group functioning in their considerations. The analyses demonstrated that teachers generally reject social exclusion, with female participants rejecting exclusion even more than male participants. Further, participants evaluated the exclusion of a Turkish protagonist as more reprehensible than the exclusion of a German protagonist. Regarding the likelihood of intervention, the origin of the excluded person was only relevant for male participants; i.e., they were less likely to intervene when the excluded person was German than when the excluded person was Turkish. Analyses of teachers' reasoning revealed their strong focus on inclusion as a social norm, especially in cases of interethnic exclusion. That is, when participants reasoned about the exclusion of the Turkish protagonist, they referred to the social norm of inclusion much more than when talking about the German protagonist. In contrast, aspects related to group functioning were scarcely of importance. In terms of the specific actions that participants would undertake as a reaction to the exclusion situation, no differences related to the origin of the excluded person were found. Hence, the origin of the excluded person factored into both the evaluation of the exclusion and the likelihood of intervention, but once the decision to intervene was made, there were no differences in the specific actions. The results are discussed in light of practical implications and teacher training as well as in terms of implications for future research.
DIPF-Abteilung: Bildung und Entwicklung
-
-
Autor*innen: Dignath, Charlotte; Sprenger, Lara
Titel: Can you only diagnose what you know? The relation between teachers' self-regulation of learning concepts and their assessment of students' self-regulation
In: Frontiers in Education, (2020) , S. 5:585683
DOI: 10.3389/feduc.2020.585683
URN: urn:nbn:de:0111-pedocs-252165
URL: https://nbn-resolving.org/urn:nbn:de:0111-pedocs-252165
Dokumenttyp: 3a. Beiträge in begutachteten Zeitschriften; Beitrag in Sonderheft
Sprache: Englisch
Schlagwörter: Schüler; Selbst gesteuertes Lernen; Lehrer; Förderung; Lernmethode; Bewertung; Kompetenz; Wissen; Diagnostik; Konzeptualisierung; Befragung; Leitfadeninterview; Codierung; Qualitative Forschung; Grundschule; Deutschland
Abstract (english): Self-regulation of learning (SRL) positively affects achievement and motivation. Therefore, teachers are supposed to foster students' SRL by providing them with strategies. However, two preconditions have to be met: teachers need to diagnose their students' SRL to take instructional decisions about promoting SRL. To this end, teachers need knowledge about SRL to know what to diagnose. Only little research has investigated teachers' knowledge about SRL and its assessment yet. Thus, the aim of this study was to identify teachers' conceptions about SRL, to investigate their ideas about how to diagnose their students' SRL, and to test relationships between both. To this end, we developed two systematic coding schemes to analyze the conceptions about SRL and the ideas about assessing SRL in the classroom among a sample of 205 teachers. The coding schemes for teachers' open answers were developed based on models about SRL and were extended by deriving codes from the empirical data and produced satisfactory interrater reliability (conceptions about SRL: κ = 0.85, SE = 0.03; ideas about assessing SRL: κ = 0.63, SE = 0.05). The results showed that many teachers did not refer to any regulation procedure at all and described SRL mainly as student autonomy and self-directedness. Only few teachers had a comprehensive conception of the entire SRL cycle. We identified three patterns of teachers' conceptualizations of SRL: a motivation-oriented, an autonomy-oriented, and a regulation-oriented conceptualization of SRL. Regarding teachers' ideas about assessing their students' SRL, teachers mainly focused on cues that are not diagnostic of SRL. Yet, many teachers knew about portfolios to register SRL among students. Finally, our results suggest that, partly, teachers' ideas about assessing SRL varied as a function of their SRL concept: teachers with an autonomy-oriented conceptualization of SRL were more likely to use cues that are not diagnostic of SRL, such as unsystematic observation or off-task behavior. The results provide insights into teachers' conceptions of SRL and of its assessment. Implications for future research in the field of SRL will be drawn, in particular about how to support teachers in diagnosing and fostering SR among their students.
DIPF-Abteilung: Bildungsqualität und Evaluation
-
-
Autor*innen: Fauth, Benjamin; Göllner, Richard; Lenske, Gerlinde; Praetorius, Anna-Katharina; Wagner, Wolfgang
Titel: Who sees what? Conceptual considerations on the measurement of teaching quality from different perspectives
In: Zeitschrift für Pädagogik. Beiheft, 66 (2020) , S. 138-155
Dokumenttyp: 3a. Beiträge in begutachteten Zeitschriften; Beitrag in Sonderheft
Sprache: Englisch
Schlagwörter: Empirische Forschung; Unterricht; Qualität; Messung; Schüler; Lehrer; Wahrnehmung; Bewertung; Erhebungsinstrument; Validität; Schülerrolle; Lehrerrolle; Taxonomie; Klassenführung; Vergleich; Fragebogen; Beobachtung; Selbsteinschätzung; Fremdeinschätzung
Abstract: One puzzling finding in education research is that teachers, students, and external observers agree only marginally on their ratings of teaching quality. In this theoretical contribution, we summarize and reappraise previous findings on agreement between different raters of teaching quality. We explain these findings by thoroughly examining the instruments that have been used to measure teaching quality. Building on this, we propose a reference perspective matrix, which should be useful in explaining perspective-specific rating mechanisms behind responses to certain survey or observation items. The reference perspective matrix could thus afford a theoretical foundation for future studies on the assessment of teaching quality. (DIPF/Orig.)
DIPF-Abteilung: Bildungsqualität und Evaluation
-
-
Autor*innen: Fischer, Jessica; He, Jia; Klieme, Eckhard
Titel: The structure of teaching practices across countries. A combination of factor analysis and network analysis
In: Studies in Educational Evaluation, 65 (2020) , S. 100861
DOI: 10.1016/j.stueduc.2020.100861
URN: urn:nbn:de:0111-pedocs-203522
URL: https://nbn-resolving.org/urn:nbn:de:0111-pedocs-203522
Dokumenttyp: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Sprache: Englisch
Schlagwörter: Unterrichtspraxis; Schülerleistung; Leistungsbewertung; Faktorenanalyse; Netzwerkanalyse; Datenanalyse; Sekundäranalyse; PISA <Programme for International Student Assessment>; Internationaler Vergleich; Interkultureller Vergleich
Abstract (english): Teaching practices are pivotal for student learning. Due to pedagogical traditions and national cultures, the structure of teaching practices may differ across countries. This study investigates the structure of teaching practices across 12 countries grouped into four major linguistic/cultural clusters. First, factor analysis is applied to investigate if the theoretical distinction between teacher-directed and student-centred practices is generalizable across countries. Then, network analysis is used to explore how individual classroom assessment practices relate to either teacher-directed or student-centred practices. Main findings include that: (1) teacher-directed and student-centred practices are two distinct factors across countries; (2) the overall structure and connectivity of teaching practices differs across countries, with smaller differences within linguistic/cultural clusters; and (3) assessment practices with the aim to structure and guide learning strongly relate to teacher-directed practices, whereas assessment practices with the aim to individualize instruction more relate to student-centred practices. We discuss the global patterning and implications.
DIPF-Abteilung: Bildungsqualität und Evaluation
-
-
Autor*innen: Göllner, Richard; Fauth, Benjamin; Lenske, Gerlinde; Praetorius, Anna-Katharina; Wagner, Wolfgang
Titel: Do student ratings of classroom management tell us more about teachers or about classroom composition?
In: Zeitschrift für Pädagogik. Beiheft, 66 (2020) , S. 156-172
Dokumenttyp: 3a. Beiträge in begutachteten Zeitschriften; Beitrag in Sonderheft
Sprache: Englisch
Schlagwörter: Einflussfaktor; Faktorenanalyse; Längsschnittuntersuchung; Deutschland; Klassenführung; Wahrnehmung; Schüler; Bewertung; Geschlechtsspezifischer Unterschied; Lehrer; Schuljahr 09; Schuljahr 10; Zusammensetzung; Mathematikunterricht; Leistungsdifferenzierung; Mathematische Kompetenz; Qualität; Unterrichtsklima; Unterrichtsstörung; Indikator
Abstract: The present study investigated whether the varying referents (i.e., teacher or student referent) of student ratings of established classroom management measures differ in their associations with compositional classroom characteristics and students' math achievement. Re-analysis of a large-scale dataset (PISA 2003) showed that classrooms with a higher proportion of male students, as well as those with lower math performance, exhibited lower scores on classroom management factors referring more to students than the teacher. These were in turn related to lower pre-adjusted math achievement of students. There were no associations with a measure referring to the teacher. Our results indicate that varying referents tap into different aspects of the classroom management process. (DIPF/Orig.)
DIPF-Abteilung: Bildungsqualität und Evaluation