Ergebnis der Suche in der DIPF Publikationendatenbank

Ihre Abfrage:

(Schlagwörter: "Bewertung")

Measurement invariance in relation to first language. An evaluation of German reading and spelling […] Visser, Linda; Cartschau, Friederike; Goldammer, Ariane von; Brandenburg, Janin; […] Zeitschriftenbeitrag | In: Applied Measurement in Education | 2023 43274 Endnote: Autor*innen: Visser, Linda; Cartschau, Friederike; Goldammer, Ariane von; Brandenburg, Janin; Timmerman, Marieke; Hasselhorn, Marcus; Mähler, Claudia
Titel: Measurement invariance in relation to first language. An evaluation of German reading and spelling tests
In: Applied Measurement in Education, 36 (2023) 2, S. 115-131
DOI: 10.1080/08957347.2023.2201701
URL: https://www.tandfonline.com/doi/full/10.1080/08957347.2023.2201701
Dokumenttyp: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Sprache: Englisch
Schlagwörter: Bewertung; deutsch; Deutsch als Zweitsprache; Deutschland; Faktorenanalyse; Grundschule; Lesen; Lesetest; Messung; Muttersprache; Rechtschreibtest; Rechtschreibung; Reliabilität; Sekundarstufe I; Testkonstruktion; Textverständnis; Varianzanalyse; Vergleich
Abstract (english): The growing number of children in primary schools in Germany who have German as their second language (L2) has raised questions about the fairness of performance assessment. Fair tests are a prerequisite for distinguishing between L2 learning delay and a specific learning disability. We evaluated five commonly-used reading and spelling tests for measurement invariance (MI) as a function of first language (German vs. other). Multi-group confirmatory factor analyses revealed strict MI for the Weingarten Basic Vocabulary Spelling Tests (WRTs) 3+ and 4+ and the Salzburger Reading (SLT) and Spelling (SRT) Tests, suggesting these instruments are suitable for assessing reading and spelling skills regardless of first language. The MI for A Reading Comprehension Test for First to Seventh Graders - 2nd Edition (ELFE II) was partly strict with unequal intercepts for the text subscale. We discuss the implications of this finding for assessing reading performance of children with L2. (DIPF/Orig.)
DIPF-Abteilung: Bildung und Entwicklung

To score or not to score. Factors influencing performance and feasibility of automatic content […] Zesch, Torsten; Horbach, Andrea; Zehner, Fabian Zeitschriftenbeitrag | In: Educational Measurement: Issues and Practice | 2023 43438 Endnote: Autor*innen: Zesch, Torsten; Horbach, Andrea; Zehner, Fabian
Titel: To score or not to score. Factors influencing performance and feasibility of automatic content scoring of text responses
In: Educational Measurement: Issues and Practice, 42 (2023) 1, S. 44-58
DOI: 10.1111/emip.12544
URL: https://onlinelibrary.wiley.com/doi/10.1111/emip.12544
Dokumenttyp: 3a. Beiträge in begutachteten Zeitschriften; Beitrag in Sonderheft
Sprache: Englisch
Schlagwörter: Antwort; Automatisierung; Bewertung; Einflussfaktor; Inhalt; Leistung; Text; Tool; Verfahren
Abstract (english): In this article, we systematize the factors influencing performance and feasibility of automatic content scoring methods for short text responses. We argue that performance (i.e., how well an automatic system agrees with human judgments) mainly depends on the linguistic variance seen in the responses and that this variance is indirectly influenced by other factors such as target population or input modality. Extending previous work, we distinguish conceptual, realization, and nonconformity variance, which are differentially impacted by the various factors. While conceptual variance relates to different concepts embedded in the text responses, realization variance refers to their diverse manifestation through natural language. Nonconformity variance is added by aberrant response behavior. Furthermore, besides its performance, the feasibility of using an automatic scoring system depends on external factors, such as ethical or computational constraints, which influence whether a system with a given performance is accepted by stakeholders. Our work provides (i) a framework for assessment practitioners to decide a priori whether automatic content scoring can be successfully applied in a given setup as well as (ii) new empirical findings and the integration of empirical findings from the literature on factors that influence automatic systems' performance. (DIPF/Orig.)
DIPF-Abteilung: Lehr und Lernqualität in Bildungseinrichtungen

Teachers' ratings of social exclusion among students. The role of situational information and the […] Szekely, Lino; Bonefeld, M.; Beißert, Hanna Zeitschriftenbeitrag | In: Open Psychology | 2022 43517 Endnote: Autor*innen: Szekely, Lino; Bonefeld, M.; Beißert, Hanna
Titel: Teachers' ratings of social exclusion among students. The role of situational information and the ethnic origin of the excluded student based on the example of Syrian refugees
In: Open Psychology, 4 (2022) 1, S. 132-151
DOI: 10.1515/psych-2022-0007
URL: https://www.degruyter.com/document/doi/10.1515/psych-2022-0007/html
Dokumenttyp: 3a. Beiträge in begutachteten Zeitschriften; Beitrag in Sonderheft
Sprache: Englisch
Schlagwörter: Bewertung; Deutschland; Ethnische Herkunft; Flüchtling; Information; Intervention; Kind; Lehrer; Online; Schule; Schüler; Situation; Soziale Ausgrenzung; Umfrage
Abstract (english): In the current study, we examined teachers' ratings of social exclusion among students. 120 teachers (Mage=24.00, SD=3.71, 88% female) evaluated a hypothetical exclusion in which the excluded student's origin (German vs. Syrian refugee) was varied as well as whether participants received additional situational information about prior norm-violating behavior of the excluded student or not. Additionally, participants rated how likely they would intervene in the situation. For the evaluation of exclusion and the likelihood of intervention, there was a main effect of additional situational information, revealing that participants evaluated exclusion as less acceptable and were more likely to intervene if there was no additional situational information. Regarding the evaluation of exclusion, there was an interaction of additional situational information and the origin of the excluded student as the effect of additional situational information was bigger if the excluded student was German. For the likelihood of intervention, this interaction was not significant; but descriptively a similar pattern emerged. Results indicate that information about prior norm-violating behavior is more relevant for teachers' reactions to social exclusion than the origin of the excluded student. However, in situations with an understandable reason for exclusion, teachers do include the origin of the excluded student in their consideration. (DIPF/Orig.)
DIPF-Abteilung: Bildung und Entwicklung

Methods and perspectives for the automated analytic assessment of free-text responses in formative […] Gombert, Sebastian Sammelbandbeitrag | Aus: Jivet, Joana; Di Mitri, Daniele; Schneider, Jan; Papamitsiou, Zacharoula; Fominykh, Mikhail (Hrsg.): Proceedings of the Doctoral Consortium of the 17th European Conference on Technology Enhanced Learning co-located with the 17th European Conference on Technology Enhanced Learning (EC-TEL 2022), Toulouse, France, September 12, 2022 | Aachen: RWTH | 2022 43505 Endnote: Autor*innen: Gombert, Sebastian
Titel: Methods and perspectives for the automated analytic assessment of free-text responses in formative scenarios
Aus: Jivet, Joana; Di Mitri, Daniele; Schneider, Jan; Papamitsiou, Zacharoula; Fominykh, Mikhail (Hrsg.): Proceedings of the Doctoral Consortium of the 17th European Conference on Technology Enhanced Learning co-located with the 17th European Conference on Technology Enhanced Learning (EC-TEL 2022), Toulouse, France, September 12, 2022, Aachen: RWTH, 2022 (CEUR Workshop Proceedings, 3292), S. 61-65
URL: https://ceur-ws.org/Vol-3292/DCECTEL2022_paper08.pdf
Dokumenttyp: 4. Beiträge in Sammelbänden; Tagungsband/Konferenzbeitrag/Proceedings
Sprache: Englisch
Schlagwörter: Antwort; Aufsatz; Automatisierung; Benotung; Bewertung; Spracherkennung; Test; Text
Abstract: Assessment is the process of testing learners' skills and knowledge. Free-text response items are well suited for the assessment of learners' active knowledge and writing skills. However, the automatic assessment of respective responses is not trivial and requires the application of natural language processing. Accordingly, the automatic assessment of free-text responses is a widely researched topic in educational natural language processing. Most past work targets holistic scoring, the process of assigning overall scores or grades to responses. This is problematic in formative scenarios because learners require feedback rather than summative scores in such scenarios. Such feedback ideally targets specific aspects of responses, and, accordingly, automated systems which only predict holistic scores cannot be used as a basis for providing the same. What is instead needed are systems which implement analytic scoring approaches. Analytic scoring targets specific aspects of responses and scores them according to corresponding criteria. This requires different systems than addressed by the broad research on automated holistic scoring. In my PhD work which is outlined by this paper, I want to explore approaches for implementing analytic scoring systems by means of state-of-the-art natural language processing. These systems are targeted at providing a basis for feedback generation. (DIPF/Orig.)
DIPF-Abteilung: Informationszentrum Bildung

Abiturprüfungspraxis und Abituraufsatz 1882 bis 1972 Kämper-van den Boogaart, Michael; Reh, Sabine Sammelbandbeitrag | Aus: Hoffmann, Lars; Schröter, Pauline; Groß, Alexander; Schmid-Kühn, Svenja Mareike; Stanat, Petra (Hrsg.): Das unvergleichliche Abitur: Entwicklungen - Herausforderungen - Empirische Analysen | Bielefeld: wbv Media | 2022 42926 Endnote: Autor*innen: Kämper-van den Boogaart, Michael; Reh, Sabine
Titel: Abiturprüfungspraxis und Abituraufsatz 1882 bis 1972
Aus: Hoffmann, Lars; Schröter, Pauline; Groß, Alexander; Schmid-Kühn, Svenja Mareike; Stanat, Petra (Hrsg.): Das unvergleichliche Abitur: Entwicklungen - Herausforderungen - Empirische Analysen, Bielefeld: wbv Media, 2022 , S. 181-211
URL: https://www.wbv.de/download/shop/download/0/_/0/0/listview/file/-direct%40I72494/area/openaccess.html
Dokumenttyp: 4. Beiträge in Sammelbänden; Sammelband (keine besondere Kategorie)
Sprache: Deutsch
Schlagwörter: 19. Jahrhundert; 20. Jahrhundert; Abiturprüfung; Abschlussprüfung; Aufsatz; Auslese; Baden; Bayern; Bewertung; Bildungsgeschichte; Deutschland-BRD; Deutschunterricht; Fachdidaktik; Fachwissen; Forschungsprojekt; Gymnasium; Normen; Preußen; Studierfähigkeit; Textproduktion; Universität; Württemberg
Abstract: Im Rahmen eines von der Leibniz-Gemeinschaft geförderten Forschungsprojekts zur Geschichte des deutschen Abituraufsatzes zwischen 1872 und 1972, dem ein umfangreiches Aufsatzkorpus zugrunde lag, wurden verschiedene Determinanten erfasst, die auf Praktiken der Aufgaben- und Texterstellung wie der Korrektur und Bewertung einwirkten. Dabei handelt es sich namentlich um rechtliche Vorgaben, um politische Erwartungen und didaktische Programme, die ihrerseits Verständnisse von Reife oder von Studierfähigkeit implizieren. Im vorliegenden Beitrag wird an Schlaglichtern aus der Geschichte dieser Prüfungsform deutlich, dass sich nicht von einem linearen Prozess sprechen lässt, sondern vielfach von Effekten, die aus einem Wechselspiel interdependenter Normierungsversuche und Diskurslagen resultieren. Im Ergebnis zeigen sich bei veränderten Vorzeichen frappierend anmutende Kontinuitäten, etwa in den Klagen über eine Überforderung der Kandidaten und Kandidatinnen oder der Frage, inwieweit fachliches und lernbares Wissen den Horizont des schriftlich Darzubietenden abstecken sollte. Zu vermuten ist mithin, dass nicht wenige der historischen Diskurskonstellationen die Beobachter:innen der gegenwärtigen Auseinandersetzungen über diesen Prüfungsteil vertraut anmuten werden. (DIPF/Orig.)
DIPF-Abteilung: Bibliothek für Bildungsgeschichtliche Forschung

Studienausstieg - und dann? Der Übergang in Ausbildung und Beruf aus Sicht von Arbeitgeberinnen und […] Neugebauer, Martin; Bröder, Charlotte; Daniel, Annabell Monographie | Bielefeld: wbv Media | 2021 42050 Endnote: Autor*innen: Neugebauer, Martin; Bröder, Charlotte; Daniel, Annabell
Titel: Studienausstieg - und dann? Der Übergang in Ausbildung und Beruf aus Sicht von Arbeitgeberinnen und Arbeitgebern
Erscheinungsvermerk: Bielefeld: wbv Media, 2021
DOI: 10.3278/I70179
URL: https://www.wbv.de/openaccess/themenbereiche/hochschule-und-wissenschaft/shop/detail/name/_/0/1/I70179/facet/I70179///////nb/0/category/1754.html
Dokumenttyp: 1. Monographien (Autorenschaft); Monographie
Sprache: Deutsch
Schlagwörter: Studienabbruch; Bewerber; Übergang Studium - Beruf; Arbeitsmarkt; Integration; Arbeitgeber; Stichprobe; Einstellung <Arbeitsrecht>; Bewertung; Befragung; Deutschland
Abstract: Angesichts steigender Studierendenzahlen und konstant hoher Studienabbruchquoten wächst die Anzahl an Studienaussteiger:innen seit Jahren an. Eine gelungene Arbeitsmarktintegration dieser wachsenden Personengruppe ist aus wirtschafts- und bildungspolitischer Sicht hoch relevant, nicht zuletzt aufgrund des regelmäßig konstatierten Fachkräftemangels. Bislang ist allerdings nur wenig darüber bekannt, bei welchen Unternehmen sich Studienaussteiger:innen bewerben und wie Personalverantwortliche in Unternehmen Studienaussteiger:innen beurteilen. Hier knüpft der vorliegende Beitrag an. Er basiert auf einer Zufallsstichprobe von über 1.000 Unternehmen, die Stellen im kaufmännischen oder im IT-Bereich ausgeschrieben hatten. Die Publikation gliedert sich in zwei Abschnitte und beantwortet folgende übergreifende Fragen: I. Wo bewerben sich Studienaussteiger:innen und wie werden diese Bewerbungen von Unternehmen bewertet? II. In welchen Unternehmen werden Studienaussteiger:innen bereits beschäftigt und welche Erfahrungen haben Unternehmen mit ihnen gemacht? Die Ergebnisse zeigen sowohl Chancen als auch Hemmnisse einer gelingenden Integration von Studienaussteiger:innen in den Ausbildungs- und Arbeitsmarkt auf. (DIPF/Orig.)
DIPF-Abteilung: Struktur und Steuerung des Bildungswesens

Change in mental health symptoms during the COVID-19 pandemic. The role of appraisals and daily […] Brose, Annette; Blanke, Elisabeth S.; Schmiedek, Florian; Kramer, Andrea C.; Schmidt, Andrea; […] Zeitschriftenbeitrag | In: Journal of Personality | 2021 40575 Endnote: Autor*innen: Brose, Annette; Blanke, Elisabeth S.; Schmiedek, Florian; Kramer, Andrea C.; Schmidt, Andrea; Neubauer, Andreas B.
Titel: Change in mental health symptoms during the COVID-19 pandemic. The role of appraisals and daily life experiences
In: Journal of Personality, 89 (2021) 3, S. 468-482
DOI: 10.1111/jopy.12592
URL: https://onlinelibrary.wiley.com/doi/10.1111/jopy.12592
Dokumenttyp: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Sprache: Englisch
Schlagwörter: Affekt; Emotionaler Zustand; Bewertung; Pandemie; Stress; Wohlbefinden; Achtsamkeit; Psychohygiene; Alltag; Erfahrung; Empirische Untersuchung; Tagebuch; Pretest; Deutschland
Abstract: Intro: When confronted with major threats, people often experience decline in well‐being. The central purpose of this study was to identify mechanisms underlying change of well‐being in times of threat, using the example of the COVID‐19 pandemic, with a focus on appraisals of the pandemic and affective states, stress, as well as mindfulness in daily life. Methods: We conducted a study across 3.5 weeks, including pretest, posttest, and a diary phase in‐between. We worked with a sample of 460 adults, pre‐ and post‐test information, as well as 7189 observations from the diary phase. Results: Results showed that deterioration in mental health symptoms across the duration of the study was associated with (a) change towards less fortunate appraisals of the pandemic and (b), more negative affect and less mindfulness in daily life. Furthermore, appraisals of the pandemic at pretest predicted experiences in daily life, with more negative appraisals of the pandemic predicting more negative affect and stressor occurrence as well as less mindfulness. Discussion: These findings speak to the dynamic nature of well‐being and appraisals in times of threat, and highlight the role of experiences in daily life in changes in well‐being. (DIPF/Orig.)
DIPF-Abteilung: Bildung und Entwicklung

Facilitation of information literacy through a multilingual MOOC considering cultural aspects Dreisiebner, Stefan; Polzer, Anna Katharina; Robinson, Lyn; Libbrecht, Paul; […] Zeitschriftenbeitrag | In: Journal of Documentation | 2021 40969 Endnote: Autor*innen: Dreisiebner, Stefan; Polzer, Anna Katharina; Robinson, Lyn; Libbrecht, Paul; Boté-Vericad, Juan-José; Urbano, Cristóbal; Mandl, Thomas; Vilar, Polona; Žumer, Maja; Pehar, Franjo; Stričević, Ivanka
Titel: Facilitation of information literacy through a multilingual MOOC considering cultural aspects
In: Journal of Documentation, 77 (2021) 3, S. 777-797
DOI: 10.1108/JD-06-2020-0099
URL: https://www.emerald.com/insight/content/doi/10.1108/JD-06-2020-0099/full/html
Dokumenttyp: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Sprache: Englisch
Schlagwörter: Deutschland; Großbritannien; Kroatien; Spanien; Katalonien; Slovenien; Österreich; Informationskompetenz; MOOC <Massive Open Online Course>; Mehrsprachigkeit; Übersetzung; Interkulturelle Differenz; Student; Wissenserwerb; Einstellung <Psy>; Hochschullehrer; Bewertung; Evaluation; Deutsch; Englisch; Katalanisch; Kroatisch; Slovenisch; Spanisch
Abstract: Purpose: The purpose of this paper is to demonstrate the rationale, technical framework, content creation workflow and evaluation for a multilingual massive open online course (MOOC) to facilitate information literacy (IL) considering cultural aspects. Design/methodology/approach: A good practice analysis built the basis for the technical and content framework. The evaluation approach consisted of three phases: first, the students were asked to fill out a short self-assessment questionnaire and a shortened adapted version of a standardized IL test. Second, they completed the full version of the IL MOOC. Third, they were asked to fill out the full version of a standardized IL test and a user experience questionnaire. Findings: The results show that first the designed workflow was suitable in practice and led to the implementation of a full-grown MOOC. Second, the implementation itself provides implications for future projects developing multilingual educational resources. Third, the evaluation results show that participants achieved significantly higher results in a standardized IL test after attending the MOOC as mandatory coursework. Variations between the different student groups in the participating countries were observed. Fourth, self-motivation to complete the MOOC showed to be a challenge for students asked to attend the MOOC as nonmandatory out-of-classroom task. It seems that multilingual facilitation alone is not sufficient to increase active MOOC participation. Originality/value: This paper presents an innovative approach of developing multilingual IL teaching resources and is one of the first works to evaluate the impact of an IL MOOC on learners' experience and learning outcomes in an international evaluation study. (DIPF/Orig.)
DIPF-Abteilung: Informationszentrum Bildung

Development and evaluation of a framework for the performance-based testing of ICT skills Engelhardt, Lena; Naumann, Johannes; Goldhammer, Frank; Frey, Andreas; Horz, Holger; Hartig, Katja; […] Zeitschriftenbeitrag | In: Frontiers in Education | 2021 41203 Endnote: Autor*innen: Engelhardt, Lena; Naumann, Johannes; Goldhammer, Frank; Frey, Andreas; Horz, Holger; Hartig, Katja; Wenzel, S. Franziska C.
Titel: Development and evaluation of a framework for the performance-based testing of ICT skills
In: Frontiers in Education, 6 (2021) , S. 668860
DOI: 10.3389/feduc.2021.668860
URL: https://www.frontiersin.org/articles/10.3389/feduc.2021.668860/full
Dokumenttyp: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Sprache: Englisch
Schlagwörter: Informations- und Kommunikationstechnologie; Praktische Fertigkeit; Wissen; Problemlösen; Textverständnis; Bildverstehen; Bewertung; Modell; Item; Entwicklung; Testvalidität; Itemanalyse; Rasch-Modell; Implementation; Evaluation; Test; Testverhalten; Schüler; Sekundarstufe I; Baden-Württemberg; Rheinland-Pfalz; Deutschland
Abstract (english): This paper addresses the development of performance-based assessment items for ICT skills, skills in dealing with information and communication technologies, a construct which is rather broadly and only operationally defined. Item development followed a construct-driven approach to ensure that test scores could be interpreted as intended. Specifically, ICT-specific knowledge as well as problem-solving and the comprehension of text and graphics were defined as components of ICT skills and cognitive ICT tasks (i.e., accessing, managing, integrating, evaluating, creating). In order to capture the construct in a valid way, design principles for constructing the simulation environment and response format were formulated. To empirically evaluate the very heterogeneous items and detect malfunctioning items, item difficulties were analyzed and behavior-related indicators with item-specific thresholds were developed and applied. The 69 item's difficulty scores from the Rasch model fell within a comparable range for each cognitive task. Process indicators addressing time use and test-taker interactions were used to analyze whether most test-takers executed the intended processes, exhibited disengagement, or got lost among the items. Most items were capable of eliciting the intended behavior; for the few exceptions, conclusions for item revisions were drawn. The results affirm the utility of the proposed framework for developing and implementing performance-based items to assess ICT skills. (DIPF/Orig.)
DIPF-Abteilung: Lehr und Lernqualität in Bildungseinrichtungen

Integrating information from multiple texts relates to pre-service teachers' epistemic products for […] Hartmann, Ulrike; Kindlinger, Marcus; Trempler, Kati Zeitschriftenbeitrag | In: Teaching and Teacher Education | 2021 40622 Endnote: Autor*innen: Hartmann, Ulrike; Kindlinger, Marcus; Trempler, Kati
Titel: Integrating information from multiple texts relates to pre-service teachers' epistemic products for reflective teaching practice.
In: Teaching and Teacher Education, 97 (2021) 1, S. 103205
DOI: 10.1016/j.tate.2020.103205
URL: https://www.sciencedirect.com/science/article/pii/S0742051X20313962?via%3Dihub
Dokumenttyp: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Sprache: Englisch
Schlagwörter: Lehrerausbildung; Referendariat; Text; Quelle; Integration; Reflexion <Phil>; Referendar; Kompetenz; Unterrichtsvorbereitung; Experte; Bewertung; Qualitative Forschung; Inhaltsanalyse
Abstract (english): Integrating information from multiple texts is a core aspect of pre-service teachers' preparation for reflective teaching practice. This study connects research on multiple text use with teachers' epistemic reflexivity. Using qualitative content analysis, we investigate how information is integrated to explain real pedagogical situations in essays by 87 pre-service teachers. Competent text integration was related to a higher number of reflexive statements. These analyses are supplemented with an expert rating, showing that essays with competent text integration were rated higher on the quality of their conclusions for professional teaching practice than those without substantial text integration.
DIPF-Abteilung: Bildungsqualität und Evaluation