-
-
Editor(s) Reiss, Kristina; Weis, Mirjam; Klieme, Eckhard; Köller, Olaf
Title: PISA 2018: Grundbildung im internationalen Vergleich
Published: Münster: Waxmann, 2019
DOI: 10.31244/9783830991007
URN: urn:nbn:de:0111-pedocs-183155
URL: http://nbn-resolving.org/urn:nbn:de:0111-pedocs-183155
Publication Type: 2. Herausgeberschaft; Sammelband (keine besondere Kategorie)
Language: Deutsch
Keywords: Leseförderung; Textverständnis; Schüler; Migrationshintergrund; Muttersprache; Außerunterrichtliche Aktivität; Soziale Herkunft; Einflussfaktor; Mathematische Kompetenz; Geschlechtsspezifischer Unterschied; Naturwissenschaftliche Kompetenz; Schulform; Methodologie; PISA <Programme for International Student Assessment>; Schülerleistung; Leistungsmessung; Lesekompetenz; Schülerleistungstest; Digitale Medien; Mediennutzung; Veränderung; Leseverhalten; Testaufgabe; Antwort; Lesen; Strategie; Selbstkonzept; Motivation; Sprachförderung; Technologiebasiertes Testen; Testkonstruktion; Konzeption; Internationaler Vergleich; OECD-LÄnder; Deutschland
Abstract: Der nationale Berichtsband stellt die Ergebnisse der Schülerinnen und Schüler in Deutschland aus der PISA-Studie 2018 vor und setzt sie in Relation zu den Ergebnissen in anderen Staaten. Der Schwerpunkt der Erhebungen und Auswertungen liegt dabei auf der Lesekompetenz. Die Rahmenkonzeption der Lesekompetenz wurde für die PISA-Studie 2018 im Vergleich zu den vorigen Runden einer Revision unterzogen und berücksichtigt nun die sich verändernde Lesepraxis, die mit der Nutzung digitaler Medien einhergeht. Es werden leistungsbezogene Variablen genauso wie motivationale Orientierungen, Einstellungen und Verhalten erfasst, die für die Lesekompetenz wichtig sein können. Daneben wird der Beitrag der sozialen Herkunft und des Zuwanderungshintergrunds für die Lesekompetenzen untersucht. Darüber hinaus werden die Kompetenzen der Jugendlichen in der Mathematik und den Naturwissenschaften vorgestellt und diskutiert. (DIPF/Verlag).
DIPF-Departments: Bildungsqualität und Evaluation
-
-
Editor(s) Titz, Cora; Geyer, Sabrina; Ropeter, Anna; Wagner, Hanna; Weber, Susanne; Hasselhorn, Marcus
Title: Konzepte zur Sprach- und Schriftsprachförderung: Praxiserfahrungen
Published: Stuttgart: Kohlhammer, 2019 (Bildung durch Sprache und Schrift, 3)
Publication Type: 2. Herausgeberschaft; Sammelband (keine besondere Kategorie)
Language: Deutsch
Keywords: Sprachförderung; Schriftsprache; Förderung; Konzeption; Kindertagesstätte; Fachkraft; Qualifizierung; Professionalisierung; Weiterbildung; Curriculum; Kompetenz; Diagnostik; Wissenstransfer; Akteur; Lerngemeinschaft; Grundschule; Mathematikunterricht; Wortschatz; Leseförderung; Lesekompetenz; Programm; Umsetzung; Projekt; Kooperation; Praxisbezug; Rheinland-Pfalz; Nordrhein-Westfalen; Baden-Württemberg; Sachsen; Bayern; Hamburg; Hessen; Deutschland
Abstract: Dieser Band demonstriert anhand einiger Beispiele die konkrete Arbeit mit Konzepten der Sprach- und Schriftsprachförderung im Elementar- und Schulbereich. Autorinnen und Autoren aus exemplarischen BiSS-Verbünden, in denen mehrere Kitas oder Schulen zusammenarbeiten, erläutern, welche Ziele der jeweilige Verbund mit seinem Konzept verfolgt und über welche Wege diese Ziele erreicht werden sollen. Ausführlich wird dabei auf Vorgehensweisen und Instrumente (Tools) eingegangen, die im Bereich Diagnostik, Förderung und Professionalisierung zum Einsatz kommen, und auch darauf, was die Autorinnen und Autoren als Belege für den Erfolg der eigenen Arbeit bewerten. Erläutert wird außerdem, wie die Kitas oder Schulen eines Verbundes ihre Kooperation organisiert haben. (DIPF/Orig.)
DIPF-Departments: Bildung und Entwicklung
-
-
Author(s): Engelhardt, Lena; Goldhammer, Frank
Title: Validating test score interpretations using time information
In: Frontiers in Psychology, (2019) , S. 10:1131
DOI: 10.3389/fpsyg.2019.01131
URN: urn:nbn:de:0111-dipfdocs-174371
URL: http://www.dipfdocs.de/volltexte/2020/17437/pdf/fpsyg-2019_may_Engelhardt_Goldhammer_Validating_test_score_interpretations_using_time_information_A.pdf
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Beitrag in Sonderheft
Language: Englisch
Keywords: Strukturgleichungsmodell; Empirische Untersuchung; Deutschland; Test; Validität; Computerunterstütztes Verfahren; Aufgabe; Antwort; Zeit; Fertigkeit; Kognitive Prozesse; Kognitive Kompetenz; Lesekompetenz; Denken; Leistungstest; PIAAC <Programme for the International Assessment of Adult Competencies>; Datenanalyse
Abstract (english): A validity approach is proposed that uses processing times to collect validity evidence for the construct interpretation of test scores. The rationale of the approach is based on current research of processing times and on classical validity approaches, providing validity evidence based on relationships with other variables. Within the new approach, convergent validity evidence is obtained if a component skill, that is expected to underlie the task solution process in the target construct, positively moderates the relationship between effective speed and effective ability in the corresponding target construct. Discriminant validity evidence is provided if a component skill, that is not expected to underlie the task solution process in the target construct, does indeed not moderate the speed-ability relation in this target construct. Using data from a study that follows up the German PIAAC sample, this approach was applied to reading competence, assessed with PIAAC literacy items, and to quantitative reasoning, assessed with Number Series. As expected from theory, the effect of speed on ability in the target construct was only moderated by the respective underlying component skill, that is, word meaning activation skill as an underlying component skill of reading competence, and perceptual speed as an underlying component skill of reasoning. Accordingly, no positive interactions were found for the component skill that should not underlie the task solution process, that is, word meaning activation for reasoning and perceptual speed for reading. Furthermore, the study shows the suitability of the proposed validation approach. The use of time information in association with task results brings construct validation closer to the actual response process than widely used correlations of test scores. (DIPF/Orig.)
DIPF-Departments: Bildungsqualität und Evaluation
-
-
Author(s): Fischer, Jessica; Praetorius, Anna-Katharina; Klieme, Eckhard
Title: The impact of linguistic similarity on cross-cultural comparability of students' perceptions of teaching quality
In: Educational Assessment, Evaluation and Accountability, 31 (2019) 2, S. 201-220
DOI: 10.1007/s11092-019-09295-7
URN: urn:nbn:de:0111-pedocs-190654
URL: http://nbn-resolving.org/urn:nbn:de:0111-pedocs-190654
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: PISA <Programme for International Student Assessment>; Unterricht; Qualität; Messung; Schüler; Wahrnehmung; Daten; Interkultureller Vergleich; Klassenführung; Kognitives Lernen; Sprache; Einflussfaktor; Messverfahren; Kulturdifferenz; Sprachproblem; Frage; Antwort; Testauswertung; Validität; Reliabilität; Vergleichende Bildungsforschung; Macau; Schanghai; Taipei; England; Schottland; Irland; Frankreich; Belgien; Schweiz; Österreich; Deutschland; Mexiko; Kolumbien
Abstract (english): Valid cross-country comparisons of student learning and pivotal factors contributing to it, such as teaching quality, offer the possibility to learn from outstandingly effective educational systems across the world and to improve learning in classrooms by providing policy relevant information. Yet, it often remains unclear whether the instruments used in international large-scale assessments work similarly across different cultural and linguistic groups, and thus can be used for comparing them. Using PISA 2012 data, we investigated data comparability of three teaching quality dimensions, namely student support, classroom management, and cognitive activation using a newly developed psychometric approach, namely alignment. Focusing on 15 countries, grouped into five linguistic clusters, we secondly assessed the impact of linguistic similarity on data comparability. Main findings include that (1) comparability of teaching quality measures is limited when comparing linguistically diverse countries; (2) the level of comparability varies across dimensions; (3) linguistic similarity considerably enhances the degree of comparability, except across the Chinese-speaking countries. Our study illustrates new and more flexible possibilities to test for data comparability and outlines the importance to consider cultural and linguistic differences when comparing teaching-related measures across groups. We discuss possible sources of lacking data comparability and implications for comparative educational research. (DIPF/Orig.)
DIPF-Departments: Bildungsqualität und Evaluation
-
-
Author(s): Kroehne, Ulf; Buerger, Sarah; Hahnel, Carolin; Goldhammer, Frank
Title: Construct equivalence of PISA reading comprehension measured with paper‐based and computer‐based assessments
In: Educational Measurement, 38 (2019) 3, S. 97-111
DOI: 10.1111/emip.12280
URL: https://onlinelibrary.wiley.com/doi/abs/10.1111/emip.12280
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Einflussfaktor; Schülerleistung; Frage; Antwort; Interaktion; Unterschied; Vergleich; Item-Response-Theory; Deutschland; PISA <Programme for International Student Assessment>; Leseverstehen; Messverfahren; Testkonstruktion; Korrelation; Äquivalenz; Papier-Bleistift-Test; Computerunterstütztes Verfahren; Technologiebasiertes Testen; Leistungsmessung; Testverfahren; Testdurchführung
Abstract: For many years, reading comprehension in the Programme for International Student Assessment (PISA) was measured via paper‐based assessment (PBA). In the 2015 cycle, computer‐based assessment (CBA) was introduced, raising the question of whether central equivalence criteria required for a valid interpretation of the results are fulfilled. As an extension of the PISA 2012 main study in Germany, a random subsample of two intact PISA reading clusters, either computerized or paper‐based, was assessed using a random group design with an additional within‐subject variation. The results are in line with the hypothesis of construct equivalence. That is, the latent cross‐mode correlation of PISA reading comprehension was not significantly different from the expected correlation between the two clusters. Significant mode effects on item difficulties were observed for a small number of items only. Interindividual differences found in mode effects were negatively correlated with reading comprehension, but were not predicted by basic computer skills or gender. Further differences between modes were found with respect to the number of missing values.
Abstract (english): For many years, reading comprehension in the Programme for International Student Assessment (PISA) was measured via paper‐based assessment (PBA). In the 2015 cycle, computer‐based assessment (CBA) was introduced, raising the question of whether central equivalence criteria required for a valid interpretation of the results are fulfilled. As an extension of the PISA 2012 main study in Germany, a random subsample of two intact PISA reading clusters, either computerized or paper‐based, was assessed using a random group design with an additional within‐subject variation. The results are in line with the hypothesis of construct equivalence. That is, the latent cross‐mode correlation of PISA reading comprehension was not significantly different from the expected correlation between the two clusters. Significant mode effects on item difficulties were observed for a small number of items only. Interindividual differences found in mode effects were negatively correlated with reading comprehension, but were not predicted by basic computer skills or gender. Further differences between modes were found with respect to the number of missing values.
DIPF-Departments: Bildungsqualität und Evaluation
-
-
Author(s): Kroehne, Ulf; Hahnel, Carolin; Goldhammer, Frank
Title: Invariance of the response processes between gender and modes in an assessment of reading
In: Frontiers in Applied Mathematics and Statistics, (2019) , S. 5:2
DOI: 10.3389/fams.2019.00002
URL: https://www.frontiersin.org/articles/10.3389/fams.2019.00002/full
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Beitrag in Sonderheft
Language: Englisch
Keywords: Lesefertigkeit; Technologiebasiertes Testen; Computerunterstütztes Verfahren; Papier-Bleistift-Test; Antwort; Zeit; Messung; Item-Response-Theory; Modell; Geschlechtsspezifischer Unterschied; Logdatei; Datenanalyse; Empirische Untersuchung; Deutschland
Abstract: In this paper, we developed a method to extract item-level response times from log data that are available in computer-based assessments (CBA) and paper-based assessments (PBA) with digital pens. Based on response times that were extracted using only time differences between responses, we used the bivariate generalized linear IRT model framework (B-GLIRT, [1]) to investigate response times as indicators for response processes. A parameterization that includes an interaction between the latent speed factor and the latent ability factor in the cross-relation function was found to fit the data best in CBA and PBA. Data were collected with a within-subject design in a national add-on study to PISA 2012 administering two clusters of PISA 2009 reading units. After investigating the invariance of the measurement models for ability and speed between boys and girls, we found the expected gender effect in reading ability to coincide with a gender effect in speed in CBA. Taking this result as indication for the validity of the time measures extracted from time differences between responses, we analyzed the PBA data and found the same gender effects for ability and speed. Analyzing PBA and CBA data together we identified the ability mode effect as the latent difference between reading measured in CBA and PBA. Similar to the gender effect the mode effect in ability was observed together with a difference in the latent speed between modes. However, while the relationship between speed and ability is identical for boys and girls we found hints for mode differences in the estimated parameters of the cross-relation function used in the B-GLIRT model. (DIPF/Orig.)
DIPF-Departments: Bildungsqualität und Evaluation
-
-
Author(s): Lonnemann, Jan; Müller, Christian; Büttner, Gerhard; Hasselhorn, Marcus
Title: The influence of visual-spatial skills on the association between processing of nonsymbolic numerical magnitude and number word sequence skills
In: Journal of Experimental Child Psychology, 178 (2019) , S. 184-197
DOI: 10.25656/01:17467
URN: urn:nbn:de:0111-pedocs-174677
URL: http://nbn-resolving.org/urn:nbn:de:0111-pedocs-174677
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Deutschland; Einflussfaktor; Raumvorstellung; Mathematische Kompetenz; Mathematisches Denken; Kind; Vorschulalter; Grundschulalter; Visuelle Wahrnehmung; Zahlensystem; Wort; Assoziation; Größe; Vergleich; Arithmetik; Test
Abstract (english): Nonsymbolic numerical magnitude processing skills are assumed to be fundamental to mathematical learning. Recent findings suggest that visual-spatial skills account for associations between children's performance in visually presented nonsymbolic numerical magnitude comparison tasks and their performance in visually presented arithmetic tasks. The aim of the current study was to examine whether associations between children's performance in visually presented tasks assessing nonsymbolic numerical magnitude processing skills and their performance in tasks assessing early mathematical skills, which do not involve visual stimulation, may also be mediated by visual-spatial skills. This line of reasoning is based on the assumption that children make use of mental visualization processes when working on tasks assessing early mathematical skills, such as knowledge of the sequence of number words, even when these tasks do not involve visual stimulation. We assessed 4- to 6-year-old children's performance in a nonsymbolic numerical magnitude comparison task, in tasks concerning knowledge of the sequence of number words, and in a developmental test to assess visual-spatial skills. Children's nonsymbolic numerical magnitude processing skills were found to be associated with their number word sequence skills. This association was fully mediated by interindividual differences in visual-spatial skills. The effect size of this mediation effect was small. We assume that the ability to construct mental visualizations constitutes the key factor under- lying this mediation effect. (DIPF/Orig.)
DIPF-Departments: Bildung und Entwicklung
-
-
Author(s): Naumann, Alexander; Rieser, Svenja; Musow, Stephanie; Hochweber, Jan; Hartig, Johannes
Title: Sensitivity of test items to teaching quality
In: Learning and Instruction, 60 (2019) , S. 41-53
DOI: 10.1016/j.learninstruc.2018.11.002
URL: https://www.sciencedirect.com/science/article/pii/S0959475217307065?via%3Dihub
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Leistungstest; Testkonstruktion; Unterricht; Qualität; Einflussfaktor; Testauswertung; Grundschule; Naturwissenschaftlicher Unterricht; Aktives Lernen; Entdeckendes Lernen; Unterrichtsmethode; Wirkung; Messverfahren; Testaufgabe; Problemlösen; Grundschüler; Dauer; Antwort; Schwierigkeit; Datenanalyse; Interpretation; Quasi-Experiment; Deutschland
Abstract: Instructional sensitivity is the psychometric capacity of tests or single items of capturing effects of classroom instruction. Yet, current item sensitivity measures' relationship to (a) actual instruction and (b) overall test sensitivity is rather unclear. The present study aims at closing these gaps by investigating test and item sensitivity to teaching quality, reanalyzing data from a quasi-experimental intervention study in primary school science education (1026 students, 53 classes, Mage = 8.79 years, SDage = 0.49, 50% female). We examine (a) the correlation of item sensitivity measures and the potential for cognitive activation in class and (b) consequences for test score interpretation when assembling tests from items varying in their degree of sensitivity to cognitive activation. Our study (a) provides validity evidence that item sensitivity measures may be related to actual classroom instruction and (b) points out that inferences on teaching drawn from test scores may vary due to test composition. (DIPF/Orig.)
DIPF-Departments: Bildungsqualität und Evaluation
-
-
Author(s): Teltemann, Janna; Jude, Nina
Title: Assessments and accountability in secondary education. International trends
In: Research in Comparative and International Education, 14 (2019) 2, S. 249-271
DOI: 10.1177/1745499919846174
URL: https://journals.sagepub.com/doi/10.1177/1745499919846174
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Sekundarbereich; Schüler; Leistungsbeurteilung; Verantwortung; Vorkommen; Schülerleistungstest; Bildungsmonitoring; Trend; Internationaler Vergleich
Abstract (english): International Large-scale Student Assessments are the most prominent example of internationalization processes in education. A number of studies have analysed the policy reactions to such studies, particularly to the Organisation for Economic Co-operation and Development (OECD) Programme for International Student Assessment (PISA) study. PISA and comparable projects have also raised concerns of a feared increase of assessments and accountability procedures. So far, systematic empirical evidence that could corroborate such concerns has been scarce. In this paper, we examine the prevalence of assessment and accountability practices at secondary education level as well as changes in these practices over time. We explicitly focus on changes over time by drawing on data from PISA 2000 to PISA 2015. Analyses over time are not straightforward with PISA, as the questionnaires change between survey rounds. This leads to different coverage of specific indicators over time. We present descriptive analyses for 20 OECD countries. The results show an increasing trend for the vast majority of the generated indicators, indicating that assessments and the use of assessments for purposes of accountability increased within the larger part of the OECD during the last 15 years. Likewise, more horizontal, peer-oriented evaluation procedures focusing on organizational learning gained importance. A cluster analysis based on selected indicators of 2015 revealed that there are four distinct groups of countries, which are mainly distinguished by different levels of the prevalence of assessment, accountability and evaluation practices. (DIPF/Orig.)
DIPF-Departments: Bildungsqualität und Evaluation
-
-
Author(s): Zehner, Fabian; Goldhammer, Frank; Lubaway, Emily; Sälzer, Christine
Title: Unattended consequences. How text responses alter alongside PISA's mode change from 2012 to 2015
In: Education Inquiry, 10 (2019) 1, S. 34-55
DOI: 10.1080/20004508.2018.1518080
URL: https://www.tandfonline.com/doi/pdf/10.1080/20004508.2018.1518080?needAccess=true
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Beitrag in Sonderheft
Language: Englisch
Keywords: PISA <Programme for International Student Assessment>; Schülerleistungstest; Design; Veränderung; Computerunterstütztes Verfahren; Lesetest; Antwort; Text; Unterschied; Information; Relevanz; Schuljahr 09; Empirische Untersuchung; Deutschland
Abstract (english): In 2015, the Programme for International Student Assessment (PISA) introduced multiple changes in its study design, the most extensive being the transition from paper- to computer-based assessment. We investigated the differences between German students' text responses to eight reading items from the paper-based study in 2012 to text responses to the same items from the computer-based study in 2015. Two response features - information quantity and relevance proportion - were extracted by natural language processing techniques because they are crucial indicators for the response process. Showcasing potential differential relationships, we additionally examined gender differences. Modelling effects of the round of assessment, gender, and response correctness on the response features, we analysed responses from 15-year-olds and ninth-graders in Germany. Results revealed differences in the text responses between the rounds of assessment in that students included more information overall in 2015, and the proportions of relevance varied substantially across items. As the study investigated the mode change in PISA's natural (not experimental) setting, the differences could mirror cohort trends or design changes. However, with the evidence reported, we conclude that the differences could indicate mode effects. (DIPF/Orig.)
DIPF-Departments: Bildungsqualität und Evaluation