-
-
Author(s): Deribo, Tobias; Goldhammer, Frank; Kröhne, Ulf
Title: Changes in the speed-ability relation through different treatments of rapid guessing
In: Educational and Psychological Measurement, 83 (2023) 3, S. 473-494
DOI: 10.1177/00131644221109490
URL: https://journals.sagepub.com/doi/10.1177/00131644221109490
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Antwort; Deutschland; Empirische Untersuchung; Fertigkeit; Informations- und Kommunikationstechnologie; Item-Response-Theory; Leistungstest; Modell; Panel; Psychometrie; Reliabilität; Student; Test; Validität; Verhalten; Zeit
Abstract (english): As researchers in the social sciences, we are often interested in studying not directly observable constructs through assessments and questionnaires. But even in a well-designed and well-implemented study, rapid-guessing behavior may occur. Under rapid-guessing behavior, a task is skimmed shortly but not read and engaged with in-depth. Hence, a response given under rapid-guessing behavior does bias constructs and relations of interest. Bias also appears reasonable for latent speed estimates obtained under rapid-guessing behavior, as well as the identified relation between speed and ability. This bias seems especially problematic considering that the relation between speed and ability has been shown to be able to improve precision in ability estimation. For this reason, we investigate if and how responses and response times obtained under rapid-guessing behavior affect the identified speed-ability relation and the precision of ability estimates in a joint model of speed and ability. Therefore, the study presents an empirical application that highlights a specific methodological problem resulting from rapid-guessing behavior. Here, we could show that different (non-)treatments of rapid guessing can lead to different conclusions about the underlying speed-ability relation. Furthermore, different rapid-guessing treatments led to wildly different conclusions about gains in precision through joint modeling. The results show the importance of taking rapid guessing into account when the psychometric use of response times is of interest. (DIPF/Orig.)
DIPF-Departments: Lehr und Lernqualität in Bildungseinrichtungen
-
-
Author(s): Becker, Michael; Maaz, Kai
Title: Zentrale quantitative Auswertungsverfahren
In: Harring, Marius; Rohlfs, Carsten; Gläser-Zikuda, Michaela (Hrsg.): Handbuch Schulpädagogik, Münster: Waxmann, 2022 , S. 948-958
Publication Type: 4. Beiträge in Sammelbänden; Lexika/Enzyklopädie o.ä.
Language: Deutsch
Keywords: Fehlende Daten; Quantitative Forschung; Schulforschung; Bildungsforschung; Statistische Methode; Modellierung; Stichprobe; Datenanalyse; Verfahren; Mehrebenenanalyse; Pfadanalyse; Skalierung; Längsschnittuntersuchung; Panel
Abstract: [Im Beitrag wird] nach einer kurzen Darstellung der Gemeinsamkeiten statistischer Modellierung, die die Schul- und Bildungsforschung mit den meisten Sozialwissenschaften teilt, ein Überblick über die spezifischen Auswertungsverfahren gegeben, die sich vor allem in der quantitativen Bildungs- und Schulforschung häufig finden. Hierbei wird weniger auf die Darstellung spezifischer Kennwerte und ihrer Eigenschaften als auf die generelle Verwendung und den epistemologischen Zweck der Verfahren eingegangen. Der Fokus ist darauf gelegt, die wichtigsten statistischen Probleme und gegenwärtig gängigen analytischen Lösungen, wie sie in einer Vielzahl von Studien der Bildungs- und Schulforschung vorzufinden und zu adressieren sind, auf einer konzeptuellen Ebene zusammenzufassen und Hinweise für einen tieferen Einstieg in die Themen zu geben. Insofern sollen - nach einer kurzen Erläuterung der beiden grundlegenden Aspekte statistischer Modellierung (Deskriptive und Inferenzstatistik) im Allgemeinen - vier Aspekte quantitativer Auswertungen vorgestellt werden, wie sie unserer Meinung nach zentral für die quantitative Bildungs- und Schulforschung sind: (1) Hierarchische Datenstrukturen und Mehrebenenmodelle, (2) Pfadanalysen und längsschnittliche cross-lagged panel-Analysen, Fragen (3) fehlender Werte und (4) der Leistungsskalierung. (DIPF/Orig.)
DIPF-Departments: Struktur und Steuerung des Bildungswesens
-
-
Author(s): Hoffmann, Lars; Hübner, Nicolas; Neumann, Marko; Schröter, Pauline
Title: Und wenn man die Abiturprüfungen einfach ausfallen ließe? Empirische Befunde zu Unterschieden zwischen Abiturprüfungsnoten und Kursnoten
In: Hoffmann, Lars; Schröter, Pauline; Groß, Alexander; Schmid-Kühn, Svenja Mareike; Stanat, Petra (Hrsg.): Das unvergleichliche Abitur: Entwicklungen - Herausforderungen - empirische Analysen, Bielefeld: wbv Media, 2022 , S. 251-287
URL: https://www.wbv.de/download/shop/download/0/_/0/0/listview/file/-direct%40I72494/area/openaccess.html?cHash=c1866ddf63d2f8fe8fa5d9c949bd05de#%5B%7B%22num%22%3A790%2C%22gen%22%3A0%7D%2C%7B%22name%22%3A%22XYZ%22%7D%2Cnull%2C606.6142%2Cnull%5D
Publication Type: 4. Beiträge in Sammelbänden; Sammelband (keine besondere Kategorie)
Language: Deutsch
Keywords: Abiturprüfung; Abschlussprüfung; Berlin; Datenanalyse; Deutschland; Deutschunterricht; Empirische Forschung; Englischunterricht; Gymnasiale Oberstufe; Kurssystem; Mathematikunterricht; Metaanalyse; Oberstufenreform; Panel; Schulnoten; Thüringen; Unterschied
Abstract: Im vorliegenden Beitrag werden die Ergebnisse von drei Studien dargestellt, die jeweils die Passung zwischen Abiturprüfungsnoten auf der einen Seite und den in der Qualifikationsphase der gymnasialen Oberstufe erreichten Kursnoten auf der anderen Seite anhand unterschiedlicher Datensätze (d.h. Daten aus der Evaluation zur Bewährung der Poolaufgaben, Daten aus einer Zusatzstudie des Nationalen Bildungspanels, Daten aus der BERLIN-Studie) und mit jeweils anderen Schwerpunktsetzungen untersucht haben. Insgesamt verdeutlichen die Ergebnisse der drei Studien, dass die in den Abiturprüfungen erzielten Noten im Mittel etwas schlechter ausfallen als die Kursnoten der Qualifikationsphase. Darüber hinaus zeigen die Befunde, dass die Passung zwischen Prüfungsnoten und Kursnoten erheblich zwischen Ländern, Schulen und Fächern variiert und möglicherweise auch von schulsystemischen Änderungen (z.B. Oberstufenreformen) beeinflusst wird. Die Ergebnisse werden im Beitrag auch vor dem Hintergrund der in der Corona-Pandemie verschiedentlich diskutierten Möglichkeit eines Anerkennungsabiturs unter Verzicht auf die Abiturprüfung beleuchtet. (DIPF/Orig.)
DIPF-Departments: Struktur und Steuerung des Bildungswesens
-
-
Author(s): Deribo, Tobias; Kröhne, Ulf; Goldhammer, Frank
Title: Model‐based treatment of rapid guessing
In: Journal of Educational Measurement, 58 (2021) 2, S. 281-303
DOI: 10.1111/jedm.12290
URL: https://onlinelibrary.wiley.com/doi/10.1111/jedm.12290?af=R
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Leistungstest; Testkonstruktion; Messverfahren; Computerunterstütztes Verfahren; Frage; Antwort; Verhalten; Dauer; Problemlösen; Modell; Student; Medienkompetenz; Item-Response-Theory; Multiple-Choice-Verfahren; Validität; Panel; Längsschnittuntersuchung
Abstract (english): The increased availability of time-related information as a result of computer-based assessment has enabled new ways to measure test-taking engagement. One of these ways is to distinguish between solution and rapid guessing behavior. Prior research has recommended response-level filtering to deal with rapid guessing. Response-level filtering can lead to parameter bias if rapid guessing depends on the measured trait or (un-)observed covariates. Therefore, a model based on Mislevy and Wu (1996) was applied to investigate the assumption of ignorable missing data underlying response-level filtering. The model allowed us to investigate different approaches to treating response-level filtered responses in a single framework through model parameterization. The study found that lower-ability test-takers tend to rapidly guess more frequently and are more likely to be unable to solve an item they guessed on, indicating a violation of the assumption of ignorable missing data underlying response-level filtering. Further ability estimation seemed sensitive to different approaches to treating response-level filtered responses. Moreover, model-based approaches exhibited better model fit and higher convergent validity evidence compared to more naïve treatments of rapid guessing. The results illustrate the need to thoroughly investigate the assumptions underlying specific treatments of rapid guessing as well as the need for robust methods. (DIPF/Orig.)
DIPF-Departments: Lehr und Lernqualität in Bildungseinrichtungen
-
-
Author(s): Hadjar, Andreas; Scharf, Jan; Hascher, Tina
Title: Who aspires to higher education? Axes of inequality, values of education and higher education aspirations in secondary schools in Luxembourg and the Swiss Canton of Bern
In: European Journal of Education, 56 (2021) 1, S. 9-26
DOI: 10.1111/ejed.12435
URL: https://onlinelibrary.wiley.com/doi/10.1111/ejed.12435
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Beitrag in Sonderheft
Language: Englisch
Keywords: Bildungsaspiration; Hochschulbildung; Schüler; Sekundarstufe I; Soziale Ungleichheit; Sozioökonomische Lage; Bildungssystem; Bildungsverhalten; Wertschätzung; Schuljahr 07; Schuljahr 08; Schuljahr 09; Schulform; Geschlechtsspezifischer Unterschied; Migrationshintergrund; Befragung; Panel; Internationaler Vergleich; Bern <Kanton>; Schweiz; Luxemburg
Abstract: This article reports a study that investigated secondary school students' higher education aspirations (towards university studies, ISCED 6 and above) and how these differ between student groups as well as how these are impacted by values of education. Panel data of more than 300 secondary school students in two countries, Luxembourg and Switzerland (the Swiss Canton of Bern) was analysed. Schools are structured differently in the education systems of Luxembourg and the Swiss Canton of Bern. The results of our analysis show that students in the Luxembourgish sample more often aspire to higher education than in the Swiss sample. Disparities in higher education aspirations were also more pronounced in the Luxembourgish sample, boys and students from families of low socio-economic status (SES) were less likely to aspire to higher education. While the effects of values of education are generally scarce, stimulation in terms of anticipated enjoyment and interest derived from participation in higher education seems to have a positive effect on higher education aspirations. (DIPF/Orig.)
DIPF-Departments: Struktur und Steuerung des Bildungswesens
-
-
Author(s): Möwisch, Dave; Brose, Annette; Schmiedek, Florian
Title: Do higher educated people feel better in everyday life? Insights from a day reconstruction method study
In: Social Indicators Research, 153 (2021) , S. 227-250
DOI: 10.1007/s11205-020-02472-y
URL: https://link.springer.com/article/10.1007%2Fs11205-020-02472-y
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Bildungsniveau; Wohlbefinden; Affektives Merkmal; Alltagserfahrung; Tagesablauf; Rekonstruktion; Datenanalyse; Sekundäranalyse; Sozioökonomisches Panel; Mehrebenenanalyse; Strukturgleichungsmodell; Deutschland
Abstract: Past research has shown a positive association between education and well-being. Much of this research has focused on the cognitive component of well-being (i.e., life satisfaction) as outcome. On the other hand, the affective component, that is, how often and intensively people experience positive affect (PA) and negative affect (NA) in their everyday lives, has received far less attention. Therefore, we examined the association between education and PA and NA in everyday life, with a particular focus on affective experiences at the sub-facet level (based on a structure of NA with multiple factors). We used data from a nationally representative sample (N = 1647) of the German Socioeconomic Panel Innovation Sample (SOEP-IS), employing the Day Reconstruction Method (DRM) to capture affective experiences of everyday activities. Multilevel structural equation models revealed that (1) education was not related to PA, but (2) was positively associated with two sub-facets of NA (mourning/worries and loneliness/boredom); (3) income might in part explain the association between education and NA; (4) education does not particularly seem to serve as a resource in times of unemployment or retirement (i.e., there were no interactions between education and unemployment/retirement regarding well-being) In essence, higher educated people reported fewer negative emotions in everyday life than their lower educated counterparts, but not more positive emotions. The findings underline that different facets of NA, in addition to life satisfaction, are relevant variables related to education and should receive more attention in order to gain a more comprehensive understanding of non-monetary correlates of education. (DIPF/Orig.)
DIPF-Departments: Bildung und Entwicklung
-
-
Author(s): Brandt, Naemi D.; Lechner, Clemens M.; Tetzner, Julia; Rammstedt, Beatrice
Title: Personality, cognitive ability, and academic performance. Differential associations across school subjects and school tracks
In: Journal of Personality, 88 (2020) 2, S. 249-265
DOI: 10.1111/jopy.12482
URL: https://onlinelibrary.wiley.com/doi/full/10.1111/jopy.12482
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Schüler; Persönlichkeitsmerkmal; Persönlichkeit; Kognitive Kompetenz; Intelligenz; Emotionale Kompetenz; Schulerfolg; Schulform; Unterrichtsfach; Deutschunterricht; Mathematikunterricht; Schuljahr 09; Einflussfaktor; Gymnasium; Realschule; Berufsschule; Schülerleistung; Test; Panel; Strukturgleichungsmodell; Längsschnittuntersuchung; Deutschland;
Abstract: Aim: Personality traits and cognitive ability are well‐established predictors of academic performance. Yet, how consistent and generalizable are the associations between personality, cognitive ability, and performance? Building on theoretical arguments that trait-performance relations should vary depending on the demands and opportunities for trait expression in the learning environment, we investigated whether the associations of personality (Big Five) and cognitive ability (fluid intelligence) with academic performance (grades and tests scores) vary across school subjects (German and math) and across ability‐grouped school tracks (academic, intermediate, and vocational). Method: Multiple group structural equation models in a large representative sample of ninth‐grade students (N = 12,915) from the German National Educational Panel Study (NEPS). Results: Differential associations across school subjects emerged for cognitive ability, Emotional Stability, and Conscientiousness (math > German); and for Openness and Extraversion (German > math). Differential associations across school tracks emerged for cognitive ability, Conscientiousness (academic > intermediate > vocational) and Agreeableness (academic > intermediate > vocational). Personality traits explained more variation in academic performance in the academic than in the other tracks. Conclusion: Most trait-performance relations varied across subjects, tracks, or both. These findings highlight the need for more nuanced and context‐minded perspective on trait-performance relations. (DIPF/Orig.)
DIPF-Departments: Struktur und Steuerung des Bildungswesens
-
-
Author(s): Hübner, Nicolas; Wagner, Wolfgang; Hochweber, Jan; Neumann, Marko; Nagengast, Benjamin
Title: Comparing apples and oranges. Curricular intensification reforms can change the meaning of students' grades!
In: Journal of Educational Psychology, 112 (2020) 1, S. 204-220
DOI: 10.18747/PHSG-coll3/id/378
URL: https://www.pedocs.de/frontdoor.php?source_opus=18970
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Schulreform; Gymnasiale Oberstufe; Curriculumreform; Einflussfaktor; Leistungsbeurteilung; Schülerleistung; Schulnoten; Lehrerurteil; Leistungstest; Standard; Englischunterricht; Mathematikunterricht; Panel; Vergleich; Thüringen; Baden-Württemberg
Abstract (english): Teacher-assigned grades provide important information that is used by universities and colleges to make admission decisions and by employers to make hiring decisions. Besides grades, the results of standardized achievement tests are frequently used for student selection and allocation. However, correlations between the two achievement measures are far from perfect, and researchers have argued that this discrepancy can be at least partially attributed to norm-referenced grading, which is based on the composition of performance in a class. In this study, we investigated the results of a curricular intensification reform, which introduced changes in the distribution of student performance by making enrollment in certain courses mandatory. We investigated whether the reform resulted in changes in the relationship between standardized achievement-test results and teacher-assigned grades. We analyzed cohort control design data from two large representative samples of students from two German states (Baden-Württemberg: N = 5,574; Thuringia: N = 2,202) before and after the reform. Results indicated that students who received a certain grade before the reform (e.g., a grade of A, B, C, or D) differed in their standardized test performance from students who received the same grade after the reform. Furthermore, in math, course-level-specific reform effects on the association between grades and standardized test performance were found to vary between groups of students receiving good and poor grades. Implications for educational policy and school reforms and suggestions for grading are discussed. (DIPF/Orig.)
DIPF-Departments: Bildungsqualität und Evaluation; Struktur und Steuerung des Bildungswesens
-
-
Author(s): Keyserlingk, Luise von; Becker, Michael; Jansen, Malte; Maaz, Kai
Title: Effects of student composition in school on young adults' educational pathways
In: Journal of Educational Psychology, 112 (2020) 6, S. 1261-1272
DOI: 10.1037/edu0000411
URL: https://doi.apa.org/doiLanding?doi=10.1037%2Fedu0000411
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Schüler; Zusammensetzung; Sekundarbereich; Wirkung; Junger Erwachsener; Bildungsbiografie; Bildungserfolg; Einflussfaktor; Fähigkeit; Selbstkonzept; Schülerleistung; Sozioökonomische Lage; Schuljahr 12; Hochschulbildung; Hochschulabschluss; Befragung; Fragebogen; Leistungstest; Panel; Längsschnittuntersuchung; Strukturgleichungsmodell; Faktorenanalyse; Empirische Untersuchung; Deutschland
Abstract (english): Which factors help young adults choose educational pathways leading to higher educational attainment? Academic self-concept (ASC), achievement, and socioeconomic background have all been found to be important predictors of postsecondary educational choices and success. Although research has shown that student composition in secondary school may affect some of these predictors, only a few studies (mostly from the United States) have investigated the effects of student composition on postsecondary educational outcomes. The results showed that students with similar individual achievement had lower postsecondary educational attainment if they graduated from secondary schools with higher mean achievement. It has been proposed that social comparison processes explain this negative context effect (big-fish-little-pond effect [BFLPE]). In contrast, students with the same individual socioeconomic status (SES) had higher postsecondary educational attainment if they graduated from secondary schools with a higher mean SES. In the present study, we investigated the effects of achievement-related and socioeconomic student composition on subsequent educational outcomes using data from a longitudinal study in Germany. Contrary to previous studies, our results showed that student composition had little relevance for later educational pathways. There was a small, long-lasting, indirect BFLPE of achievement-related composition in secondary school on postsecondary educational outcomes through students' ASC. Furthermore, individual SES was strongly related to postsecondary educational outcomes, whereas being in an academic-track school with a higher or lower mean SES was not relevant for postsecondary educational pathways.
DIPF-Departments: Struktur und Steuerung des Bildungswesens
-
-
Author(s): Ehm, Jan-Henning; Hasselhorn, Marcus; Schmiedek, Florian
Title: Analyzing the developmental relation of academic self-concept and achievement in elementary school children. Alternative models point to different results
In: Developmental Psychology, 55 (2019) 11, S. 2336-2351
DOI: 10.1037/dev0000796
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Selbstkonzept; Einflussfaktor; Schülerleistung; Grundschule; Grundschüler; Lesen; Lesekompetenz; Messverfahren; Panel; Modell; Methode; Unterschied; Leistungstest; Schulnoten; Vergleich; Längsschnittuntersuchung; Analyse; Baden-Württemberg; Deutschland
Abstract: The association between academic self-concept and achievement is assumed to be reciprocal. Typically, the association is analyzed by variants of the classical cross-lagged panel model. Results with more recently developed methodological approaches, for example, the random intercept cross-lagged panel model, its continuous-time implementation, and the latent change score model, are not available so far. The present study aimed to test the association between reading self-concept and reading achievement with different models to evaluate whether positive cross-lagged effects can be demonstrated with the alternative models. Drawing on a sample of 2,014 elementary students from Grade 1 to Grade 4, results of alternative models yielded noticeable differences. Support for reciprocal effects between reading self-concept and reading achievement was found using the classical and the full-forward cross-lagged panel models. No such effects were found in the other models. Thus, the reciprocal effect model can be called into question for elementary schoolchildren. The results are discussed with regard to the theoretical assumptions and appropriate methodological approaches. (DIPF/Orig.)
DIPF-Departments: Bildung und Entwicklung