Search results in the DIPF database of publications

Your query:

(Personen: "Buchholz," und "Janine")

Cross-cultural comparability of latent constructs in ILSAs He, Jia; Buchholz, Janine; Fischer, Jessica Book Chapter | Aus: Nilsen, Trude; Stancel-Piątak, Agnes; Gustafsson, Jan-Eric (Hrsg.): International handbook of comparative large-scale studies in education: Perspectives, methods and findings | Cham: Springer | 2022 44309 Endnote: Author(s): He, Jia; Buchholz, Janine; Fischer, Jessica
Title: Cross-cultural comparability of latent constructs in ILSAs
In: Nilsen, Trude; Stancel-Piątak, Agnes; Gustafsson, Jan-Eric (Hrsg.): International handbook of comparative large-scale studies in education: Perspectives, methods and findings, Cham: Springer, 2022 (Springer International Handbooks of Education), S. 845-870
DOI: 10.1007/978-3-030-88178-8_58
URL: https://link.springer.com/referenceworkentry/10.1007/978-3-030-88178-8_58
Publication Type: 4. Beiträge in Sammelbänden; Sammelband (keine besondere Kategorie)
Language: Englisch
Abstract: This chapter provides an overview of issues of cross-cultural comparability of latent constructs such as attitudes, values, and beliefs measured in ILSA questionnaires. We present the aim and scope of statistical approaches to analyze cross-cultural comparability in four parts. We first introduce the framework of bias and equivalence to guide the methodological considerations for valid cross-cultural comparisons. We then review the practice of construct validation and equivalence testing that has been documented across various ILSAs in the past, highlighting the urgency and challenges in ensuring comparability for multiple group comparisons as well as the related scientific discussions. Next, we briefly describe the recently developed approaches to accommodate the nonequivalence due to the numerous cultural, linguistic, and psychological differences in ILSA data. Finally, we discuss findings and provide recommendations for future applications for ILSA data publishers, policy stakeholders, and researchers who use ILSA data for secondary analysis or who collect data for cross-cultural comparisons. (DIPF/Orig.)
DIPF-Departments: Lehr und Lernqualität in Bildungseinrichtungen

Wie sind die Kenntnisse pädagogischer Fachkräfte zu Diagnostik und Förderung einzuschätzen? Peters, Svenja; Höltge, Lea; Buchholz, Janine; Hartmann, Ulrike; Ehm, Jan-Henning; […] Journal Article | In: Frühe Bildung | 2021 41015 Endnote: Author(s): Peters, Svenja; Höltge, Lea; Buchholz, Janine; Hartmann, Ulrike; Ehm, Jan-Henning; Hasselhorn, Marcus
Title: Wie sind die Kenntnisse pädagogischer Fachkräfte zu Diagnostik und Förderung einzuschätzen?
In: Frühe Bildung, 10 (2021) 1, S. 39-48
DOI: 10.1026/2191-9186/a000503
URL: https://econtent.hogrefe.com/doi/10.1026/2191-9186/a000503
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Deutsch
Keywords: Erzieher; Fachkraft; Diagnostik; Förderung; Kind; Vorschulalter; Schulreife; Kompetenz; Kindergarten; Kindertagesstätte; Frühkindliche Bildung; Einstellung <Psy>; Belastung; Wahrnehmung; Wissen; Sprache; Mathematik; Beobachtung; Dokumentation; Vignette <Methode>; Fallbeispiel; Latent-Class-Analyse; Deutschland
Abstract: Diagnostik und Förderung werden zunehmend als Bestandteile professioneller Bildungsarbeit in Kitas anerkannt. Ihre Umsetzung bedarf entsprechender Kompetenzen. In einer Fragebogenstudie wurden Kenntnisse zu Diagnostik und Förderung von 285 pädagogischen Fachkräften mittels einer Fallvignette untersucht. Die schriftlichen Antworten gaben erste Hinweise darauf, über welche diagnostischen und förderbezogenen Kenntnisse sie in den Bereichen Sprache und Mathematik verfügen. Durch latente Klassenanalysen wurden fünf Fachkraftgruppen mit unterschiedlichen Kenntnissen identifiziert. Diese zeigten sich entweder vorwiegend austauschorientiert (C1), vorwiegend ideenlos (C2), vorwiegend beobachtungs- und förderorientiert (C3), vorwiegend förderorientiert (C4) oder vorwiegend beobachtungs- und austauschorientiert (C5). Die Gruppen unterschieden sich u.a. hinsichtlich der Einstellungen gegenüber Diagnostik sowie der wahrgenommenen beruflichen Belastung. Konsequenzen für die Professionalisierung pädagogischer Fachkräfte sowie für zukünftige Forschung werden diskutiert. (DIPF/Orig.)
Abstract (english): Diagnostics and instructional support are key components of professional early education, and are associated with specific competencies. To contribute to this field of research, we conducted a survey study with 285 early childhood educators. Educators' knowledge of diagnostics and instructional support regarding children's language and mathematic development was assessed by a case vignette. Latent class analysis identified five groups of educators differing with regard to their knowledge. Their written answers were characterized by either pointing out cooperation with others (C1), a lack of one's own ideas (C2), emphasizing both observation and support methods (C3), a focus on support methods (C4), or focusing on methods of observation and cooperation (C5). Moreover, these groups differed in other facets of professional competencies and structural features. Significant differences were revealed regarding their beliefs toward assessment of skills and their individual stress level. Implications for professional development and further research are discussed. (DIPF/Orig.)
DIPF-Departments: Bildung und Entwicklung; Lehr und Lernqualität in Bildungseinrichtungen

Measurement invariance testing in questionnaires. A comparison of three Multigroup-CFA and […] Buchholz, Janine; Hartig, Johannes Journal Article | In: Psychological Test and Assessment Modelling | 2020 39818 Endnote: Author(s): Buchholz, Janine; Hartig, Johannes
Title: Measurement invariance testing in questionnaires. A comparison of three Multigroup-CFA and IRT-based approaches
In: Psychological Test and Assessment Modelling, 62 (2020) 1, S. 29-54
URL: https://www.psychologie-aktuell.com/fileadmin/Redaktion/Journale/ptam-2020-1/03_Buchholz.pdf
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Beitrag in Sonderheft
Language: Englisch
Keywords: PISA <Programme for International Student Assessment>; Item-Response-Theorie; Faktorenanalyse; Schülerleistung; Leistungsmessung; Messung; Invarianz; Validität; Statistische Methode
Abstract (english): International Large-Scale Assessments aim at comparisons of countries with respect to latent constructs such as attitudes, values and beliefs. Measurement invariance (MI) needs to hold in order for such comparisons to be valid. Several statistical approaches to test for MI have been proposed: While Multigroup Confirmatory Factor Analysis (MGCFA) is particularly popular, a newer, IRT-based approach was introduced for non-cognitive constructs in PISA 2015, thus raising the question of consistency between these approaches. A total of three approaches (MGCFA for ordinal and continuous data, multi-group IRT) were applied to simulated data containing different types and extents of MI violations, and to the empirical non-cognitive PISA 2015 data. Analyses are based on indices of the magnitude (i.e., parameter-specific modification indices resulting from MGCFA and group-specific item fit statistics resulting from the IRT approach) and direction of local misfit (i.e., standardized parameter change and mean deviation, respectively). Results indicate that all measures were sensitive to (some) MI violations and more consistent in identifying group differences in item difficulty parameters.
DIPF-Departments: Bildungsqualität und Evaluation

Scaling procedures and construct validation of context questionnaire data Buchholz, Janine; Jude, Nina; Komorowski, Felix; Leck, Johanna Working Papers | 2020 39820 Endnote: Author(s): Buchholz, Janine; Jude, Nina; Komorowski, Felix; Leck, Johanna
Title: Scaling procedures and construct validation of context questionnaire data
Published: Paris: OECD, 2020 (PISA 2018 technical report)
URL: https://www.oecd.org/pisa/data/pisa2018technicalreport/PISA2018_Technical-Report-Chapter-16-Background-Questionnaires.pdf
Publication Type: 5. Arbeits- und Diskussionspapiere; Forschungsbericht/Projektberichte/Schulrückmeldungen
Language: Englisch
Keywords: PISA <Programme for International Student Assessment>; Fragebogen; Skalierung; Validität; Variable; Index; Berechnung; OECD-Staaten
Abstract: The PISA 2018 Context Questionnaire are based on the questionnaire framework (OECD, 2018), described in chapter 3 of this report. Many questionnaire items were designed to be combined in some way in order to measure latent constructs that cannot be observed directly (e.g. a student's achievement motivation or economic, social and cultural background). To these items, transformations or scaling procedures were applied to contruct meaningful indices. In the following, these indices are referred to as "derived variables". This chapter describes derived variables based on one or more items that were constructed and validated for all questionnaires administered in PISA 2018.
DIPF-Departments: Bildungsqualität und Evaluation

Comparing attitudes across groups. An IRT-based item-fit statistic for the analysis of measurement […] Buchholz, Janine; Hartig, Johannes Journal Article | In: Applied Psychological Measurement | 2019 37766 Endnote: Author(s): Buchholz, Janine; Hartig, Johannes
Title: Comparing attitudes across groups. An IRT-based item-fit statistic for the analysis of measurement invariance
In: Applied Psychological Measurement, 43 (2019) 3, S. 241-250
DOI: 10.1177/0146621617748323
URN: urn:nbn:de:0111-dipfdocs-174393
URL: http://www.dipfdocs.de/volltexte/2020/17439/pdf/APM_2019_3_Buchholz_Hartig_Comparing_attitudes_across_groups_A.pdf
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Einstellung <Psy>; Messung; Fragebogen; Internationaler Vergleich; Gruppe; Vergleich; Item-Response-Theory; Skalierung; Modell; Statistische Methode; Simulation
Abstract (english): Questionnaires for the assessment of attitudes and other psychological traits are crucial in educational and psychological research, and Item Response Theory (IRT) has become a viable tool for scaling such data. Many international large-scale assessments aim at comparing these constructs across countries, and the invariance of measures across countries is thus required. In its most recent cycle, the Programme for International Student Assessment (PISA 2015) implemented an innovative approach for testing the invariance of IRT-scaled constructs in the context questionnaires administered to students, parents, school principals and teachers. On the basis of a concurrent calibration with equal item parameters across all groups (i.e., languages within countries), a group-specific item-fit statistic (root-mean-square deviance; RMSD) was used as a measure for the invariance of item parameters for individual groups. The present simulation study examines the statistic's distribution under different types and extents of (non-) invariance in polytomous items. Responses to five four-point Likert-type items were generated under the Generalized Partial Credit Model (GPCM) for 1000 simulees in 50 groups each. For one of the five items, either location or discrimination parameters were drawn from a normal distribution. In addition to this type of non-invariance, we varied the extent of non-invariance by manipulating the variation of these distributions. Results indicate that the RMSD statistic is better at detecting non-invariance related to between-group differences in item location than in item discrimination. The study's findings may be used as a starting point to sensitivity analysis aiming to define cut-off values for determining (non-) invariance. (DIPF/Orig.)
DIPF-Departments: Bildungsqualität und Evaluation

Cross-cultural comparability of noncognitive constructs in TIMSS and PISA He, Jia; Barrera-Pedemonte, Fabian; Buchholz, Janine Journal Article | In: Assessment in Education | 2019 38403 Endnote: Author(s): He, Jia; Barrera-Pedemonte, Fabian; Buchholz, Janine
Title: Cross-cultural comparability of noncognitive constructs in TIMSS and PISA
In: Assessment in Education, 26 (2019) 4, S. 369-385
DOI: 10.1080/0969594X.2018.1469467
URL: https://www.tandfonline.com/doi/full/10.1080/0969594X.2018.1469467
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Beitrag in Sonderheft
Language: Englisch
Keywords: PISA <Programme for International Student Assessment>; TIMSS <Third International Mathematics and Science Study>; Schülerleistung; Leistungsmessung; Mathematikunterricht; Naturwissenschaftlicher Unterricht; Freude; Motivation; Schule; Identifikation <Psy>; Sekundarstufe I; Schüler; Messverfahren; Vergleich; Item-Response-Theory; Faktorenanalyse; OECD-Länder
Abstract: Noncognitive assessments in Programme for International Student Assessment (PISA) and Trends in International Mathematics and Science Study share certain similarities and provide complementary information, yet their comparability is seldom checked and convergence not sought. We made use of student self-report data of Instrumental Motivation, Enjoyment of Science and Sense of Belonging to School targeted in both surveys in 29 overlapping countries to (1) demonstrate levels of measurement comparability, (2) check convergence of different scaling methods within survey and (3) check convergence of these constructs with student achievement across surveys. We found that the three scales in either survey (except Sense of Belonging to School in PISA) reached at least metric invariance. The scale scores from the multigroup confirmatory factor analysis and the item response theory analysis were highly correlated, pointing to robustness of scaling methods. The correlations between each construct and achievement was generally positive within each culture in each survey, and the correlational pattern was similar across surveys (except for Sense of Belonging), indicating certain convergence in the cross-survey validation. We stress the importance of checking measurement invariance before making comparative inferences, and we discuss implications on the quality and relevance of these constructs in understating learning. (DIPF/Orig.)
DIPF-Departments: Bildungsqualität und Evaluation

Student-perceived teaching quality. How is it related to different achievement emotions in […] Lazarides, Rebecca; Buchholz, Janine Journal Article | In: Learning and Instruction | 2019 38908 Endnote: Author(s): Lazarides, Rebecca; Buchholz, Janine
Title: Student-perceived teaching quality. How is it related to different achievement emotions in mathematics classrooms?
In: Learning and Instruction, 61 (2019) , S. 45-59
DOI: 10.1016/j.learninstruc.2019.01.001
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Abstract (english): Achievement emotions are important prerequisites for academic outcomes and well-being, yet little is known about their relation to teaching quality. This study examines the relation between student-perceived teaching quality in mathematics classrooms in grade 9 and enjoyment, anxiety, and boredom in grade 10, at both the student and classroom levels. The original data set included 6,020 students who participated in the German national extension of the Programme for International Student Assessment (PISA). Multilevel regression analyses showed that teacher support and classroom management were negatively related to student-level anxiety and boredom. Teacher support was positively related to enjoyment and negatively related to anxiety at the classroom level. Cognitive activation was positively related to enjoyment and negatively related to boredom at the classroom level. Classroom management was negatively related to classroom-level boredom. These results provide insight into differential classroom processes regarding the role of teaching quality in various aspects of student achievement emotions. (DIPF/Orig.)
DIPF-Departments: Bildungsqualität und Evaluation

Predictors of individual performance changes related to item positions in PISA assessments Wu, Qian; Debeer, Dries; Buchholz, Janine; Hartig, Johannes; Janssen, Rianne Journal Article | In: Large-scale Assessments in Education | 2019 39021 Endnote: Author(s): Wu, Qian; Debeer, Dries; Buchholz, Janine; Hartig, Johannes; Janssen, Rianne
Title: Predictors of individual performance changes related to item positions in PISA assessments
In: Large-scale Assessments in Education, (2019) , S. 7:5
DOI: 10.1186/s40536-019-0073-6
URL: https://largescaleassessmentsineducation.springeropen.com/articles/10.1186/s40536-019-0073-6
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Leistungstest; Testaufgabe; Design; Wirkung; PISA <Programme for International Student Assessment>; Naturwissenschaftliche Kompetenz; Lesekompetenz; Mathematische Kompetenz; Schülerleistung; Fragebogen; Mehrebenenanalyse; Item-Response-Theory
Abstract (english): Background: Item position effects have been a common concern in large-scale assessments as changing the order of items in booklets may have an undesired effect on test performance. If every test taker would be affected by the effect in the very same way, comparisons between groups of individuals would still be valid. However, research has shown that in addition to a general fixed effect of item positions, the extent of the effect varies considerably across individuals. These individual differences are referred to as persistence. Test takers with a high level of persistence are able to keep up their performance better throughout the test administration, whereas those with a lower level of persistence show a larger decline in their test performance. Methods: The present study applied a multilevel extended item response theory (IRT ) framework and used the data from the PISA 2006 science, 2009 reading, and 2012 mathematics assessments. The first objective of this study is to provide a systematic investigation of item position effects across the three PISA domains, partially replicating the previous studies on PISA 2006 and 2009. Second, this study aims to gain a better understanding of the nature of individual differences in position effects by relating them to student characteristics. Gender, socio-economic status, language spoken at home, and three motivational scales (enjoyment of doing the subject being assessed, effort thermometer, perseverance) were used as person covariates for persistence. Results: This study replicated and extended the results found in previous studies. An overall negative item cluster position effect and significant individual differences in this effect were found in all the countries in the three PISA domains. Furthermore, the most frequently observed effect of person covariates on persistence is gender, with girls keeping up their performance better than boys. Other predictors showed little or inconsistent effects on persistence. Conclusions: Our study demonstrated inter-individual differences as well as group differences in item position effects, which may threaten the comparability between persons and groups. The consequences and implications of item position effects and persistence for the interpretation of PISA results are discussed.
DIPF-Departments: Bildungsqualität und Evaluation

The impact of ignoring the partially compensatory relation between ability dimensions on […] Buchholz, Janine; Hartig, Johannes Journal Article | In: Psychological Test and Assessment Modeling | 2018 38692 Endnote: Author(s): Buchholz, Janine; Hartig, Johannes
Title: The impact of ignoring the partially compensatory relation between ability dimensions on norm-referenced test scores
In: Psychological Test and Assessment Modeling, 60 (2018) 3, S. 369-385
URL: https://www.psychologie-aktuell.com/fileadmin/Redaktion/Journale/ptam_3-2018_369-385.pdf
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Beitrag in Sonderheft
Language: Englisch
Keywords: Schülerleistung; Leistungsmessung; Test; Interpretation; Item-Response-Theory; Modell; Methode; Validität; Mathematische Kompetenz; Sprachfertigkeit; Simulation; Empirische Untersuchung
Abstract (english): The IRT models most commonly employed to estimate within-item multidimensionality are compensatory and suggest that some dimensions (e.g., traits or abilities) can make up for a lack in others. However, many assessment frameworks in educational large-scale assessments suggest partially compensatory relations among dimensions. In two Monte-Carlo simulation studies we varied the loading pattern, the latent correlation between dimensions and the ability distribution to evaluate the impact on test scores when a compensatory model is incorrectly applied onto partially compensatory data. Findings imply only negligible effects when true abilities are bivariate normal. Assuming a uniform distribution, however, analyses of differences in test scores demonstrated systematic effects for specific patterns of true ability: High abilities are largely underestimated when the other ability required to solve some of the items was low. These findings highlight the necessity of applying the partially compensatory model under data conditions likely to occur in educational large-scale assessments. (DIPF/Orig.)
DIPF-Departments: Bildungsqualität und Evaluation

Teacher enthusiasm and self-efficacy, student-perceived mastery goal orientation, and student […] Lazarides, Rebecca; Buchholz, Janine; Rubach, Charlott Journal Article | In: Teaching and Teacher Education | 2018 37736 Endnote: Author(s): Lazarides, Rebecca; Buchholz, Janine; Rubach, Charlott
Title: Teacher enthusiasm and self-efficacy, student-perceived mastery goal orientation, and student motivation in mathematics classrooms
In: Teaching and Teacher Education, 69 (2018) 1, S. 1-10
DOI: 10.1016/j.tate.2017.08.017
URN: urn:nbn:de:0111-pedocs-155971
URL: http://nbn-resolving.org/urn:nbn:de:0111-pedocs-155971
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Mathematiklehrer; Freude; Selbstwirksamkeit; Verhalten; Wirkung; Schüler; Motivation; Überzeugung; Wahrnehmung; Mathematikunterricht; Gymnasium; Integrierte Sekundarschule; Schuljahr 09; Schuljahr 10; Lernziel; Wertorientierung; Unterrichtsklima; Klassenführung; Strukturgleichungsmodell; Berlin
Abstract (english): The purpose of this study is to examine whether teacher enthusiasm and classroom management self-efficacy are related to classroom mastery orientation and student motivation. We used data from 803 students in grades 9 and 10 (53.3% girls) and their mathematics teachers (N = 41; 58.5% men). Student-perceived teacher enthusiasm was related to classroom mastery orientation as well as to intrinsic value and cost at the student level. Teacher-reported selfefficacy was related to classroom mastery orientation at the classroom level. At both the individual and the classroom level, classroom mastery orientation was related to attainment and utility value. (DIPF/Orig.)
DIPF-Departments: Bildungsqualität und Evaluation