Search results in the DIPF database of publications

Your query:

(Schlagwörter: "Reliabilität")

Changes in the speed-ability relation through different treatments of rapid guessing Deribo, Tobias; Goldhammer, Frank; Kröhne, Ulf Journal Article | In: Educational and Psychological Measurement | 2023 42903 Endnote: Author(s): Deribo, Tobias; Goldhammer, Frank; Kröhne, Ulf
Title: Changes in the speed-ability relation through different treatments of rapid guessing
In: Educational and Psychological Measurement, 83 (2023) 3, S. 473-494
DOI: 10.1177/00131644221109490
URL: https://journals.sagepub.com/doi/10.1177/00131644221109490
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Antwort; Deutschland; Empirische Untersuchung; Fertigkeit; Informations- und Kommunikationstechnologie; Item-Response-Theory; Leistungstest; Modell; Panel; Psychometrie; Reliabilität; Student; Test; Validität; Verhalten; Zeit
Abstract (english): As researchers in the social sciences, we are often interested in studying not directly observable constructs through assessments and questionnaires. But even in a well-designed and well-implemented study, rapid-guessing behavior may occur. Under rapid-guessing behavior, a task is skimmed shortly but not read and engaged with in-depth. Hence, a response given under rapid-guessing behavior does bias constructs and relations of interest. Bias also appears reasonable for latent speed estimates obtained under rapid-guessing behavior, as well as the identified relation between speed and ability. This bias seems especially problematic considering that the relation between speed and ability has been shown to be able to improve precision in ability estimation. For this reason, we investigate if and how responses and response times obtained under rapid-guessing behavior affect the identified speed-ability relation and the precision of ability estimates in a joint model of speed and ability. Therefore, the study presents an empirical application that highlights a specific methodological problem resulting from rapid-guessing behavior. Here, we could show that different (non-)treatments of rapid guessing can lead to different conclusions about the underlying speed-ability relation. Furthermore, different rapid-guessing treatments led to wildly different conclusions about gains in precision through joint modeling. The results show the importance of taking rapid guessing into account when the psychometric use of response times is of interest. (DIPF/Orig.)
DIPF-Departments: Lehr und Lernqualität in Bildungseinrichtungen

Measurement invariance in relation to first language. An evaluation of German reading and spelling […] Visser, Linda; Cartschau, Friederike; Goldammer, Ariane von; Brandenburg, Janin; […] Journal Article | In: Applied Measurement in Education | 2023 43274 Endnote: Author(s): Visser, Linda; Cartschau, Friederike; Goldammer, Ariane von; Brandenburg, Janin; Timmerman, Marieke; Hasselhorn, Marcus; Mähler, Claudia
Title: Measurement invariance in relation to first language. An evaluation of German reading and spelling tests
In: Applied Measurement in Education, 36 (2023) 2, S. 115-131
DOI: 10.1080/08957347.2023.2201701
URL: https://www.tandfonline.com/doi/full/10.1080/08957347.2023.2201701
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Bewertung; deutsch; Deutsch als Zweitsprache; Deutschland; Faktorenanalyse; Grundschule; Lesen; Lesetest; Messung; Muttersprache; Rechtschreibtest; Rechtschreibung; Reliabilität; Sekundarstufe I; Testkonstruktion; Textverständnis; Varianzanalyse; Vergleich
Abstract (english): The growing number of children in primary schools in Germany who have German as their second language (L2) has raised questions about the fairness of performance assessment. Fair tests are a prerequisite for distinguishing between L2 learning delay and a specific learning disability. We evaluated five commonly-used reading and spelling tests for measurement invariance (MI) as a function of first language (German vs. other). Multi-group confirmatory factor analyses revealed strict MI for the Weingarten Basic Vocabulary Spelling Tests (WRTs) 3+ and 4+ and the Salzburger Reading (SLT) and Spelling (SRT) Tests, suggesting these instruments are suitable for assessing reading and spelling skills regardless of first language. The MI for A Reading Comprehension Test for First to Seventh Graders - 2nd Edition (ELFE II) was partly strict with unequal intercepts for the text subscale. We discuss the implications of this finding for assessing reading performance of children with L2. (DIPF/Orig.)
DIPF-Departments: Bildung und Entwicklung

Developing and validating a short-form questionnaire for the assessment of seven facets of […] Franzen, Patrick; Arens, A. Katrin; Greiff, Samuel; Westhuizen, Lindie van der; Fischbach, Antoine; […] Journal Article | In: Journal of Personality Assessment | 2022 42077 Endnote: Author(s): Franzen, Patrick; Arens, A. Katrin; Greiff, Samuel; Westhuizen, Lindie van der; Fischbach, Antoine; Wollschläger, Rachel; Niepel, Christoph
Title: Developing and validating a short-form questionnaire for the assessment of seven facets of conscientiousness in large-scale assessments
In: Journal of Personality Assessment, 104 (2022) 6, S. 759-773
DOI: 10.1080/00223891.2021.1998083
URL: https://www.tandfonline.com/doi/full/10.1080/00223891.2021.1998083
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Auswahl; Bildungsforschung; Datenanalyse; Entwicklung; Item; Luxemburg; Messbarkeit; Messinstrument; Pflichtbewusstsein; Psychometrie; Reliabilität; Schüler; Schülerin; Schuljahr 09; Studie; Validität
Abstract: Conscientiousness is the most important personality predictor of academic achievement. It consists of several lower order facets with differential relations to academic achievement. There is currently no short instrument assessing facets of conscientiousness in the educational context. Therefore, in the present multi-study report, we develop and validate a short-form questionnaire for the assessment of seven Conscientiousness facets, namely Industriousness, Perfectionism, Tidiness, Procrastination Refrainment, Control, Caution, and Task Planning. To this end, we examined multiple representative samples totaling N = 14,604 Grade 9 and 10 students from Luxembourg. The questionnaire was developed by adapting and shortening an existing scale using an exhaustive search algorithm. The algorithm was specified to select the best item combination based on model fit, reliability, and measurement invariance across the German and French language versions. The resulting instrument showed the expected factorial structure. The relations of the facets with personality constructs and academic achievement were in line with theoretical assumptions. Reliability was acceptable for all facets. Measurement invariance across language versions, gender, immigration status and cohort was established. We conclude that the presented questionnaire provides a short measurement of seven facets of Conscientiousness with valid and reliable scores. (DIPF/Orig.)
DIPF-Departments: Bildung und Entwicklung

Are questionnaire scales which measure non-cognitive constructs suitable as school effectiveness […] Grützmacher, Luisa; Vieluf, Svenja; Hartig, Johannes Journal Article | In: School Effectiveness and School Improvement | 2021 41472 Endnote: Author(s): Grützmacher, Luisa; Vieluf, Svenja; Hartig, Johannes
Title: Are questionnaire scales which measure non-cognitive constructs suitable as school effectiveness criteria? A measurement invariance analysis
In: School Effectiveness and School Improvement, 32 (2021) 3, S. 430-447
DOI: 10.1080/09243453.2021.1903511
URL: https://www.tandfonline.com/doi/full/10.1080/09243453.2021.1903511
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Schule; Effektivität; Messverfahren; Fragebogen; Reliabilität; Validität; Schüler; Sekundarbereich; Deutschunterricht; Lesen; Einstellung <Psy>; Mathematikunterricht; Interesse; Selbstkonzept; Mitarbeit; Längsschnittuntersuchung; Datenanalyse; Hamburg; Deutschland
Abstract (english): This study aimed at examining the suitability of questionnaire instruments commonly used in large-scale assessments for measuring non-cognitive school effectiveness criteria. It focused on questions of reliability and validity for capturing changes in students within schools across time and the instruments' sensitivity to school effects. The aim was also to propose an approach for analyzing measurement invariance across levels and time simultaneously. The study used longitudinal data from the KESS (Kompetenzen und Einstellungen von Schülerinnen und Schülern [competencies and attitudes of students]) study, conducted in Hamburg, Germany. The sample is comprised of 17,926 students in 189 secondary schools. Data were collected three or four times. The results of the analyses show that all investigated scales are suitable in terms of reliability and validity. However, only the scale interest in mathematics is sensitive to school effects and therefore suitable as a school effectiveness criterion. (DIPF/Orig.)
DIPF-Departments: Lehr und Lernqualität in Bildungseinrichtungen

Questionnaire scale characteristics Mihaly, Kata; Klieme, Eckhard; Fischer, Jessica; Doan, Sy Book Chapter | Aus: OECD (Hrsg.): Global teaching insights technical report | Paris: OECD Publishing | 2021 42209 Endnote: Author(s): Mihaly, Kata; Klieme, Eckhard; Fischer, Jessica; Doan, Sy
Title: Questionnaire scale characteristics
In: OECD (Hrsg.): Global teaching insights technical report, Paris: OECD Publishing, 2021 , S. 1-22
URL: https://www.oecd.org/education/school/GTI-TechReport-Chapter18.pdf
Publication Type: 4. Beiträge in Sammelbänden; Sammelband (keine besondere Kategorie)
Language: Englisch
Keywords: Psychometrie; Fragebogen; Skala; Itemanalyse; Reliabilität; Messung; Invarianz; Unterrichtsforschung
Abstract: Der Beitrag beschreibt die psychometrischen Kennwerte der in der TALIS Video Studie verwendeten Fragebogen-Skalen einschließlich einer Überprüfung der Messinvarianz. (DIPF/Autor)
Abstract (english): The chapter documents the psychometric features of all questionnaire scales used in the TALIS Vdieo Study, including checks of measurement invariance for some of the scales. (DIPF/Orig.)
DIPF-Departments: Lehr und Lernqualität in Bildungseinrichtungen

Validation of the network of relationship inventory in female and male adolescents Ackermann, Katharina; Martinelli, Anne; Bernhard, Anka; Ueno, Kathrin; Freitag, Christine M.; […] Journal Article | In: European Journal of Psychological Assessment | 2020 40425 Endnote: Author(s): Ackermann, Katharina; Martinelli, Anne; Bernhard, Anka; Ueno, Kathrin; Freitag, Christine M.; Büttner, Gerhard; Schmiedek, Florian; Schwenck, Christina
Title: Validation of the network of relationship inventory in female and male adolescents
In: European Journal of Psychological Assessment, 36 (2020) 2, S. 220-228
DOI: 10.1027/1015-5759/a000508
URN: urn:nbn:de:0111-pedocs-218947
URL: https://nbn-resolving.org/urn:nbn:de:0111-pedocs-218947
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Deutschland; Jugendlicher; Freundschaft; Qualität; Messung; Junge; Mädchen; Geschlechtsspezifischer Unterschied; Soziale Beziehung; Einflussfaktor; Verhalten; Aggression; Validität; Erhebungsinstrument; Reliabilität; Fragebogenerhebung; Faktorenanalyse
Abstract: Friendships and their different qualities have been shown to be important for adolescents' socio-emotional development and psychological adjustment. In empirical research on such friendship qualities, the Network of Relationship Inventory-Relationship Quality Version (NRI-RQV) is a widely used questionnaire. Here, we conduct an extensive validation of a German version of the NRI-RQV, investigating its factor structure, reliability, and concurrent validity, in a sample of N= 679 adolescents aged 13-18 years. Applying multigroup confirmatory factor analysis, we further test whether the factor structure of the friendship quality construct holds across groups of males and females. Results showed that a structure with nine correlated first-order factors fit the data well, indicating nine distinct friendship qualities in males and females. Measurement invariance testing suggested the same underlying friendship quality construct, albeit differences in mean scores per gender. As evidence for concurrent validity, closeness and discordant friendship qualities showed expected correlations with empathy and social problems, respectively, but not with aggressive behavior. Overall, results indicate good psychometric properties for the German version of the NRI-RQV as a measure of friendship qualities in both males and females. (DIPF/Orig.)
DIPF-Departments: Bildung und Entwicklung

Do student ratings provide reliable and valid information about teaching quality at the school […] Aditomo, Anindito; Köhler, Carmen Journal Article | In: Educational Assessment, Evaluation and Accountability | 2020 40512 Endnote: Author(s): Aditomo, Anindito; Köhler, Carmen
Title: Do student ratings provide reliable and valid information about teaching quality at the school level? Evaluating measures of science teaching in PISA 2015
In: Educational Assessment, Evaluation and Accountability, 32 (2020) 3, S. 275-310
DOI: 10.1007/s11092-020-09328-6
URL: https://link.springer.com/article/10.1007/s11092-020-09328-6
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Naturwissenschaftlicher Unterricht; Qualität; Messung; Evaluation; Schüler; Bewertung; Schülerurteil; Validität; Reliabilität; Schulklima; Einflussfaktor; Wirkung; PISA <Programme for International Student Assessment>; Modell; Faktorenanalyse; Empirische Untersuchung; OECD-Länder
Abstract: Large-scale educational surveys, including PISA, often collect student ratings to assess teaching quality. Because of the sampling design in PISA, student ratings must be aggregated at the school level instead of the classroom level. To what extent does school-level aggregation of student ratings yield reliable and valid measures of teaching quality? We investigate this question for six scales measuring classroom management, emotional support, inquiry-based instruction, teacher-directed instruction, adaptive instruction, and feedback provided by PISA 2015. The sample consisted of 503,146 students from 17,678 schools in 69 countries/regions. Multilevel CFA and SEM were conducted for each scale in each country/region to evaluate school-level reliability (intraclass correlations 1 and 2), factorial validity, and predictive validity. In most countries/regions, school-level reliability was found to be adequate for the classroom management scale, but only low to moderate for the other scales. Examination of factorial and predictive validity indicated that the classroom management, emotional support, adaptive instruction, and teacher-directed instruction scales capture meaningful differences in teaching quality between schools. Meanwhile, the inquiry scale exhibited poor validity in almost all countries/regions. These findings suggest the possibility of using student ratings in PISA to investigate some aspects of school-level teaching quality in most countries/regions. (DIPF/Orig.)
DIPF-Departments: Bildungsqualität und Evaluation

Personality across the lifespan. Exploring measurement invariance of a short big five inventory […] Brandt, Naemi; Becker, Michael; Tetzner, Julia; Brunner, Martin; Kuhl, Poldi; Maaz, Kai Journal Article | In: European Journal of Psychological Assessment. | 2020 38867 Endnote: Author(s): Brandt, Naemi; Becker, Michael; Tetzner, Julia; Brunner, Martin; Kuhl, Poldi; Maaz, Kai
Title: Personality across the lifespan. Exploring measurement invariance of a short big five inventory from ages 11 to 84
In: European Journal of Psychological Assessment., 36 (2020) 1, S. 162-173
DOI: 10.1027/1015-5759/a000490
URL: https://econtent.hogrefe.com/doi/10.1027/1015-5759/a000490
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Altersgruppe; Vergleich; Validität; Reliabilität; Persönlichkeit; Messung; Indikator; Lebensalter; Erhebungsinstrument; Persönlichkeitsmerkmal; Kind; Grundschulalter; Erwachsener; Alter Mensch;
Abstract: Personality is a relevant predictor for important life outcomes across the entire lifespan. Although previous studies have suggested the comparability of the measurement of the Big Five personality traits across adulthood, the generalizability to childhood is largely unknown. The present study investigated the structure of the Big Five personality traits assessed with the Big Five Inventory-SOEP Version (BFI-S; SOEP = Socio-Economic Panel) across a broad age range spanning 11-84 years. We used two samples of N = 1,090 children (52% female, Mage = 11.87) and N = 18,789 adults (53% female, Mage = 51.09), estimating a multigroup CFA analysis across four age groups (late childhood: 11-14 years; early adulthood: 17-30 years; middle adulthood: 31-60 years; late adulthood: 61-84 years). Our results indicated the comparability of the personality trait metric in terms of general factor structure, loading patterns, and the majority of intercepts across all age groups. Therefore, the findings suggest both a reliable assessment of the Big Five personality traits with the BFI-S even in late childhood and a vastly comparable metric across age groups. (DIPF/Orig.)
DIPF-Departments: Struktur und Steuerung des Bildungswesens

The measurement of within-person affect variation Brose, Annette; Schmiedek, Florian; Gerstorf, Denis; Voelkle, Manuel C. Journal Article | In: Emotion | 2020 39994 Endnote: Author(s): Brose, Annette; Schmiedek, Florian; Gerstorf, Denis; Voelkle, Manuel C.
Title: The measurement of within-person affect variation
In: Emotion, 20 (2020) 4, S. 677-699
DOI: 10.1037/emo0000583
URN: urn:nbn:de:0111-dipfdocs-206192
URL: http://www.dipfdocs.de/volltexte/2020/20619/pdf/Emotion_2020_4_Brose_et_al_Measurement_of_within-person_affect_variation_A.pdf
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Aufsatz (keine besondere Kategorie)
Language: Englisch
Keywords: Emotion; Emotionaler Zustand; Veränderung; Messung; Erhebungsinstrument; Messverfahren; Reliabilität; Validität; Qualität; Konzeption; Vergleich; Empirische Forschung; Forschungsstand; Längsschnittuntersuchung; Literaturbericht
Abstract: The number of intensive longitudinal studies that investigate affective experiences at the within-person rather than the between-person level is rapidly increasing. This paradigmatic shift comes with new challenges, such as questions revolving around how to measure within-person affect variation or more fundamental questions about the reliability and validity of constructs at the within-person level. We provide a review of substantive research published in Emotion since 2005, which revealed that to date no consensus has been established on measurement instruments for assessing within-person affective experiences. Our review also showed that researchers who are interested in within-person affect variation sometimes rely on measurement instruments that were established at the between-person level, which we think should be reconsidered. Finally, reliability estimates of state variation have been developed but are not comprehensively reported in studies on within-person affect variation. The purpose of this article is therefore to alert the reader to these issues and to highlight relevant criteria for selecting items and measurement instruments when studying within-person affect variation in intensive longitudinal studies. We recommend establishing common standards for measuring within-person affect variation and drawing from a common pool of instruments, which would allow direct comparison of results across studies. (DIPF/Orig.)
DIPF-Departments: Bildung und Entwicklung

Entwicklung eines Fragebogens zur Erfassung von Überzeugungen Lehramtsstudierender zum Unterrichten […] Dignath, Charlotte; Meschede, Nicola; Kunter, Mareike; Hardy, Ilonca Journal Article | In: Psychologie in Erziehung und Unterricht | 2020 40511 Endnote: Author(s): Dignath, Charlotte; Meschede, Nicola; Kunter, Mareike; Hardy, Ilonca
Title: Entwicklung eines Fragebogens zur Erfassung von Überzeugungen Lehramtsstudierender zum Unterrichten in heterogenen Klassen: Befunde zur Kriteriumsvalidität und Veränderungssensitivität
In: Psychologie in Erziehung und Unterricht, 67 (2020) 3, S. 194-211
DOI: 10.2378/peu2020.art16d
URL: https://reinhardt-journals.de/index.php/peu/article/view/152742
Publication Type: 3a. Beiträge in begutachteten Zeitschriften; Beitrag in Sonderheft
Language: Deutsch
Keywords: Heterogenität; Lehramtsstudent; Schulklasse; Überzeugung; Einstellung <Psy>; Messverfahren; Fragebogen; Konzeption; Validität; Reliabilität; Berufliche Kompetenz; Behindertes Kind; Inklusion; Migrationshintergrund; Leistung; Leistungsdifferenzierung; Schulform; Modell
Abstract: Es liegen bisher nur wenige Instrumente vor, um Überzeugungen von Lehrpersonen zum Unterrichten in heterogenen Klassen im Sinne eines breiten Inklusionsbegriffs zu untersuchen. In drei Studien wurden die Struktur und die Validität von drei Skalen zu Überzeugungen zum Unterrichten in heterogenen Klassen von Lehramtsstudierenden untersucht. Die Skalen wurden in Anlehnung an das Instrument KIESEL von Bosse und Spörer (2014) zusätzlich zur Heterogenitätsdimension Behinderung (d.h. die gemeinsame Regelbeschulung von Schülerinnen und Schülern mit und ohne sonderpädagogischen Förderbedarf) um die Heterogenitätsdimensionen Kulturelle Heterogenität und Leistungsbezogene Heterogenität erweitert. Im Rahmen mehrerer Validierungsstudien wurde die Sensitivität des Instruments mit Blick auf erwartete Gruppenunterschiede sowie auf die Veränderung durch Instruktion überprüft. Die Skalen können weiterhin zwischen verschiedenen Konstrukten, z.B. Selbstwirksamkeit, diskriminieren.
Abstract (english): Only few instruments have been developed to assess such beliefs in teachers concerning various dimensions of heterogeneity following the wide definition of inclusion. In three studies we investigated the structure and the validity of three scales on heterogeneity beliefs. The scales were developed based on the instrument KIESEL by Bosse and Spörer (2014), adding two belief scales regarding cultural heterogeneity and performance-based heterogeneity to their scale on inclusive beliefs (i. e., teaching students with and without special educational needs within the same classroom). In several validity studies, we tested the sensitivity of the instrument regarding differences between groups as well as the assessment of beliefs development over time. Furthermore, the scales can discriminate between different constructs, for example, beliefs and self-efficacy.
DIPF-Departments: Bildungsqualität und Evaluation