Search results in the DIPF database of publications

Your query:

(Schlagwörter: "Testkonstruktion")

PISA 2018: Grundbildung im internationalen Vergleich Reiss, Kristina; Weis, Mirjam; Klieme, Eckhard; Köller, Olaf (Hrsg.) Compilation Book | Münster: Waxmann | 2019 39839 Endnote: Editor(s) Reiss, Kristina; Weis, Mirjam; Klieme, Eckhard; Köller, Olaf
Title: PISA 2018: Grundbildung im internationalen Vergleich
Published: Münster: Waxmann, 2019
DOI: 10.31244/9783830991007
URN: urn:nbn:de:0111-pedocs-183155
URL: http://nbn-resolving.org/urn:nbn:de:0111-pedocs-183155
Publication Type: Sammelbände; Werkeditionen (Herausgeberschaft); Sammelband (keine besondere Kategorie)
Language: Deutsch
Keywords: Leseförderung; Textverständnis; Schüler; Migrationshintergrund; Muttersprache; Außerunterrichtliche Aktivität; Soziale Herkunft; Einflussfaktor; Mathematische Kompetenz; Geschlechtsspezifischer Unterschied; Naturwissenschaftliche Kompetenz; Schulform; Methodologie; PISA <Programme for International Student Assessment>; Schülerleistung; Leistungsmessung; Lesekompetenz; Schülerleistungstest; Digitale Medien; Mediennutzung; Veränderung; Leseverhalten; Testaufgabe; Antwort; Lesen; Strategie; Selbstkonzept; Motivation; Sprachförderung; Technologiebasiertes Testen; Testkonstruktion; Konzeption; Internationaler Vergleich; OECD-LÄnder; Deutschland
Abstract: Der nationale Berichtsband stellt die Ergebnisse der Schülerinnen und Schüler in Deutschland aus der PISA-Studie 2018 vor und setzt sie in Relation zu den Ergebnissen in anderen Staaten. Der Schwerpunkt der Erhebungen und Auswertungen liegt dabei auf der Lesekompetenz. Die Rahmenkonzeption der Lesekompetenz wurde für die PISA-Studie 2018 im Vergleich zu den vorigen Runden einer Revision unterzogen und berücksichtigt nun die sich verändernde Lesepraxis, die mit der Nutzung digitaler Medien einhergeht. Es werden leistungsbezogene Variablen genauso wie motivationale Orientierungen, Einstellungen und Verhalten erfasst, die für die Lesekompetenz wichtig sein können. Daneben wird der Beitrag der sozialen Herkunft und des Zuwanderungshintergrunds für die Lesekompetenzen untersucht. Darüber hinaus werden die Kompetenzen der Jugendlichen in der Mathematik und den Naturwissenschaften vorgestellt und diskutiert. (DIPF/Verlag).
DIPF-Departments: Bildungsqualität und Evaluation

Do practical and academic preparation paths lead to differential commercial teacher "quality"? Holtsch, Doreen; Hartig, Johannes; Shavelson, Richard Journal Article | In: Vocations and Learning | 2019 38707 Endnote: Author(s): Holtsch, Doreen; Hartig, Johannes; Shavelson, Richard
Title: Do practical and academic preparation paths lead to differential commercial teacher "quality"?
In: Vocations and Learning, 12 (2019) , S. 23-46
DOI: 10.1007/s12186-018-9208-0
URL: http://link.springer.com/article/10.1007/s12186-018-9208-0
Publication Type: Zeitschriftenbeiträge; Zeitschriftenbeiträge
Language: Englisch
Keywords: Berufsschullehrer; Lehrerausbildung; Fachhochschule; Universität; Fachwissen; Fachkompetenz; Pädagogik; Wissen; Berufsschule; Unterricht; Qualität; Schulpraktikum; Test; Testkonstruktion; Bachelor-Studiengang; Master-Studiengang; Messverfahren; Validität; Reliabilität; Wirtschaftskunde; Zertifizierung; Vergleich; Multivariate Analyse; Schweiz
Abstract (english): The Swiss teacher education and training system offers a practically and academically oriented path for aspiring commercial vocational education and training (VET) teachers. Although teachers' content knowledge (CK) and pedagogical content knowledge (PCK) are considered crucial for teaching quality and students' achievement, little is known about Swiss VET teachers' Economics CK and PCK. Using assessments of teachers' economics CK and PCK as proxies of "quality" we found that teachers regardless of practical or academic preparation were similar in CK and PCK once in the teaching profession. This finding contradicts popular belief that academic preparation with its selectivity and education would produce higher quality teachers. (DIPF/Orig.)
DIPF-Departments: Bildungsqualität und Evaluation

Construct equivalence of PISA reading comprehension measured with paper‐based and computer‐based […] Kroehne, Ulf; Buerger, Sarah; Hahnel, Carolin; Goldhammer, Frank Journal Article | In: Educational Measurement | 2019 39814 Endnote: Author(s): Kroehne, Ulf; Buerger, Sarah; Hahnel, Carolin; Goldhammer, Frank
Title: Construct equivalence of PISA reading comprehension measured with paper‐based and computer‐based assessments
In: Educational Measurement, 38 (2019) 3, S. 97-111
DOI: 10.1111/emip.12280
URL: https://onlinelibrary.wiley.com/doi/abs/10.1111/emip.12280
Publication Type: Zeitschriftenbeiträge; Zeitschriftenbeiträge
Language: Englisch
Keywords: Einflussfaktor; Schülerleistung; Frage; Antwort; Interaktion; Unterschied; Vergleich; Item-Response-Theory; Deutschland; PISA <Programme for International Student Assessment>; Leseverstehen; Messverfahren; Testkonstruktion; Korrelation; Äquivalenz; Papier-Bleistift-Test; Computerunterstütztes Verfahren; Technologiebasiertes Testen; Leistungsmessung; Testverfahren; Testdurchführung
Abstract: For many years, reading comprehension in the Programme for International Student Assessment (PISA) was measured via paper‐based assessment (PBA). In the 2015 cycle, computer‐based assessment (CBA) was introduced, raising the question of whether central equivalence criteria required for a valid interpretation of the results are fulfilled. As an extension of the PISA 2012 main study in Germany, a random subsample of two intact PISA reading clusters, either computerized or paper‐based, was assessed using a random group design with an additional within‐subject variation. The results are in line with the hypothesis of construct equivalence. That is, the latent cross‐mode correlation of PISA reading comprehension was not significantly different from the expected correlation between the two clusters. Significant mode effects on item difficulties were observed for a small number of items only. Interindividual differences found in mode effects were negatively correlated with reading comprehension, but were not predicted by basic computer skills or gender. Further differences between modes were found with respect to the number of missing values.
Abstract (english): For many years, reading comprehension in the Programme for International Student Assessment (PISA) was measured via paper‐based assessment (PBA). In the 2015 cycle, computer‐based assessment (CBA) was introduced, raising the question of whether central equivalence criteria required for a valid interpretation of the results are fulfilled. As an extension of the PISA 2012 main study in Germany, a random subsample of two intact PISA reading clusters, either computerized or paper‐based, was assessed using a random group design with an additional within‐subject variation. The results are in line with the hypothesis of construct equivalence. That is, the latent cross‐mode correlation of PISA reading comprehension was not significantly different from the expected correlation between the two clusters. Significant mode effects on item difficulties were observed for a small number of items only. Interindividual differences found in mode effects were negatively correlated with reading comprehension, but were not predicted by basic computer skills or gender. Further differences between modes were found with respect to the number of missing values.
DIPF-Departments: Bildungsqualität und Evaluation

Sensitivity of test items to teaching quality Naumann, Alexander; Rieser, Svenja; Musow, Stephanie; Hochweber, Jan; Hartig, Johannes Journal Article | In: Learning and Instruction | 2019 38989 Endnote: Author(s): Naumann, Alexander; Rieser, Svenja; Musow, Stephanie; Hochweber, Jan; Hartig, Johannes
Title: Sensitivity of test items to teaching quality
In: Learning and Instruction, 60 (2019) , S. 41-53
DOI: 10.1016/j.learninstruc.2018.11.002
URL: https://www.sciencedirect.com/science/article/pii/S0959475217307065?via%3Dihub
Publication Type: Zeitschriftenbeiträge; Zeitschriftenbeiträge
Language: Englisch
Keywords: Leistungstest; Testkonstruktion; Unterricht; Qualität; Einflussfaktor; Testauswertung; Grundschule; Naturwissenschaftlicher Unterricht; Aktives Lernen; Entdeckendes Lernen; Unterrichtsmethode; Wirkung; Messverfahren; Testaufgabe; Problemlösen; Grundschüler; Dauer; Antwort; Schwierigkeit; Datenanalyse; Interpretation; Quasi-Experiment; Deutschland
Abstract: Instructional sensitivity is the psychometric capacity of tests or single items of capturing effects of classroom instruction. Yet, current item sensitivity measures' relationship to (a) actual instruction and (b) overall test sensitivity is rather unclear. The present study aims at closing these gaps by investigating test and item sensitivity to teaching quality, reanalyzing data from a quasi-experimental intervention study in primary school science education (1026 students, 53 classes, Mage = 8.79 years, SDage = 0.49, 50% female). We examine (a) the correlation of item sensitivity measures and the potential for cognitive activation in class and (b) consequences for test score interpretation when assembling tests from items varying in their degree of sensitivity to cognitive activation. Our study (a) provides validity evidence that item sensitivity measures may be related to actual classroom instruction and (b) points out that inferences on teaching drawn from test scores may vary due to test composition. (DIPF/Orig.)
DIPF-Departments: Bildungsqualität und Evaluation

Assessment of competences in sustainability management. Analyses to the construct dimensionality Seeber, Susan; Michaelis, Christian; Repp, Anton; Hartig, Johannes; Aichele, Christine; […] Journal Article | In: Zeitschrift für Pädagogische Psychologie | 2019 39562 Endnote: Author(s): Seeber, Susan; Michaelis, Christian; Repp, Anton; Hartig, Johannes; Aichele, Christine; Schumann, Matthias; Anke, Jan Moritz; Dierkes, Stefan; Siepelmeyer, David
Title: Assessment of competences in sustainability management. Analyses to the construct dimensionality
In: Zeitschrift für Pädagogische Psychologie, 33 (2019) 2, S. 148-158
DOI: 10.1024/1010-0652/a000240
URN: urn:nbn:de:0111-pedocs-237802
URL: https://nbn-resolving.org/urn:nbn:de:0111-pedocs-237802
Publication Type: Zeitschriftenbeiträge; Zeitschriftenbeiträge
Language: Englisch
Keywords: Nachhaltige Entwicklung; Kompetenz; Diagnostik; Messung; Modell; Student; Wirtschaftswissenschaft; Unternehmen; Simulation; Management; Curriculum; Messverfahren; Diagnostischer Test; Testkonstruktion; Erhebungsinstrument; Faktorenanalyse; Strukturgleichungsmodell
Abstract: Dieser Beitrag thematisiert die Messung von Kompetenzen für das Nachhaltigkeitsmanagement. Eine zentrale Annahme des zugrunde gelegten Kompetenzmodells ist, dass sich die Dimensionen nach der Wissensrepräsentation (deklaratives vs. schematisches und strategisches Wissen) und nach inhaltlichen Bereichen (Betriebswirtschaft, Nachhaltigkeit aus gesellschaftlicher Perspektive und Nachhaltigkeitsmanagement) unterscheiden. An der Studie nahmen 850 Studierende aus 16 deutschen Universitäten wirtschaftswissenschaftlicher Studiengänge teil. Die Analysen wurden auf der Grundlage von Strukturgleichungsmodellierungen durchgeführt. Die Ergebnisse zeigen einen erwartungskonformen Befund dahingehend, dass die über unterschiedliche Assessmentformate und inhaltliche Anforderungen adressierten Wissensarten zwei disjunkte Dimensionen darstellen. Die Modellanalysen zeigen eine bessere Passung zum mehrdimensionalen Modell, bei dem zwischen deklarativem Wissen im Bereich der Betriebswirtschaftslehre und der Nachhaltigkeit aus gesellschaftlicher Perspektive einerseits und dem Nachhaltigkeitsmanagement andererseits unterschieden wird. (DIPF/Orig.)
Abstract (english): The paper discusses an examination of the dimensions of a competence model for sustainability management. A central assumption is that the dimensions of the competence model differ according to knowledge representation (i. e., declarative vs. schematic and strategic knowledge) and content area (i. e., business administration and sustainability from a societal perspective, as well as sustainability management). Study participants included 850 students from 16 universities in Germany, and the analyses were conducted on the basis of structural equation modeling. The results reveal an expectation-compliant finding whereby the types of knowledge addressed by different assessment formats and content requirements can be presented in two disjunct dimensions. On the one hand, the model analyses indicate a better fit to the multidimensional model, which distinguishes between declarative knowledge in the field of business administration and sustainability from a social perspective, while on the other hand, the analyses suggest a better fit to sustainability management. (DIPF/Orig.)
DIPF-Departments: Bildungsqualität und Evaluation

Adaptive item selection under matroid constraints Bengs, Daniel; Brefeld, Ulf; Kröhne, Ulf Journal Article | In: Journal of Computerized Adaptive Testing | 2018 38642 Endnote: Author(s): Bengs, Daniel; Brefeld, Ulf; Kröhne, Ulf
Title: Adaptive item selection under matroid constraints
In: Journal of Computerized Adaptive Testing, 6 (2018) 2, S. 15-36
DOI: 10.7333/1808-0602015
URN: urn:nbn:de:0111-dipfdocs-166953
URL: http://www.dipfdocs.de/volltexte/2020/16695/pdf/JCAT_2018_2_Bengs_Brefeld_Kroehne_Adaptive_item_selection_under_matroid_constraints_A.pdf
Publication Type: Zeitschriftenbeiträge; Zeitschriftenbeiträge
Language: Englisch
Keywords: Adaptives Testen; Algorithmus; Computerunterstütztes Verfahren; Itembank; Messverfahren; Technologiebasiertes Testen; Testkonstruktion
Abstract (english): The shadow testing approach (STA; van der Linden & Reese, 1998) is considered the state of the art in constrained item selection for computerized adaptive tests. The present paper shows that certain types of constraints (e.g., bounds on categorical item attributes) induce a matroid on the item bank. This observation is used to devise item selection algorithms that are based on matroid optimization and lead to optimal tests, as the STA does. In particular, a single matroid constraint can be treated optimally by an efficient greedy algorithm that selects the most informative item preserving the integrity of the constraints. A simulation study shows that for applicable constraints, the optimal algorithms realize a decrease in standard error (SE) corresponding to a reduction in test length of up to 10% compared to the maximum priority index (Cheng & Chang, 2009) and up to 30% compared to Kingsbury and Zara's (1991) constrained computerized adaptive testing.
DIPF-Departments: Bildungsqualität und Evaluation

How to conceptualize, represent, and analyze log data from technology-based assessments? A generic […] Kroehne, Ulf; Goldhammer, Frank Journal Article | In: Behaviormetrika | 2018 38895 Endnote: Author(s): Kroehne, Ulf; Goldhammer, Frank
Title: How to conceptualize, represent, and analyze log data from technology-based assessments? A generic framework and an application to questionnaire items
In: Behaviormetrika, 45 (2018) 2, S. 527-563
DOI: 10.1007/s41237-018-0063-y
Publication Type: Zeitschriftenbeiträge; Zeitschriftenbeiträge
Language: Englisch
Keywords: Bildungsforschung; Empirische Forschung; Logdatei; Datenanalyse; Technologiebasiertes Testen; PISA <Programme for International Student Assessment>; Fragebogen; Konzeption; Testkonstruktion; Daten; Typologie; Hardware; Antwort; Verhalten; Dauer; Interaktion; Mensch-Maschine-Kommunikation; Indikator
Abstract: Log data from educational assessments attract more and more attention and large-scale assessment programs have started providing log data as scientific use files. Such data generated as a by-product of computer-assisted data collection has been known as paradata in survey research. In this paper, we integrate log data from educational assessments into a taxonomy of paradata. To provide a generic framework for the analysis of log data, finite state machines are suggested. Beyond its computational value, the specific benefit of using finite state machines is achieved by separating platform-specific log events from the definition of indicators by states. Specifically, states represent filtered log data given a theoretical process model, and therefore, encode the information of log files selectively. The approach is empirically illustrated using log data of the context questionnaires of the Programme for International Student Assessment (PISA). We extracted item-level response time components from questionnaire items that were administered as item batteries with multiple questions on one screen and related them to the item responses. Finally, the taxonomy and the finite state machine approach are discussed with respect to the definition of complete log data, the verification of log data and the reproducibility of log data analyses. (DIPF/Orig.)
DIPF-Departments: Bildungsqualität und Evaluation

Bedeutung und Berechnung der Prozentränge und T-Werte beim Erstellen von Testnormen. Anmerkungen […] Woerner, Wolfgang; Müller, Christian; Hasselhorn, Marcus Book Chapter | Aus: Trautwein, Ulrich; Hasselhorn, Marcus (Hrsg.): Begabungen und Talente | Göttingen: Hogrefe | 2017 37063 Endnote: Author(s): Woerner, Wolfgang; Müller, Christian; Hasselhorn, Marcus
Title: Bedeutung und Berechnung der Prozentränge und T-Werte beim Erstellen von Testnormen. Anmerkungen und Empfehlungen
In: Trautwein, Ulrich; Hasselhorn, Marcus (Hrsg.): Begabungen und Talente, Göttingen: Hogrefe, 2017 (Test und Trends. N. F., 15), S. 245-263
Publication Type: Beiträge in Sammelbänden; Sammelband (keine besondere Kategorie)
Language: Deutsch
Keywords: Pädagogische Diagnostik; Begabtenauslese; Leistungstest; Testkonstruktion; Testmethodik; Qualität; Testauswertung; SPSS; Stichprobe; Testverfahren; Testtheorie
Abstract: Die Nützlichkeit und der wissenschaftliche Wert eines pädagogisch-psychologischen Diagnoseinstruments setzen neben dem Nachweis von angemessen erfüllten Gütekriterien und einer ausreichend detaillierten Dokumentation der verwendeten Methodik aus voraus, dass geeignete Normwerte vorliegen. Angesichts der zentralen Rolle des Normierungsprozesses überrascht - auch bei aktuell verwendeten (Schul-)Leistungstests - eine bedauerliche Heterogenität der methodisch-rechnerischen Bestimmung von Normwerten mit bisweilen erheblichen Konsequenzen für individualdiagnostische Entscheidungen. Einschlägige Lehrbücher beschreiben zwar verschiedene alternative Methoden, ohne jedoch konkrete Empfehlungen zu deren Verwendung anzusprechen. Um dies nachzuholen, wird in diesem Beitrag ausführlich auf die Bedeutung und Berechnung von Prozentrang-Werten und darauf aufbauenden Standardnorm-Äquivalenten eingegangen. Insbesondere wird der Unterschied zwischen kumulativen Prozentwerten und dem hier nachdrücklich empfohlenen Intervallmitten-Prozentrang (IM-PR) erläutert. Um künftigen Testentwicklern die Berechnung von IM-PR-Werten zu erleichtern, werden im Appendix entsprechende SPSS-Mustersyntaxen zur Verfügung gestellt - in der Hoffnung, dass sich dadurch in Zukunft eine einheitliche Berechnungsgrundlage der Normwerte von psychodiagnostischen Verfahren erzielen lässt. (DIPF/Orig.)
DIPF-Departments: Bildung und Entwicklung

Experimental validation strategies for heterogeneous computer-based assessment items Engelhardt, Lena; Goldhammer, Frank; Naumann, Johannes; Frey, Andreas Journal Article | In: Computers in Human Behavior | 2017 37464 Endnote: Author(s): Engelhardt, Lena; Goldhammer, Frank; Naumann, Johannes; Frey, Andreas
Title: Experimental validation strategies for heterogeneous computer-based assessment items
In: Computers in Human Behavior, 76 (2017) , S. 683-692
DOI: 10.1016/j.chb.2017.02.020
URN: urn:nbn:de:0111-dipfdocs-176056
URL: http://www.dipfdocs.de/volltexte/2019/17605/pdf/Engelhardt_et_al._2017_ManuscriptAccepted_A.pdf
Publication Type: Zeitschriftenbeiträge; Zeitschriftenbeiträge
Language: Englisch
Keywords: Leistungstest; Leistungsmessung; Medienkompetenz; Computerunterstütztes Verfahren; Validität; Testaufgabe; Testkonstruktion; Anpassung; Strategie; Veränderung; Testmethodik; Testtheorie
Abstract (english): Computer-based assessments open up new possibilities to measure constructs in authentic settings. They are especially promising to measure 21st century skills, as for instance information and communication technologies (ICT) skills. Items tapping such constructs may be diverse regarding design principles and content and thus form a heterogeneous item set. Existing validation approaches, as the construct representation approach by Embretson (1983), however, require homogenous item sets in the sense that a particular task characteristic can be applied to all items. To apply this validation rational also for heterogeneous item sets, two experimental approaches are proposed based on the idea to create variants of items by systematically manipulating task characteristics. The change-approach investigates whether the manipulation affects construct-related demands and the eliminate-approach whether the test score represents the targeted skill dimension. Both approaches were applied within an empirical study (N = 983) using heterogeneous items from an ICT skills test. The results show how changes of ICT-specific task characteristics influenced item difficulty without changing the represented construct. Additionally, eliminating the intended skill dimension led to easier items and changed the construct partly. Overall, the suggested experimental approaches provide a useful validation tool for 21st century skills assessed by heterogeneous items. (DIPF/Orig.)
DIPF-Departments: Bildungsqualität und Evaluation

Modeling individual response time effects between and within experimental speed conditions. A GLMM […] Goldhammer, Frank; Steinwascher, Merle A.; Kroehne, Ulf; Naumann, Johannes Journal Article | In: British Journal of Mathematical and Statistical Psychology | 2017 37357 Endnote: Author(s): Goldhammer, Frank; Steinwascher, Merle A.; Kroehne, Ulf; Naumann, Johannes
Title: Modeling individual response time effects between and within experimental speed conditions. A GLMM approach for speeded tests
In: British Journal of Mathematical and Statistical Psychology, 70 (2017) 2, S. 238-256
DOI: 10.1111/bmsp.12099
Publication Type: Zeitschriftenbeiträge; Zeitschriftenbeiträge
Language: Englisch
Keywords: Test; Testkonstruktion; Antwort; Dauer; Unterschied; Messverfahren; Entscheidung; Einflussfaktor; Fehler; Modell; Vergleich
Abstract: Completing test items under multiple speed conditions avoids the performance measure being confounded with individual differences in the speed-accuracy compromise, and offers insights into the response process, that is, how response time relates to the probability of a correct response. This relation is traditionally represented by two conceptually different functions: the speed-accuracy trade-off function (SATF) across conditions relating the condition average response time to the condition average of accuracy, and the conditional accuracy function (CAF) within a condition describing accuracy conditional on response time. Using a generalized linear mixed modelling approach, we propose an item response modelling framework that is suitable for item response and response time data from experimental speed conditions. The proposed SATF and CAF model accommodates response time effects between conditions (i.e., person and item SATF slope) and within conditions (i.e., residual CAF slopes), captures person and item differences in these effects, and is suitable for measures with a strong speed component. Moreover, for a single condition a CAF model is proposed distinguishing person, item and residual CAF. The properties of the models are illustrated with an empirical example. (DIPF/Orig.)
DIPF-Departments: Bildungsqualität und Evaluation