Logo: Deutsches Institut für Internationale Pädagogische Forschung

Research

InSe - Instructional Sensitivity of Test Items

The InSe-Project investigates the extent to which tests and test items are capable of capturing effects of instruction.

Project Description

Researchers as well as policy-makers regularly rely on student performance data to draw inferences on schools, teachers or teaching. Yet, valid inferences drawn from student test scores require that instruments are sensitive to the instruction that students have received in class. Accordingly, measures of test and test items’ instructional sensitivity may provide empirical support for validity claims about the inferences on instruction derived from student test scores.

The InSe-Project focuses on two questions:

(a) how can the instructional sensitivity of tests and items be measured empirically?

(b) how can information on instructional sensitivity be used in educational and psychological testing?

Objectives

The project builds on a longitudinal multilevel DIF-model (LML-DIFmodel) for evaluating instructional sensitivity (Naumann, Hochweber, & Hartig, 2014). The LML-DIF model integrates current approaches for measuring the instructional sensitivity of items by considering both differences between time points of measurement as well as differences between learning groups (i.e. classes).

Four major objectives are pursued: 

  1. further development and extension of the LML-DIF model
  2. examination of methods and conditions of parameter estimation
  3. the validation of the statistical indicators of instructional sensitivity
  4. development of criteria to classify the instructional sensitivity on the test and the item level

To achieve these objectives, two specific work packages are necessary. The first package includes the model development, simulation studies and secondary analyses, while the second work package relates to an empirical study in Swiss Schools.

Funding

The cooperation project "Instructional Sensitivity of Test Items” of the German Institute for International Educational Research (DIPF) and the University of Teacher Education St. Gallen (PHSG) is funded by the German Research Foundation (DFG) and the Swiss National Science Foundation (SNF).

Cooperations

  • Min Li, University of Washington, Seattle

Publications

  • Naumann, A., Hartig, J., & Hochweber, J. (2017). Absolute and relative measures of instructional sensitivity. Journal of Educational and Behavioral Statistics. Advance Online Publication. DOI: 10.3102//1076998617703649
  • Naumann, A., Hochweber, J., & Klieme, E. (2016). A psychometric framework for the evaluation of instructional sensitivity. Educational Assessment, 21(2), S. 89–101, DOI: 10.1080/10627197.2016.1167591
  • Naumann, A., Hochweber, J., & Hartig, J. (2014). Modeling instructional sensitivity using a longitudinal multilevel differential item functioning approach. Journal of Educational Measurement, 51, S. 381–399, DOI: 10.1111/jedm.12051

Project management

Project team

Project Details

State:
Current projects
Duration:
04/2015 – 04/2018
Funding:
External funding
Department: Educational Quality and Evaluation
Contact:
last modified Oct 16, 2017