教育认知诊断测验与认知模型一致性的评估

Evaluating the Consistency of Test Items Relative to the Cognitive Model for Educational Cognitive Diagnosis

构建正确的认知模型是成功进行认知诊断的关键之一,如果认知诊断测验不能完整准确地代表这个认知模型,这个测验的效度就存在问题。属性及其层级可以表示一个认知模型。在认知模型正确基础上,给出了一个计量公式以衡量认知诊断测验能够多大程度上代表认知模型;对于不止包含一个知识状态的等价类及其形成原因进行了分析,对Cui等人的属性层级相合性指标(HCI)提出修改建议,以更好地探查数据与专家给出的认知模型的一致性。

Attributes and their hierarchy may present a cognitive model. Building a cognitive model is one of key steps for cognitive diagnosis as it is directly related to the validity and usefulness of test results. It is very important to detect whether the test specification coincides with the cognitive model before administering the test. In this paper, an explicit index is given to measure the extent to which the cognitive model is represented by the test items of the diagnostic test. We call this index as the theoretical construct validity （TCV）. In terms of TCV, the test reported by Tatsuoka and her colleagues （1988） is reanalyzed, and the TCV of the test is only 9/24. That is, 24 knowledge states are obtained from the theoretic cognitive model but the test specification could distinguish only 9 knowledge states. Cui and her colleagues established a person fit index named hierarchy consistency index （HCI） to detect the fitness of an examinee＇s observed response pattern （ORP） to an expected response pattern （ERP）. HCI is not defined well when an examinee mastered one attribute only and there is only one item measuring the same attribute in the test, and the examinee responses correctly to the item. The original HCI could not compute for the number of comparisons being zero, hence the denominator being zero. In addition, the HCI includes the slipping only. Combining the slipping and guessing in the response to an item, we proposed a new index called modified HCI （MHCI）. MHCI could detect the consistency of the response data and the cognitive model that is obtained by experts.