SEPT口试方案设计

A Preliminary Study on SEPT Speaking Test Design

采用随机双面(two-facet crossed)p×i×r设计,通过对SEPT口试问答题的概化研究对其设计方案进行了探讨.结果表明,评分者之间具有较高的一致性.被试水平在不同试题上发挥极不一致,减少试题数量,概化系数急剧降低.优化SEPT口语测试方案设计的关键在于固定评分员水平为2对试题量进行调整.就现有试题质量而言,SEPT口试问答题部分采用问答题数为10,评分员为2的设计信度较好(Ep2=0.7)且经济可行,较为理想.作为一门专门用途英语(ESP),SEPT口试不能盲目地照搬TOEFL等考试的模式,应根据所涉及专业的特点,考虑各种测试情景进行测试方案的设计.笔者哌对样本同质性对概化系数的影响进行了实证探讨.

Generalizability Theory was applied to estimate generalizability coefficients of SEPT Speaking Test with a two-facet crossed design (p × i × r). The G-study and D-study results yielded a moderate but acceptable generalizability coefficient (Eρ 2 = 0.698) of the test and suggested that the assessment made by the raters on all subjects' performance were highly consistent. The subjects performed extremely different on different testing items. Adjustment of rater levels hardly affects the generalizability coefficient thereof, while generalizability coefficient would be greatly lowered if numbers of testing items reduced. It may be accounted for by the fact that individual testing item involves a different area of Ocean Shipping in which individual subject's qualification is different. Therefore, the optimization of the design of SEPT Speaking Test may be achieved by adjusting the sample capacity of the testing items.