招生选才过程中不同甄审项目的实质影响力

Real vs. Claimed Weights ofDifferent Assessment Proceduresin the College Admission orPersonnel Selection Process

学校机关进行招生选才时,甄审项目经常包含客观笔试及主观评审(如口试、书面数据审查等),并且会在招生简章中明白公布其不同的占分比例,但这些占分比例往往只是假象,实际上常观察到客观笔试的实质影响力较大,主观评审的决定力较低,原因包括评审委员之间没有共识、评审委员自限给分范围、口试委员所提问题无区辨力、笔试委员刻意拉开分数距离等。因此,将客观笔试及主观评审项目依公布的占分比例加总后,笔试的实质影响力往往高于公布的占分比例,反之,主观评审项目的实质影响力则缩减。本研究以计算机仿真方式实验前述现象。实验结果发现:主观评审项目对总成绩的实质影响力,的确大部分低于预设的占分比例,此一现象,随着评审评分范围的缩小及评审共识程度的降低而更趋严重,最严重的状况例如:评审评分范围限于70-90分且缺乏共识、存有矛盾时,纵然在总成绩当中设定为应占70%,但实际上最后真正的影响力约只有7%,其余都是笔试的影响力。反过来说,主观评审项目的实质影响力,只有当评分范围拥有0-100分的自由范围且评审间具有高度共识时,其实质影响力才可能达到设定值。此外,本研究也选取台湾北区一所大学九个学系在2009年的大学甄选分数为真实数据进行分析,发现'主观或客观评量'不是关键,例如一向被认为客观的'学科能力测验'(学测),因分数截断(进入第二阶段甄试的门槛)而影响力缩小,但各学系自订的指定科目笔试,影响力果然高于预期,甚至高于公告值的2-3倍以上。面试成绩影响力则有时高于有时低于公告值,但影响力皆相对大于学测。追究这些现象的原因,发现都和该项成绩分布范围有关,甄审项目的区辨力才是关键。本研究的结果可提供招生选才机构思考评分机制、计分程序及选才效果等问题,并提供外界判断各种甄选项目的实质影响力。

College Admission or Personnel Selection Process usually adopts both subjective and objective assessment procedures.The weight of each procedure might be claimed publicly,but it often did not reflect its real influence on the final grade.The current study simulated this phenomenon in the computer and employed data of real cases to investigate some factors behind the phenomenon.In the Monte Carlo study,the following variables were manipulated;1.ranges of subjective rating scores;2.levels of inter-rater consensus;3.weights of different assessment procedures.A total of 36 experimental conditions were formed,and 20 replicated samples were generated in each condition.Normally distributed random numbers between 0-100 were generated as the scores of the objective assessment procedure,and they were transformed according to the experimental conditions to form the scores of subjective assessment procedures.Results showed that real weights of subjective procedures were lower than claimed weights of them in most conditions.The phenomenon became acute when narrowing the rating range and reducing the degree of inter-rater consensus.Some analyses were also performed on admission scores of year 2009 from real cases,including 9 departments of one university at north Taiwan.Results indicated that the property of objectivity or subjectivity of assessment did not play an important role.The key factor was the discrimination power of assessment procedures.Those results have some implications for school admission or personnel screening process as well as for applicants in the process.