Development of a new 17-item Asthenopia Survey Questionnaire using Rasch analysis

AIM:To develop the 17-item Asthenopia Survey Questionnaire(ASQ)-17 by Rasch analysis,and to generate a predictiveness score.METHODS:Totally 739 participants were recruited and 680 were involved in the result analysis in this prospective,cross-sectional study.Three rounds of Rasch analysis were used to analyze the psychometric characteristics of items and options.RESULTS:Phase 1 assessed the original ASQ-19,adjusted the item scoring mode to a four-point Likert response rating scale and combined the 18th and 19th items into a new item.Phase 2 deleted the 11th item.Phases 3 and 4 assessed the new ASQ-17.All the evaluation indexes of ASQ-17 were acceptable.The Infit and Outfit MnSq values of items were 0.67-1.48,the variance explained by the principal component and the unexplained variance explained by the first contrast were 53.90%-59.40%and 1.50-1.80 in three dimensions.The curve peaks of scores in each dimension were separated and in the same order.The PSR and PSI values were 2.80 and 0.89,respectively.The mean scores of dimensions A(9.5±4.1 vs 3.5±3.2),B(7.3±3.3 vs 2.5±2.7),C(4.3±2.2 vs 1.4±2.0)and total(21.1±8.1 vs 7.4±7.0)in asthenopia participants were significantly higher than those without asthenopia(all P<0.001).The area under the curve in two groups was 0.899(P<0.001).Youden’s index was up to the maximum value of 0.784 when the cut-off value was 12.5.CONCLUSION:ASQ-17 has stronger option sorting and suitability than ASQ-19.It is an effective assessment tool for asthenopia with an optimal cut-off threshold value of 12.5,which is suitable for diagnosis and curative effect evaluation.

Item-Level Analysis of the Revised Occupational Therapy Fieldwork Performance Evaluation Applied in Practice

Competency-based assessments for healthcare professionals are critical for safe and effective client outcomes. Rehabilitation clinical skill competency assessments must be validated and revised to produce safe and skilled practitioners. The revised American Occupational Therapy Association (AOTA) Fieldwork Performance Evaluation (FWPE) instrument measures occupational therapy student performance to determine readiness for practice. The assessment includes thirty-seven competencies that address both profession specific clinical skills and general professional behavior skills. The objective of this study was to use Rasch methods to explore the use of the revised FWPE in actual fieldwork practice and to determine the instrument’s psychometric properties when separating the item components into two distinct subdomains: General Health Professions Competences and Occupational Therapy-Specific Competencies. Internal construct validity and test reliability were analyzed using data from 149 occupational therapy students after completing their initial Level II A fieldwork clinical internship. This study examined the item difficulty hierarchy, item fit, person-fit to model, person separation index, person separation reliability coefficient, strata, ceiling and floor effect, and unidimensionality of the FWPE instrument as a whole and as two separate domains. With the exception of not meeting the criteria for unidimensionality, the full FWPE instrument and the Occupational Therapy-Specific Competencies subdomain showed acceptable item-level psychometrics for reliability and precision. While the General Health Professions Competencies subdomain showed good item-level psychometrics, it was below the criterion for reliability and only separated the sample into two strata. Results support the validity, reliability, and clinical use of the revised FWPE full instrument and the Occupational Therapy-Specific Competencies subdomain to measure entry-level clinical skill competencies in practice.

Towards a profile-based writing scale for a local post-admission ESL writing assessment

Post-admission language tests tend to have a restricted range of proficiency levels among test-takers due to considerations made during the admission selection process.Although range restriction can present challenges for proficiency-focused assessment,it can also bring opportunities to zoom in on fine-grained performance profiles of test-takers.This study reports on the validation of a profile-based rating scale for an ESL writing placement test in a US university.The profile-based rating scale was created by employing a three-staged,hybrid scale development approach,to provide not only accurate placement decisions but also fine-grained diagnostic information regarding ESL students’writing performance profiles.The scale strikes a balance between argument development and lexico-grammar,to better account for the range of writing performances among test-takers.To gather validity evidence for the profile-based rating scale,this study employs a sequential,mixed-methods approach to examine the quality of test-taker performances across profiles and rater perceptions on the scale.Nine certified raters were recruited to conduct independent evaluations of lexicogrammar and argumentation on a sample of 150 test-taker performances.These evaluations were subjected to many-facet Rasch measurement analysis to examine the differences across writing performance profiles included in the rating scale.Next,semi-structured,follow-up interviews were conducted with the raters,to complement the quantitative findings on the usability and effectiveness of the scale.The findings provide supportive evidence for the validity of the profile-based rating scale.I argue that by focusing on performance profiles,post-admission language tests can strengthen the alignment across curriculum,instruction,and assessment in ESL writing programs.

Reliability of Chinese web-based ocular surface disease index questionnaire in dry eye patients:a randomized,crossover study

AIM:To assess the reliability of web-based version of ocular surface disease index in Chinese(C-OSDI)on clinically diagnosed dry eye disease(DE)patients.METHODS:A total of 254 Chinese participants(51%male,129/254;mean age:27.90±9.06 y)with DED completed paper-and web-based versions of C-OSDI questionnaires in a randomized crossover design.Ophthalmology examination and DED diagnosis were performed prior to the participants being invited to join the study.Participants were randomly designated to either group A(paper-based first and webbased second)or group B(web-based first and paper-based second).Final data analysis included participants that had successfully completed both versions of the C-OSDI.Demographic characteristics,test-retest reliability,and agreement of individual items,subscales,and total score were evaluated with intraclass correlation coefficients(ICC),Spearman rank correlation,Wilcoxon test and Rasch analysis.RESULTS:Reliability indexes were adequate,Pearson correlation was greater than 0.8 and ICCs range was 0.827 to 0.982;total C-OSDI score was not statistically different between the two versions.The values of mean-squares fit statistics were very low compared to 1,indicating that the responses to the items by the model had a high degree of predictability.While comparing the favorability 72%(182/254)of the participants preferred web-based assessment.CONCLUSION:Web-based C-OSDI is reliable in assessing DED and correlation with the paper-based version is significant in all subscales and overall total score.Webbased C-OSDI can be administered to assess individuals with DED as participants predominantly favored online assessment.

Development and Validation of a Test Measuring Language Analytic Ability for Chinese FL learners:A Rasch-Based Pilot Study

This study reports the development, piloting and initial validation of a test measuring language analytic ability - one foreign language aptitude component for Chinese learners of foreign languages （FL）. A test with 50 items was constructed and administered to 53 third-year English majors. Rasch analyses showed that the subtest of inductive language learning ability was too easy. After removing misfitting items, the reduced grammatical sensitivity subtest showed satisfactory psychometric properties. The Rasch measures of the students＇ grammatical sensitivity were also found to be correlated significantly with their TEM-4 scores and their English reading grades, thus providing further evidence for the validity of the this subtest.

Catquest-9SF functioning over a decade–a study from the Swedish National Cataract Register

Background:The Swedish National Cataract Register(NCR)collects data on cataract surgery outcomes during March,including patient-reported outcomes using the Catquest-9SF questionnaire for over 11 years.Previous studies from NCR have shown that the preoperative visual acuity has improved over time.The main purpose of this study was to evaluate the Catquest-9SF Rasch scoring performance in this changing environment.A second purpose was to describe clinical data over the same period for those who completed the questionnaire.Methods:The performance of the Catquest-9SF was analysed by a separate Rasch analysis for each year,resulting in a preoperative and postoperative score for each participating patient in the annual cohorts.The clinical data and questionnaire scoring were analysed for each year in the period 2008-2018 inclusive.Results:Data were available for 42,023 eyes for 11 annual cohorts(2008-2018).The psychometric properties of the questionnaire were stable during the study period.Person separation(precision)for the whole period was 2.58 and varied between 2.45 and 2.72.The person reliability was 0.87 and varied between 0.86 and 0.88.The targeting of question difficulty to person ability became less accurate over time meaning that the item activities became easier to carry out without difficulty.The average targeting for the whole period was−2.06 and changed from−1.92 in 2008 to−2.31 in 2018.The person score improved both before surgery and after surgery,indicating that patients are undergoing surgery at a more able level and getting better outcomes.The average improvement by surgery decreased from 3.41 logits in 2008 to 3.21 logits in 2018(p=0.003).Over time,patient age decreased from 75 to 74 years(p<0.001)and the proportion of women decreased from 63.9 to 57.9%(p<0.001).The mean preoperative visual acuity in both the operated eye and the better eye improved over time(0.47 to 0.40 logMAR,p<0.001 and 0.22 to 0.19 logMAR,p<0.001,respectively),as did the mean postoperative visual acuity in the operated eye(0.14 to 0.09 logMAR,p<0.001).Conclusions:The Catquest-9SF retained stable psychometric properties over this 11-year period although more recent cohorts included slightly younger patients with somewhat better vision.

The Italian Catquest-9SF cataract questionnaire: translation, validation and application

Background:To validate the Catquest-9SF questionnaire in Italian,assess the change in visual disability with cataract surgery and determine the correlation between pre-operative Catquest-9SF scores and Lens Opacities Classification System(LOCS)III cataract grading.Methods:Prospective,questionnaire validation study.The Catquest-9SF questionnaire was forward and back translated and completed by 209 Italian patients before and three months following cataract surgery.Rasch analysis was used to assess its psychometric properties.Results:The Italian Catquest-9SF demonstrated ordered response categories,unidimensionality(item fit statistics range:0.73–1.34),adequate person separation(2.04),and no differential item functioning.Mistargeting was evident with a mean difference in item difficulty and person ability of 2.04 logits but improved with inclusion of pre-operative data only.There was a statistically significant(Friedman tests,p<0.001)median improvement in visual disability of 1.92,3.57,1.44 and 2.94 logits in patients undergoing first eye surgery with and without ocular comorbidity,and second eye surgery with and without ocular comorbidity respectively.There was no statistically significant difference in the improvements among the four groups(Kruskal-Wallis H test,X^(2)(3)=5.445,p=0.142).There was no correlation between Catquest-9SF scores and nuclear opalescence(r_(s)=0.049,p=0.478),nuclear colour(r_(s)=0.008,p=0.909),cortical(r_(s)=0.066,p=0.341),and posterior subcapsular components(r_(s)=0.048,p=0.494).Conclusions:The Italian Catquest-9SF demonstrated good psychometric properties and is suitable for use in Italian speaking patients.There were similar improvements in visual disability in patients undergoing first or second eye surgery,with or without ocular comorbidity.There was no correlation between pre-operative Catquest-9SF scores and LOCS III cataract grading.