Assessment of the diagnostic performance and interobserver variability of endocytoscopy in Barrett's esophagus:A pilot ex-vivo study

AIM:To investigate a classification of endocytoscopy(ECS)images in Barrett’s esophagus(BE)and evaluate its diagnostic performance and interobserver variability.METHODS:ECS was applied to surveillance endoscopic mucosal resection(EMR)specimens of BE ex-vivo.The mucosal surface of specimen was stained with 1%methylene blue and surveyed with a catheter-type endocytoscope.We selected still images that were most representative of the endoscopically suspect lesion and matched with the final histopathological diagnosis to accomplish accurate correlation.The diagnostic performance and inter-observer variability of the new classification scheme were assessed in a blinded fashion by physicians with expertise in both BE and ECS and inexperienced physicians with no prior exposure to ECS.RESULTS:Three staff physicians and 22 gastroenterology fellows classified eight randomly assigned unknown still ECS pictures(two images per each classification)into one of four histopathologic categories as follows:(1)BEC1-squamous epithelium;(2)BEC2-BE without dysplasia;(3)BEC3-BE with dysplasia;and(4)BEC4-esophageal adenocarcinoma(EAC)in BE.Accuracy of diagnosis in staff physicians and clinical fellows were,respectively,100%and 99.4%for BEC1,95.8%and83.0%for BEC2,91.7%and 83.0%for BEC3,and95.8%and 98.3%for BEC4.Interobserver agreement of the faculty physicians and fellows in classifying each category were 0.932 and 0.897,respectively.CONCLUSION:This is the first study to investigate classification system of ECS in BE.This ex-vivo pilot study demonstrated acceptable diagnostic accuracy and excellent interobserver agreement.

Interobserver agreement for contrast-enhanced ultrasound of liver imaging reporting and data system:A systematic review and metaanalysis

BACKGROUND Hepatocellular carcinoma is the most common primary liver malignancy.From the results of previous studies,Liver Imaging Reporting and Data System(LIRADS)on contrast-enhanced ultrasound(CEUS)has shown satisfactory diagnostic value.However,a unified conclusion on the interobserver stability of this innovative ultrasound imaging has not been determined.The present metaanalysis examined the interobserver agreement of CEUS LI-RADS to provide some reference for subsequent related research.AIM To evaluate the interobserver agreement of LI-RADS on CEUS and analyze the sources of heterogeneity between studies.METHODS Relevant papers on the subject of interobserver agreement on CEUS LI-RADS published before March 1,2020 in China and other countries were analyzed.The studies were filtered,and the diagnostic criteria were evaluated.The selected references were analyzed using the“meta”and“metafor”packages of R software version 3.6.2.RESULTS Eight studies were ultimately included in the present analysis.Meta-analysis results revealed that the summary Kappa value of included studies was 0.76[95%confidence interval,0.67-0.83],which shows substantial agreement.Higgins I2 statistics also confirmed the substantial heterogeneity(I2=91.30%,95%confidence interval,85.3%-94.9%,P<0.01).Meta-regression identified the variables,including the method of patient enrollment,method of consistency testing,and patient race,which explained the substantial study heterogeneity.CONCLUSION CEUS LI-RADS demonstrated overall substantial interobserver agreement,but heterogeneous results between studies were also obvious.Further clinical investigations should consider a modified recommendation about the experimental design.

Interobserver Agreement for Classifying Infections in Patients with Decompensated Cirrhosis Based on Centers for Disease Control and Prevention Criteria

Background:Bacterial infections are common in patients with decompensated cirrhosis,largely owing to bacterial translocation and cirrhosis-associated immune dysfunction.This study aims to determine the reliability for classifying infections in patients with decompensated cirrhosis based on the Centers for Disease Control and Prevention(CDC)criteria.Methods:The patients with decompensated cirrhosis with suspicious infection in a registered prospective cohort of cirrhosis from May 1,2014 to February 25,2015 in the ward of First Affiliated Hospital of Zhejiang University were retrospectively identified.Agreement assessment was conducted focusing on site of infection,the possibility of infection,and pathogens of infection on both system level and specific diagnosis level.A subgroup analysis was performed based on with/without acute-on-chronic liver failure(ACLF).Results:A total of 402 infectious episodes among 351 patients were enrolled for consistency analysis.The overall agreement for site of infection was 94%(378/402)(k=0.90,95%CI 0.86–0.94)on system level and 86%(346/402)(k=0.84,95%CI 0.80–0.88)on specific diagnosis level.On possibility of infection,the overall agreement was 81%(306/378)(weighted k=0.71,95%CI 0.65–0.77),with 84%(224/267)(weighted k=0.75,95%CI 0.63–0.87)in patients with ACLF and 80%(70/88)(weighted k=0.68,95%CI 0.60–0.76)in patients without ACLF,respectively.On pathogens of infection,the overall agreement was 72%(60/83)(k=0.70,95%CI 0.60–0.80)among most frequent infections.Conclusion:The agreement for classifying infections in patients with decompensated cirrhosis based on CDC criteria is acceptable overall,suggesting that it can be a useful tool for clinical management in patients with decompensated cirrhosis with suspicious infections.

Elaboration and validation of Crohn's disease anoperineal lesions consensual definitions

AIM To establish consensual definitions of anoperineal lesions of Crohn's(APLOC) disease and assess interobserver agreement on their diagnosis between experts.METHODS A database of digitally recorded pictures of APLOC was examined by a coordinating group who selected two series of 20 pictures illustrating the various aspects of APLOC. A reading group comprised: eight experts from the Société Nationale Fran?aise de Colo Proctologie group of study and research in proctology and one academic dermatologist. All members of the coordinating and reading groups participated in dedicated meetings. The coordinating group initially conducted a literature review to analyse verbatim descriptions used to evaluate APLOC. The study included two phases: establishment of consensual definitions using a formal consensus method and later assessment of interobserver agreement on the diagnosis of APLOC using photos of APLOC, a standardised questionnaire and Fleiss' s kappa test or descriptive statistics.RESULTS Terms used in literature to evaluate visible APLOC did not include precise definitions or reference to definitions. Most of the expert reports on the first set of photos agreed with the main diagnosis but their verbatim reporting contained substantial variation. The definitions of ulceration(entity, depth, extension), anal skin tags(entity, inflammatory activity, ulcerated aspect), fistula(complexity, quality of drainage, inflammatory activity of external openings), perianal skin lesions(abscess, papules, edema, erythema) and anoperineal scars were validated. For fistulae, they decided to follow the American Gastroenterology Association's guidelines definitions. The diagnosis of ulceration(κ = 0.70), fistulae(κ = 0.75), inflammatory activity of external fistula openings(86.6% agreement), a b s c e s s e s( 8 4. 6 % a g r e e m e n t) a n d e r y t h e m a(100% agreement) achieved a substantial degree of interobserver reproducibility. CONCLUSION This study constructed consensual definitions of APLOC and their characteristics and showed that experts have a fair level of interobserver agreement when using most of the definitions.

Improving the accuracy and consistency of clinical target volume delineation for rectal cancer by an education program

BACKGROUND Accurate target volume delineation is the premise for the implementation of precise radiotherapy.Inadequate target volume delineation may diminish tumor control or increase toxicity.Although several clinical target volume(CTV)delineation guidelines for rectal cancer have been published in recent years,significant interobserver variation(IOV)in CTV delineation still exists among radiation oncologists.However,proper education may serve as a bridge that connects complex guidelines with clinical practice.AIM To examine whether an education program could improve the accuracy and consistency of preoperative radiotherapy CTV delineation for rectal cancer.METHODS The study consisted of a baseline target volume delineation,a 150-min education intervention,and a follow-up evaluation.A 42-year-old man diagnosed with stage IIIC(T3N2bM0)rectal adenocarcinoma was selected for target volume delineation.CTVs obtained before and after the program were compared.Dice similarity coefficient(DSC),inclusiveness index(IncI),conformal index(CI),and relative volume difference[ΔV(%)]were analyzed to quantitatively evaluate the disparities between the participants’delineation and the standard CTV.Maximum volume ratio(MVR)and coefficient of variation(CV)were calculated to assess the IOV.Qualitative analysis included four common controversies in CTV delineation concerning the upper boundary of the target volume,external iliac area,groin area,and ischiorectal fossa.RESULTS Of the 18 radiation oncologists from 10 provinces in China,13 completed two sets of CTVs.In quantitative analysis,the average CTV volume decreased from 809.82 cm3 to 705.21 cm3(P=0.001)after the education program.Regarding the indices for geometric comparison,the mean DSC,IncI,and CI increased significantly,whileΔV(%)decreased remarkably,indicating improved agreement between participants’delineation and the standard CTV.Moreover,an 11.80%reduction in MVR and 18.19%reduction in CV were noted,demonstrating a smaller IOV in delineation after the education program.Regarding qualitative analysis,the greatest variations in baseline were observed at the external iliac area and ischiorectal fossa;61.54%(8/13)and 53.85%(7/13)of the participants unnecessarily delineated the external iliac area and the ischiorectal fossa,respectively.However,the education program reduced these variations.CONCLUSION Wide variations in CTV delineation for rectal cancer are present among radiation oncologists in China's Mainland.A well-structured education program could improve delineation accuracy and reduce IOVs.

Automatic and Manual Proliferation Rate Estimation from Digital Pathology Images

Digital pathology is a major revolution in pathology and is changing the clinical routine for pathologists. We work on providing a computer aided diagnosis system that automatically and robustly provides the pathologist with a second opinion for many diagnosis tasks. However, inter-observer variability prevents thorough validation of any proposed technique for any specific problems. In this work, we study the variability and reliability of proliferation rate estimation from digital pathology images for breast cancer proliferation rate estimation. We also study the robustness of our recently proposed method CAD system for PRE estimation. Three statistical significance tests showed that our automated CAD system was as reliable as the expert pathologist in both brown and blue nuclei estimation on a dataset of 100 images.

Observer error in vegetation surveys:a review

Aims Vegetation sampling employing observers is prone to both inter-observer and intra-observer error.Three types of errors are common:(i)overlooking error(i.e.not observing species actually present),(ii)misidentification error(i.e.not correctly identifying species)and(iii)estimation error(i.e.not accurately estimating abundance).I conducted a literature review of 59 articles that provided quantitative estimates or statistical inferences regarding observer error in vegetation studies.Important FindingsAlmost all studies(92%)that tested for a statistically significant effect of observer error found at least one significant comparison.In surveys of species composition,mean pseudoturnover(the percentage of species overlooked by one observer but not another)was 10-30%.Species misidentification rates were on the order of 5-10%.The mean coefficient of variation(CV)among observers in surveys of vegetation cover was often several hundred%for species with low cover,although CVs of 25-50%were more representative of species with mean covers of>50%.A variety of metrics and indices(including commonly used diversity indices)and multivariate data analysis techniques(including ordinations and classifications)were found to be sensitive to observer error.Sources of error commonly include both characteristics of the vegetation(e.g.small size of populations,rarity,morphology,phenology)and attributes of the observers(e.g.mental fatigue,personal biases,differences in experience,physical stress).The use of multiple observers,additional training including active feedback approaches,and continual evaluation and calibration among observers are recommended as strategies to reduce observer error in vegetation surveys.