Key Technologies in Mobile Visual Search and MPEG Standardization Activities

Visual search has been a long-standing problem in applications such as location recognition and product search. Much research has been done on image representation, matching, indexing, and retrieval. Key component technologies for visual search have been developed, and numerous real-world applications are emerging. To ensure application interoperability, the Moving Picture Experts Group （MPEG） has begun standardizing visuaJ search technologies and is developing the compact descriptors for visua） search （CDVS） standard. MPEG seeks to develop a collaborative platform for evaluating existing visual search technologies. Peking University has participated in this standardization since the 94th MPEG meeting, and significant progress has been made with the various proposals. A test model （TM） has been selected to determine the basic pipeline and key components of visual search. However, the first-version TM has high computational complexity and imperfect retrieval and matching. Core experiments have therefore been set up to improve TM. In this article, we summarize key technologies for visual search and report the progress of MPEG CDVS. We discuss Peking University＇ s efforts in CDVS and also discuss unresolved issues.

AB011.Reliance on central vs.peripheral vision for visual search in younger and older adults

Background:It has been suggested that older adults show a reduced attentional field compared to younger adults.This may be attributed to a poorer utilization of peripheral vision(i.e.,peripheral attentional allocation)and a higher reliance on central vision compared to younger adults.To test this,we examined the importance of central,peri-foveal and near periphery information in younger and older adults by comparing their visual search performance while their central vision was blocked,in the presence of different sized artificial central scotomas.We tested participants in two versions of visual search,pop-out and serial search,because they require a different use of central and peripheral attention.Pop-out search relies on processing of the entire visual scene(i.e.,global processing)whereas serial search requires processing of each feature serially(i.e.,local processing).Methods:Thirteen healthy younger(M=21.8,SD=1.5)and 15 older adults(M=69.1 years,SD=7.3)performed a pop-out and a serial version of a visual search task in the presence of different sized gaze-contingent artificial central scotomas(no scotoma,3°diameter,5°and 7°).Participants were asked to indicate as quickly as possible whether a target was present or not among distractors whose number varied(16,32 or 64 objects).Results:We found evidence for a greater decline in peripheral processing in older adults compared to younger in pop-out but not in serial search.For the pop-out condition with no scotoma,we found that the further the target in the periphery,the longer the search time,and that this increase was proportionally greater for older adults compared to younger adults.Further,increases in scotoma size were associated with a greater increase in reaction times for older adults compared to younger participants.For the serial condition,both groups showed similar increases in reaction times with target distance from center and scotoma size.We surmise that this may be due to task difficulty in serial search;central vision is necessary for both groups.Conclusions:In conclusion,these findings suggest that,in global processing,older adults distribute more resources towards central vision compared to younger adults.

AB060.Peripheral attentional allocation during visual search in the presence of an artificial scotoma in younger and older adults

Background:Age related macular degeneration(AMD)is one of the main causes of vision loss in older adults,generating,in most cases,a central scotoma that reduces central visual acuity(Noble&Chaudhary,2010).People affected by AMD have to rely on peripheral visual information and would highly benefit from efficiently allocating their attention to the periphery.Indeed,attention can improve peripheral spatial resolution(Carrasco,Ling&Read,2004)and can be allocated to a certain expanse of space outside of the central visual span,known as the attentional span.Attentional span has been shown to be decreased in people with AMD with less attention allocated to the periphery and more to the central visual field(Cheong et al.,2008),however it remains unknown whether aging is also a contributing factor.Methods:Fourteen healthy younger(mean age=21.8 years,SD=1.5)and 8 older adults(mean age=69.6 years,SD=7.3)performed a pop-out and a serial version of a visual search task,in the presence of different sized gaze-contingent invisible and visible artificial central scotomata(no scotoma,3°diameter,5°and 7°).Participants were asked to indicate as quickly as possible whether a target was present or not among distractors whose number varied(16,32 or 64 objects).We wished to determine whether the size of the scotoma,occluding different degrees of central vision,affected visual search differently for younger vs.older participants.Results:Both the younger and older participants showed higher reaction times(RTs)to find the target for the serial version(M=2,074 ms for younger adults,M=3,853 ms for older adults)compared to the pop-out version(M=866 ms,M=1,475 ms,P<0.001)and for more distractors(32 distractors compared to 16,and 64 compared to 32,P<0.01).Older adults showed longer RTs than younger adults for both versions of the task(P<0.01).We found a significant effect of scotoma size on older adults(3°scotoma M=3,276 ms;7°scotoma M=3,877 ms,P<0.05),however,accurate performance was higher with no scotoma(96%vs.92%,P<0.05)in the pop-out search task.This suggests that older participants privileged a fast decision at the expense of performance in those cases.For the younger adults,RTs were higher in the serial search task in the presence of a scotoma(M=2,074 ms)compared to the control condition(M=1,665 ms,P>0.05).Conclusions:These results suggest that older adults take longer to perform visual search compared to younger adults and tend to use peripheral visual less than younger adults;larger central scotomas disrupted their performance but not that of younger participants,who performed equally well with different central scotoma sizes.These findings suggest that aging is a contributing factor in the decrease of the peripheral attentional span.

Analysis on the Efficiency of Visual Search of Taekwondo Athletes with Trait Anxiety

In order to discuss the efficiency of visual search of the taekwondo athletes with different kinds of trait anxieties, this article has selected 30 taekwondo athletes with high trait anxieties and another 30 ones with low trait anxieties as the testees so as to conduct respective investigations on their visual search reaction time and accuracy of reaction. The results show that the reaction time of individuals with high trait anxieties is significantly longer than that of the individuals with low trait anxieties; the reaction time under threatening stimuli is significantly longer than that under no conditions of threatening stimuli; the reaction accuracy rate of visual search reaction of taekwondo athletes under threatening stimuli is significantly lower than that under no threatening stimuli.

Synchronization between frontal eye field and area V4 during free-gaze visual search

Information flow between the prefrontal and visual cortices is critical for visual behaviors such as visual search. To investigate its mechanisms, we simultaneously recorded spike and local field potential (LFP) signals in the frontal eye field (FEF) and area V4 while monkeys performed a free-gaze visual search task. During free-gaze search, spike-LFP coherence between FEF and V4 was enhanced in the theta rhythm (4–8 Hz) but suppressed in the alpha rhythm (8–13 Hz). Cross-frequency couplings during the Cue period before the search phase were related to monkey performance, with higher FEF theta-V4 gamma coupling and lower FEF alpha-V4 gamma coupling associated with faster search. Finally, feature-based attention during search enhanced spike-LFP coherence between FEF and V4 in the gamma and beta rhythms, whereas overt spatial attention reduced coherence at frequencies up to 30 Hz. These results suggest that oscillatory coupling may play an important role in mediating interactions between the prefrontal and visual cortices during visual search.

Effects of smartphone icon background shapes and figure/background area ratios on visual search performance and user preferences

Smartphones are becoming increasingly popular, users are provided with various interface styles with different designed icons. Icon, as an important competent of user interface, is regarded to be more efficient and pleasurable. However, compared with desktop computers, fewer design principles on smartphone icon were proposed. This paper investigated the effects of icon background shape and the figure/background area ratio on visual search performance and user preference. Icon figures combined with six different geometric background shapes and five different figure/ background area ratios were studied on three different screens in experiments with 40 subjects. The results of an analysis of variance （ANOVA） showed that these two inde- pendent variables （background shape and figure/background area ratio） significantly affected the visual search performance and user preference. On 3.5-in （1 in=0.025 4 m） and 4.0-in displays, unified backgroundwould be optimal, shapes such as square, circle and transitions between them （e.g., rounded square, squircle, etc.） are recommended because backgrounds in these shapes yield a better search time performance and subjective satisfaction for ease of use, search and visual preference. A 60% figure/background area ratio is the most appropriate for smartphone icon design on the 3.5-in screen, while a 50% area ratio could be a suggestion for both relatively optimized search performance and user preference on 4.0-in. In terms of the 4.7-in, icon figure is used di- rectly for its better performance and preference compared with icons with background.

Modulation of Neuronal Activity and Saccades at Theta Rhythm During Visual Search in Non-human Primates

Active exploratory behaviors have often been associated with theta oscillations in rodents,while theta oscillations during active exploration in non-human primates are still not well understood.We recorded neural activities in the frontal eye field(FEF)and V4 simultaneously when monkeys performed a free-gaze visual search task.Saccades were strongly phase-locked to theta oscillations of V4 and FEF local field potentials,and the phase-locking was dependent on saccade direction.The spiking probability of V4 and FEF units was significantly modulated by the theta phase in addition to the time-locked modulation associated with the evoked response.V4 and FEF units showed significantly stronger responses following saccades initiated at their preferred phases.Granger causality and ridge regression analysis showed modulatory effects of theta oscillations on saccade timing.Together,our study suggests phase-locking of saccades to the theta modulation of neural activity in visual and oculomotor cortical areas,in addition to the theta phase locking caused by saccade-triggered responses.

不同复杂度干扰影响下疏散标志搜索行为研究

为分析不同复杂度干扰因素对疏散标志搜索的影响,招募33名志愿者进行疏散标志搜索眼动试验。组织被试者完成了3×3因素设计的疏散标志视觉搜索试验,并对眼动数据进行了非参数检验。结果显示:不同复杂度各类型干扰物对疏散标志搜索任务均具有显著影响。在低复杂度水平中,物品类材料标志搜索用时最长;当复杂度增加至中等水平时,文字类、物品类材料标志搜索用时更长;至高复杂度时,图像类材料标志搜索用时更长。眼动数据分析显示,不同复杂度水平下,各类型材料眼动数据表现出显著差异。在低复杂度水平下,物品类材料注视点数量、扫视长度数据水平更高;在中复杂度水平下,文字类材料注视点数量、扫视长度明显增加;在高复杂度水平下,图像类材料注视点数量、扫视长度大幅增加,各注视点注视时长显著低于文字类、物品类材料。研究分析了不同复杂度水平各类型干扰因素对疏散标志搜索效率影响,分解了疏散人员视觉搜索眼动行为,可为提高疏散引导系统效能提供实证依据。

航空货物分类分级安检模式安全性分析

为保障航空安检环节安全高效,部分机场探索实施航空货物分类分级安检模式。研究以安全性为目标,综合考虑安检错误报警概率及货物安检处理时间,分析传统安检模式和分类分级安检模式在安全性方面的差异。使用错误放行概率作为衡量安检模式安全性的指标,利用目视搜索-决策模型分析安检员判图过程,为两种安检模式建立非线性规划模型。研究显示:与传统安检模式相比,分类分级安检模式中高风险货物的比例m在(0,0.45)内才能提高系统安全性,且m最优取值为0.12;与传统安检模式相比,分类分级安检模式在货量较大、对安检处理时间较为敏感或是对安检错误报警概率容忍度较高时更能显著提高系统安全性。

雾气下心电监护仪界面视觉搜索效率研究

目的研究护目镜产生雾气情况下心电监护仪界面的字符大小及颜色一致性对信息视觉搜索效率的影响。方法通过模拟护目镜在不同雾气程度下的场景,控制不同字符大小、颜色一致性,考察三者对医护人员视觉搜索效率的影响。实验中37位被试依次完成眼动实验,记录其行为数据和眼动数据,深入探讨三种变量因素的交互效应。结果使用心电监护仪时不同雾气程度、字符大小、颜色一致性三种因素的交互效应显著,其中行为数据的三种因素交互效应显著P<0.001,眼动数据的三种因素交互效应显著P<0.05。未产生雾气与低雾气场景下,大字符、颜色不一致视觉搜索效率最高;中雾气与高雾气场景下,中字符、颜色不一致的视觉搜索效率最高。结论三种因素在不同条件下,交互效应的视觉搜索效率不同,设计人员可根据护目镜产生的雾气程度,调整心电监护仪界面字符大小与颜色一致性,以提升医护人员的视觉搜索效率。

竞走裁判员判罚决策过程中的视觉搜索特征

目的比较不同经验水平和判罚难度条件下,竞走裁判员判罚决策的行为绩效和视觉搜索特征。方法采用眼动追踪技术和专家—新手研究范式,采集被试在竞走执裁判罚时的行为学指标和眼动指标数据。结果行为学指标:专家组正确率均显著高于新手组,反应时显著少于新手。在干扰条件下,新手组正确率显著降低、反应时显著升高,而专家组无显著性差异。眼动指标:专家组的注视时间、注视次数和眼跳次数均显著低于新手。在干扰条件下,新手组的注视时间、注视次数和眼跳次数均显著高于无干扰条件,专家组无显著性差异。兴趣区眼动指标显示,专家组对运动员膝和踝的注视时间显著高于新手组,注视次数显著低于新手组。可视化图像观察到,专家组注视轨迹清晰、注视热点集中;新手组注视轨迹复杂无规律,注视热点分散。结论高水平竞走裁判员具有较好的判罚决策能力和视觉信息加工能力,判罚过程中的视觉搜索更为简洁、高效,抗场外干扰能力强,体现出自上而下的视觉加工特征,表现出专家认知优势。

多层次矿床本体的构建及在知识图谱中的应用

将具有多元异构性和复杂语义的矿床数据转化为结构化数据,是目前矿产资源勘查大数据领域面临的关键问题。传统的机器学习方法无法精确描述实体概念、属性及其属性值的语义信息,导致多源异构数据的可解释性较差。因此,可解释性的知识图谱已成为当前研究的热点。然而,当前矿床领域本体构建研究仍相对匮乏,这阻碍了矿床知识图谱的研究。本文聚焦于矿床领域的概念、关系、属性描述,结合知识工程、叙词表、复用前人本体及专家知识,采用基于知识工程和基于顶层本体相结合的本体构建方法,使用本体开发工具Protégé构建了以时空矿床文本为基础的矿床领域本体库,实现了矿床知识概念、关系的系统化、规范化、形式化表达。然后运用Neo4j构建本体库知识图谱,并以庞西垌多金属矿床为案例,将矿床本体与矿床数据进行了知识图谱连接,展示了矿床本体作为知识图谱骨架的重要性。本文研究对下一步矿床知识图谱推理分析具有一定的指导意义。

色彩对比度及信息密度对老年用户视觉搜索的影响

通过动态导向显示屏呈现就诊信息的医院逐年增多,而老年人群由于视觉功能逐渐衰退,在医院检索及识别动态导向分诊信息时经常出现困难。为提升老年用户观察动态导向信息界面的绩效,对影响老年用户视觉搜索绩效明显的色彩对比度及信息密度因素展开研究。基于医院动态分诊显示器界面及E-prime实验平台,以色彩对比度及信息密度作为自变量,对20位60岁以上的老年用户展开视觉搜索实验,采集不同自变量水平下的视觉搜索时间、视觉搜索正确率及清晰度、舒适度评价。结果表明,色彩对比度为13.7∶1时,老年用户的视觉搜索绩效及评价最佳,当数值上升至17.4∶1及21∶1时,绩效及评价开始显著下降,但仍优于8.7∶1水平。在研究选用的字体及显示器下,老年用户在0.46及0.55(5~6行)信息密度水平的视觉搜索绩效及评价结果显著优于其他水平,且0.74(8行)信息密度水平为适用于老年用户的最大信息密度设计。

基于Visual C++6.0的机器人控制系统软件实现

三自由度插拔机器人控制系统软件根据开放式设计方法 ,采用VisualC ++语言编制控制程序。负责底层伺服驱动的函数库以标准C给出 ,可以直接被调用 ;同时 ,控制软件内嵌入了机器人研究所自行开发的机器人结构化编程语言———HRL语言 ,用以实现目标规划 ,它也是基于VisualC ++语言开发的。整个控制软件用以完成数据及状态显示、伺服驱动、机器人运动规划 ,插拔目标的搜索、定位、运动过程模拟显示等任务。

导视地图界面维度差异下的视觉搜索效率研究

图像的维度差异会对用户的信息获取及视觉交互效率产生一定程度的影响,以导视地图界面设计为例,建立三种不同维度的地图样本,利用眼动追踪实验收集用户搜索指定目标时的眼动数据进行分析,同时基于SUS系统可用性量表收集用户主观感受数据。结合定性与定量方法,比较分析了不同维度的地图呈现下用户的视觉搜索效率高低与规律差异,结果表明:在2D的呈现方式下,用户的综合搜索效率最高,而随着界面维度的升高,用户的搜索行为因受到透视因素干扰,综合搜索效率逐渐降低。

语义大小与刺激数量在非对称性视觉搜索中的作用

为了探讨语义与刺激数量在非对称视觉搜索中的作用,研究采用不同语义大小的名词汉字,并指导被试在6和12两种刺激数量中搜索目标刺激。结果发现:(1)当刺激数量为6时,语义较小目标刺激的搜索效率高于语义较大目标刺激;(2)当刺激数量为12时,语义较大目标刺激的搜索效率高于语义较小目标刺激;(3)当搜索的干扰刺激和目标刺激语义大小一致时,数量多且语义较大的干扰刺激搜索效率更低。这表明语义大小与刺激数量对非对称视觉搜索存在交互影响,支持具身认知理论,并扩充了特征整合理论和引导搜索模型。

Navi:基于自然语言交互的数据分析系统

随着大数据时代的到来,数据分析的作用日益显著.它能够从海量数据中发现有价值的信息,从而更有效地指导用户决策.然而,数据分析流程中存在三大挑战:分析流程高耦合、交互接口种类多和探索分析高耗时.为了应对上述挑战,提出了基于自然语言交互的数据分析系统Navi.该系统采用模块化的设计原则,抽象出主流数据分析流程的3个核心功能模块:数据查询、可视化生成和可视化探索模块,从而降低系统设计的耦合度.同时,Navi以自然语言作为统一的交互接口,并通过一个任务调度器实现了各功能模块的有效协同.此外,为了解决可视化探索中搜索空间指数级和用户意图不明确的问题,提出了一种基于蒙特卡洛树搜索的可视化自动探索方法,并设计了基于可视化领域知识的剪枝算法和复合奖励函数,提高了搜索效率和结果质量.最后,通过量化实验和用户实验验证了Navi的有效性.

高校专利专题数据库的建设研究

论述了专利专题库对大学学科发展的重要服务作用,探讨了专利专题库的一般建设思路。以首个能源电力行业专利专题库为例,详述了建库的背景,针对超大行业专利数据库的特点和能源电力行业的特色,提出技术分解的创新思路——以电能生产消费的过程为基本框架,同时突出在此过程中研究人员关注的重点技术内容,检索策略则综合检索专家的检索技能和各领域技术专家的专业知识联合确定,数据库软件的功能方面,除了基于技术分解的专利数据分类导航,还设计了丰富的专利检索和专利分析功能,专利分析结果以多维度可视化的方式展示。集专利导航、检索、分析功能为一体的专题数据库能满足科研用户的各种需求。

考虑视觉搜索特性的指挥控制系统KLSM

传统的击键水平模型(KLM)专注于键盘和鼠标的操作分析,适用于简单软件界面任务完成时间的快速定量评估,但将其迁移到包含复杂信息特征的指控系统软件界面评估时,却会产生较大误差。为此,基于KLM,结合用户视觉特性,面向指挥控制系统典型任务,提出基于用户视觉搜索处理时间的物理操作时间单元指标,并通过实验获取用户视觉搜索处理时间典型参数,建立更精确的指控系统任务操作分析模型—击键和视觉搜索模型(KLSM),即考虑视觉搜索特性因素的影响。选取5类典型指控任务,进行对比实验验证,KLSM对任务完成时间的预测评估更为准确。

智能化信息系统对飞行员视觉搜索特性的影响

为探究飞行员使用智能化信息系统的视觉搜索特性,设计以信息系统等级(交互系统和电子手册)为自变量、以视觉搜索特性(视觉搜索特征和绩效水平)为因变量的对比实验,记录20名航线飞行员在模拟机实验场景中所关注的各人机交互界面的眼动数据,通过统计分析、构建视觉搜索绩效评估模型,反映飞行员视觉搜索特征及绩效水平。结果表明:两组飞行员的注视总持续时间、注视总持续时间占比、注视次数等视觉搜索特征存在显著差异,且随着信息系统智能化等级的提高,飞行员在单发失效和襟翼卡阻故障场景中的视觉搜索绩效水平有效提高。由此,验证了信息系统智能化程度的提高在复杂场景下能对飞行员的工作绩效产生正效应,可为驾驶舱人机交互界面的设计、优化及迭代提供理论参考,保证飞行安全。