Watermarking through image geometry change tracking

Most of the digital image watermarking techniques are susceptible to geometric attacks such as cropping,rotation and scaling.These attacks are the easiest yet most successful in rendering the survival of watermark difficult.Such geometric operations alter the pixel orientation in the cover thereby rendering the watermark difficult to locate and extract.However,if the alterations produced by the geometric attacks such as scaling,cropping and rotation can be modeled in terms of the change in the image geometry,it is possible to relocate the watermark even after the original cover has suffered an attack.This paper contributes to the state of the art by proposing an image watermarking technique that attempts to model the attacks like cropping,scaling and rotation in terms of the image geometry.The proposed scheme is acceptably resistant to common geometric attacks and common image processing attacks.The watermark embedding is also done efficiently to offer resistance to image processing attacks.The watermark detection procedure is blind and key based,also not requiring the original cover work for watermark extraction.Efforts have been given to ensure that the proposed scheme conforms to robustness against attacks and exhibits high visual fidelity of the watermarked cover.

Visualization Laboratory at University of Notre Dame

This article introduces the Visualization Laboratory at the Department of Computer Science&Engineering,the University of Notre Dame,including the lab’s overview,current research directions,facilities,and international collaborations.

A uncertainty visual analytics approach for bus travel time

Bus travel time is uncertain due to the dynamic change in the environment.Passenger analyzing bus travel time uncertainty has significant implications for understanding bus running errors and reducing travel risks.To quantify the uncertainty of the bus travel time prediction model,a visual analysis method about the bus travel time uncertainty is proposed in this paper,which can intuitively obtain uncertain information of bus travel time through visual graphs.Firstly,a Bayesian encoder–decoder deep neural network(BEDDNN)model is proposed to predict the bus travel time.The BEDDNN model outputs results with distributional properties to calculate the prediction model uncertainty degree and provide the estimation of the bus travel time uncertainty.Second,an interactive uncertainty visualization system is developed to analyze the time uncertainty associated with bus stations and lines.The prediction model and the visualization model are organically combined to better demonstrate the prediction results and uncertainties.Finally,the model evaluation results based on actual bus data illustrate the effectiveness of the model.The results of the case study and user evaluation show that the visualization system in this paper has a positive impact on the effectiveness of conveying uncertain information and on user perception and decision making.

JobViz:Skill-driven visual exploration of job advertisements

Online job advertisements on various job portals or websites have become the most popular way for people to find potential career opportunities nowadays.However,the majority of these job sites are limited to offering fundamental filters such as job titles,keywords,and compensation ranges.This often poses a challenge for job seekers in efficiently identifying relevant job advertisements that align with their unique skill sets amidst a vast sea of listings.Thus,we propose well-coordinated visualizations to provide job seekers with three levels of details of job information:a skill-job overview visualizes skill sets,employment posts as well as relationships between them with a hierarchical visualization design;a post exploration view leverages an augmented radar-chart glyph to represent job posts and further facilitates users’swift comprehension of the pertinent skills necessitated by respective positions;a post detail view lists the specifics of selected job posts for profound analysis and comparison.By using a real-world recruitment advertisement dataset collected from 51Job,one of the largest job websites in China,we conducted two case studies and user interviews to evaluate JobViz.The results demonstrated the usefulness and effectiveness of our approach.

Geo-Coordinated Parallel Coordinates (GCPC): Field trial studies of environmental data analysis

The large number of environmental problems faced by society in recent years has driven researchers to collect and study massive amounts of data in order to understand the complex relations that exist between people and the environment in which we live.Such datasets are often high dimensional and heterogeneous in nature,with complex geospatial relations.Analysing such data can be challenging,especially when there is a need to maintain spatial awareness as the non-spatial attributes are studied.Geo-Coordinated Parallel Coordinates(GCPC)is a geovisual analytics approach designed to support exploration and analysis within complex geospatial environmental data.Parallel coordinates are tightly coupled with a geospatial representation and an investigative scatterplot,all of which can be used to show,reorganize,filter,and highlight the high dimensional,heterogeneous,and geospatial aspects of the data.Two sets of field trials were conducted with expert data analysts to validate the real-world benefits of the approach for studying environmental data.The results of these evaluations were positive,providing real-world evidence and new insights regarding the value of using GCPC to explore among environmental datasets when there is a need to remain aware of the geospatial aspects of the data as the non-spatial elements are studied.

Visualization and visual analysis of vessel trajectory data: A survey

Maritime transports play a critical role in international trade and commerce.Massive vessels sailing around the world continuously generate vessel trajectory data that contain rich spatial–temporal patterns of vessel navigations.Analyzing and understanding these patterns are valuable for maritime traffic surveillance and management.As essential techniques in complex data analysis and understanding,visualization and visual analysis have been widely used in vessel trajectory data analysis.This paper presents a literature review on the visualization and visual analysis of vessel trajectory data.First,we introduce commonly used vessel trajectory data sets and summarize main operations in vessel trajectory data preprocessing.Then,we provide a taxonomy of visualization and visual analysis of vessel trajectory data based on existing approaches and introduce representative works in details.Finally,we expound on the prospects of the remaining challenges and directions for future research.

Evaluation on interactive visualization data with scatterplots

Scatterplots and scatterplot matrix methods have been popularly used for showing statistical graphics and for exposing patterns in multivariate data.A recent technique,called Linkable Scatterplots,provides an interesting idea for interactive visual exploration which provides a set of necessary plot panels on demand together with interaction,linking and brushing.This article presents a controlled study with a mixed-model design to evaluate the effectiveness and user experience on the visual exploration when using a Sequential-Scatterplots who a single plot is shown at a time,Multiple-Scatterplots who number of plots can be specified and shown,and Simultaneous-Scatterplots who all plots are shown as a scatterplot matrix.Results from the study demonstrated higher accuracy using the Multiple-Scatterplots visualization,particularly in comparison with the Simultaneous-Scatterplots.While the time taken to complete tasks was longer in the Multiple-Scatterplots technique,compared with the simpler Sequential-Scatterplots,Multiple-Scatterplots is inherently more accurate.Moreover,the Multiple-Scatterplots technique is the most highly preferred and positively experienced technique in this study.Overall,results support the strength of Multiple-Scatterplots and highlight its potential as an effective data visualization technique for exploring multivariate data.

Towards the automation of book typesetting

This paper proposes a generative approach for the automatic typesetting of books in desktop publishing.The presented system consists in a computer script that operates inside a widely used design software tool and implements a generative process based on several typographic rules,styles and principles which have been identified in the literature.The performance of the proposed system is tested through an experiment which included the evaluation of its outputs with people.The results reveal the ability of the system to consistently create varied book designs from the same input content as well as visually coherent book designs with different contents while complying with fundamental typographic principles.

The use of facial expressions in measuring students’interaction with distance learning environments during the COVID-19 crisis

Digital learning is becoming increasingly important in the crisis COVID-19 and is widespread in most countries.The proliferation of smart devices and 5G telecommunications systems are contributing to the development of digital learning systems as an alternative to traditional learning systems.Digital learning includes blended learning,online learning,and personalized learning which mainly depends on the use of new technologies and strategies,so digital learning is widely developed to improve education and combat emerging disasters such as COVID-19 diseases.Despite the tremendous benefits of digital learning,there are many obstacles related to the lack of digitized curriculum and collaboration between teachers and students.Therefore,many attempts have been made to improve the learning outcomes through the following strategies:collaboration,teacher convenience,personalized learning,cost and time savings through professional development,and modeling.In this study,facial expressions and heart rates are used to measure the effectiveness of digital learning systems and the level of learners’engagement in learning environments.The results showed that the proposed approach outperformed the known related works in terms of learning effectiveness.The results of this research can be used to develop a digital learning environment.

IVMS:An immersive virtual meteorological sandbox based on WYSIWYG

A novel approach to visually represent meteorological data has emerged with the maturation of Immersive Analytics(IA).We have proposed an immersive meteorological virtual sandbox as a solution to the limitations of 2D analysis in expressing and perceiving data.This innovative visual method enables users to interact directly with data through non-contact aerial gestures(NCAG).Referring to the“What you see is what you get”concept in scientific visualization,we proposed a novel approach for the visual exploration of meteorological data that aims to immerse users in the analysis process.We hope this approach can inspire immersive visualization techniques for other types of geographic data as well.Finally,we conducted a user questionnaire to evaluate our system and work.The evaluation results demonstrate that our system effectively reduces cognitive burden,alleviates mental workload,and enhances users’retention of analysis findings.

G6:A web-based library for graph visualization

Authoring graph visualization poses great challenges to developers due to its high requirements on both domain knowledge and development skills.Although existing libraries and tools reduce the difficulty of generating graph visualization,there are still many challenges.We work closely with developers and formulate several design goals,then design and implement G6,a web-based library for graph visualization.It combines template-based configuration for high usability and flexible customization for high expressiveness.To enhance development efficiency,G6 proposes a range of optimizations,including state management and interaction modes.We demonstrate its capabilities through an extensive gallery,a quantitative performance evaluation,and an expert interview.G6 was first released in 2017 and has been iterated for 317 versions.It has served as a web-based library for thousands of applications and received 8312 stars on GitHub.

TBSSvis:Visual analytics for Temporal Blind Source Separation

Temporal Blind Source Separation(TBSS)is used to obtain the true underlying processes from noisy temporal multivariate data,such as electrocardiograms.TBSS has similarities to Principal Component Analysis(PCA)as it separates the input data into univariate components and is applicable to suitable datasets from various domains,such as medicine,finance,or civil engineering.Despite TBSS’s broad applicability,the involved tasks are not well supported in current tools,which offer only text-based interactions and single static images.Analysts are limited in analyzing and comparing obtained results,which consist of diverse data such as matrices and sets of time series.Additionally,parameter settings have a big impact on separation performance,but as a consequence of improper tooling,analysts currently do not consider the whole parameter space.We propose to solve these problems by applying visual analytics(VA)principles.Our primary contribution is a design study for TBSS,which so far has not been explored by the visualization community.We developed a task abstraction and visualization design in a user-centered design process.Task-specific assembling of well-established visualization techniques and algorithms to gain insights in the TBSS processes is our secondary contribution.We present TBSSvis,an interactive web-based VA prototype,which we evaluated extensively in two interviews with five TBSS experts.Feedback and observations from these interviews show that TBSSvis supports the actual workflow and combination of interactive visualizations that facilitate the tasks involved in analyzing TBSS results.

CARVING-DETC: A network scaling and NMS ensemble for Balinese carving motif detection method

Balinese carvings are cultural objects that adorn sacred buildings. The carvings consist of several motifs,each representing the values adopted by the Balinese people. Detection of Balinese carving motifs ischallenging due to the unavailability of a Balinese carving dataset for detection tasks, high variance,and tiny-size carving motifs. This research aims to improve carving motif detection performance onchallenging Balinese carving motifs detection task through a modification of YOLOv5 to support adigital carving conservation system. We proposed CARVING-DETC, a deep learning-based Balinesecarving detection method consisting of three steps. First, the data generation step performs dataaugmentation and annotation on Balinese carving images. Second, we proposed a network scalingstrategy on the YOLOv5 model and performed non-maximum suppression (NMS) on the modelensemble to generate the most optimal predictions. The ensemble model utilizes NMS to producehigher performance by optimizing the detection results based on the highest confidence score andsuppressing other overlap predictions with a lower confidence score. Third, performance evaluation onscaled-YOLOv5 versions and NMS ensemble models. The research findings are beneficial in conservingthe cultural heritage and as a reference for other researchers. In addition, this study proposed a novelBalinese carving dataset through data collection, augmentation, and annotation. To our knowledge,it is the first Balinese carving dataset for the object detection task. Based on experimental results,CARVING-DETC achieved a detection performance of 98%, which outperforms the baseline model.

Message from the Editors-in-Chief

It is our great pleasure to announce the launch of a new journal,Visual Informatics,which will publish original articles and survey papers on theories and algorithms for visual information modeling,synthesis and processing.Visual information is the major channel for human and ma-chines to perceive the surrounding world,whereas images,videos,graphics and animations are the most popular visual media.Be-sides real-time generation and transmission of visual information,efficient acquisition and perception of the features and semantics behind visual information have long been the grand challenges of multiple areas in computer science.

Examining the use of narrative constructs in data videos

Data videos are a highly impactful method of communication and are becoming a prevalent medium for communicating information.While the majority of current research focuses on the cinematic aspects of data videos,very little is known about the narrative methodologies involved.This paper presents our insights derived from an initial exploration of this area.We present a taxonomy based on the analysis of 70 existing data videos examining their narrative and visual approaches.We propose that our taxonomy can be used to explain the characteristics or design of data videos.Applying this taxonomy,we present our observations,including the trend of popular technologies applied in current data videos,the under-utilization of promising methods,and highlight research opportunities in the field.

Hermes:Guidance-enriched Visual Analytics for economic network exploration

The economy of a country can be modeled as a complex system in which several players buy and sell goods from each other.By analyzing the investment flows,it is possible to reconstruct the supply chain for the production of most goods,whose understanding is important to analysts and public officials interested in creating and evaluating strategies for informed and strategic decision making,for instance,adjusting tax policies.Those networks of players and investments,however,tend to be complex and very dense,which leads to over-plotted visualizations that obfuscate precious information such as the dependencies between productive sectors and regions.In this paper,we propose Hermes,a guidanceenriched Visual Analytics environment(named after the Greek God of Commerce)for the exploration of complex economic networks,to uncover supply chains,regions’productivity,and sector-to-sector relationships.With practical knowledge regarding guidance,we designed and implemented a visual sub-graph querying approach to extract patterns from such complex investment graphs obtained from real-world data.We present a three-fold evaluation of the system:we perform a qualitative evaluation of our approach with three domain experts,a separate assessment of the proposed guidance features with an expert researcher in this field,and a case study of Hermes using a bank account network dataset to demonstrate the generalizability of our approach.

Visual exploration of movement and event data with interactive time masks

We introduce the concept of time mask,which is a type of temporal filter suitable for selection of multiple disjoint time intervals in which some query conditions fulfil.Such a filter can be applied to time-referenced objects,such as events and trajectories,for selecting those objects or segments of trajectories that fit in one of the selected time intervals.The selected subsets of objects or segments are dynamically summarized in various ways,and the summaries are represented visually on maps and/or other displays to enable exploration.The time mask filtering can be especially helpful in analysis of disparate data(e.g.,event records,positions of moving objects,and time series of measurements),which may come from different sources.To detect relationships between such data,the analyst may set query conditions on the basis of one dataset and investigate the subsets of objects and values in the other datasets that co-occurred in time with these conditions.We describe the desired features of an interactive tool for time mask filtering and present a possible implementation of such a tool.By example of analysing two real world data collections related to aviation and maritime traffic,we show the way of using time masks in combination with other types of filters and demonstrate the utility of the time mask filtering.

Design and validation of a navigation system of multimodal medical images for neurosurgery based on mixed reality

Purpose: This paper aims to develop a navigation system based on mixed reality, which can displaymultimodal medical images in an immersive environment and help surgeons locate the target areaand surrounding important tissues precisely.Methods: To be displayed properly in mixed reality, medical images are processed in this system.High-quality cerebral vessels and nerve fibers with proper colors are reconstructed and exported tomixed reality environment. Multimodal images and models are registered and fused, extracting theirkey information. The multiple processed images are fused with the real patient in the same coordinatesystem to guide the surgery.Results: The multimodal image system is designed and validated properly. In phantom experiments,the average error of preoperative registration is 1.003 mm and the standard deviation is 0.096 mm.The average proportion of well-registered areas is 94.9%. In patient experiments, the surgeons whoparticipated in the experiments generally indicated that the system had excellent performance andgreat application prospect for neurosurgery.Conclusion: This article proposes a navigation system of multimodal images for neurosurgery basedon mixed reality. Compared with other navigation methods, this system can help surgeons locate thetarget area and surrounding important tissues more precisely and rapidly.

AFExplorer:Visual analysis and interactive selection of audio features

Acoustic quality detection is vital in the manufactured products quality control field since it represents the conditions of machines or products.Recent work employed machine learning models in manufactured audio data to detect anomalous patterns.A major challenge is how to select applicable audio features to meliorate model’s accuracy and precision.To relax this challenge,we extract and analyze three audio feature types including Time Domain Feature,Frequency Domain Feature,and Cepstrum Feature to help identify the potential linear and non-linear relationships.In addition,we design a visual analysis system,namely AFExplorer,to assist data scientists in extracting audio features and selecting potential feature combinations.AFExplorer integrates four main views to present detailed distribution and relevance of the audio features,which helps users observe the impact of features visually in the feature selection.We perform the case study with AFExplore according to the ToyADMOS and MIMII Dataset to demonstrate the usability and effectiveness of the proposed system.