Multimodal documents combining language and graphs are wide-spread in print media as well as in electronic media. One of the most important tasks to be solved in comprehending graph-text combinations is construction o...Multimodal documents combining language and graphs are wide-spread in print media as well as in electronic media. One of the most important tasks to be solved in comprehending graph-text combinations is construction of causal chains among the meaning entities provided by modalities. In this study we focus on the role of annotation position and shape of graph lines in simple line graphs on causal attributions concerning the event presented by the annotation and the processes （i.e, increases and decreases） and states （no-changes） in the domain value of the graphs presented by the process-lines and state-lines. Based on the experimental investigation of readers＇ inferences under different conditions, guidelines for the design of multimodal documents including text and statistical information graphics are suggested. One suggestion is that the position and the number of verbal annotations should be selected appropriately, another is that the graph line smoothing should be done cautiously.展开更多
基金Supported in part by DFG(German Science Foundation) in ITRG1247‘Cross-modal Interaction in Natural and Artificial Cognitive Systems’(CI-NACS)
文摘Multimodal documents combining language and graphs are wide-spread in print media as well as in electronic media. One of the most important tasks to be solved in comprehending graph-text combinations is construction of causal chains among the meaning entities provided by modalities. In this study we focus on the role of annotation position and shape of graph lines in simple line graphs on causal attributions concerning the event presented by the annotation and the processes （i.e, increases and decreases） and states （no-changes） in the domain value of the graphs presented by the process-lines and state-lines. Based on the experimental investigation of readers＇ inferences under different conditions, guidelines for the design of multimodal documents including text and statistical information graphics are suggested. One suggestion is that the position and the number of verbal annotations should be selected appropriately, another is that the graph line smoothing should be done cautiously.
