Distributed Shared Memory (DSM) systems have gained popularacceptance by combining the scalability and low cost of distributed system with theease of use of single address space. Many new hardware DSM and software DSM...Distributed Shared Memory (DSM) systems have gained popularacceptance by combining the scalability and low cost of distributed system with theease of use of single address space. Many new hardware DSM and software DSMsystems have been proposed in recent years. In general, benchmarking is widely usedto demonstrate the performance advantages of new systems. However, the commonmethod used to summarize the measured results is the arithmetic mean of ratios,which is incorrect in some cases. Furthermore, many published papers list a lot ofdata only, and do not summarize them effectively, which confuse users greatly. Infact, many users want to get a single number as conclusion, which is not providedin old summarizing techniques. Therefore, a new data-summarizing technique basedon confidence interval is proposed in this paper. The new technique includes twodata-summarizing methods: (1) paired confidence interval method; (2) unpairedconfidence interval method. With this new technique, it is concluded that at someconfidence one system is better than others. Four examples are shown to demonstratethe advantages of this new technique. Furthermore, with the help of confidence level,it is proposed to standardize the benchmarks used for evaluating DSM systems sothat a convincing result can be got. In addition, the new summarizing technique fitsnot only for evaluating DSM systems, but also for evaluating other systems, such asmemory system and communication systems.展开更多
文摘Distributed Shared Memory (DSM) systems have gained popularacceptance by combining the scalability and low cost of distributed system with theease of use of single address space. Many new hardware DSM and software DSMsystems have been proposed in recent years. In general, benchmarking is widely usedto demonstrate the performance advantages of new systems. However, the commonmethod used to summarize the measured results is the arithmetic mean of ratios,which is incorrect in some cases. Furthermore, many published papers list a lot ofdata only, and do not summarize them effectively, which confuse users greatly. Infact, many users want to get a single number as conclusion, which is not providedin old summarizing techniques. Therefore, a new data-summarizing technique basedon confidence interval is proposed in this paper. The new technique includes twodata-summarizing methods: (1) paired confidence interval method; (2) unpairedconfidence interval method. With this new technique, it is concluded that at someconfidence one system is better than others. Four examples are shown to demonstratethe advantages of this new technique. Furthermore, with the help of confidence level,it is proposed to standardize the benchmarks used for evaluating DSM systems sothat a convincing result can be got. In addition, the new summarizing technique fitsnot only for evaluating DSM systems, but also for evaluating other systems, such asmemory system and communication systems.
