期刊文献+
任意字段
题名或关键词
题名
关键词
文摘
作者
第一作者
机构
刊名
分类号
参考文献
作者简介
基金资助
栏目信息

M2M:A Simple Matlab-to-MapReduce Translator for Cloud Computing 被引量：1

M2M: A Simple Matlab-to-MapReduce Translator for Cloud Computing
原文传递
导出
摘要 MapReduce is a very popular parallel programming model for cloud computing platforms, and has become an effective method for processing massive data by using a cluster of computers. X-to-MapReduce （X is a program language） translator is a possible solution to help traditional programmers easily deploy an application to cloud systems through translating sequential codes to MapReduce codes. Recently, some SQL- to-MapReduce translators emerge to translate SQL-like queries to MapReduce codes and have good performance in cloud systems. However, SQL-to-MapReduce translators mainly focus on SQL-like queries, but not on numerical computation. Matlab is a high-level language and interactive environment for numerical computation, visualization, and programming, which is very popular in engineering. We propose and develop a simple Matlab-to-MapReduce translator for cloud computing, called M2M, for basic numerical computations. M2M can translate a Matlab code with up to 100 commands to MapReduce code in few seconds, which may cost a proficient Hadoop MapReduce programmer some days on coding so many commands. In addition, M2M can also recognize the dependency between complex commands, which is always confusing during hand coding. We implemented M2M with evaluation for Matlab commands on a cluster. Several common commands are used in our experiments. The results show that M2M is comparable in performance with hand-coded programs. MapReduce is a very popular parallel programming model for cloud computing platforms, and has become an effective method for processing massive data by using a cluster of computers. X-to-MapReduce （X is a program language） translator is a possible solution to help traditional programmers easily deploy an application to cloud systems through translating sequential codes to MapReduce codes. Recently, some SQL- to-MapReduce translators emerge to translate SQL-like queries to MapReduce codes and have good performance in cloud systems. However, SQL-to-MapReduce translators mainly focus on SQL-like queries, but not on numerical computation. Matlab is a high-level language and interactive environment for numerical computation, visualization, and programming, which is very popular in engineering. We propose and develop a simple Matlab-to-MapReduce translator for cloud computing, called M2M, for basic numerical computations. M2M can translate a Matlab code with up to 100 commands to MapReduce code in few seconds, which may cost a proficient Hadoop MapReduce programmer some days on coding so many commands. In addition, M2M can also recognize the dependency between complex commands, which is always confusing during hand coding. We implemented M2M with evaluation for Matlab commands on a cluster. Several common commands are used in our experiments. The results show that M2M is comparable in performance with hand-coded programs.
作者 Junbo Zhang Dong Xiang Tianrui Li Yi Pan
机构地区 the School of Information Scienceand Technology the Department of Computer Science the School of Software the School of Information Science and Technology Department of Computer Science
出处 《Tsinghua Science and Technology》 SCIE EI CAS 2013年第1期1-9,共9页 清华大学学报（自然科学版（英文版）
基金 supported by the National Natural Science Foundation of China(Nos.61175047,61100117,and 61202043) the US National Science Foundation(No.OCI-1156733)
关键词 MAPREDUCE MATLAB TRANSLATOR cloud computing MapReduce Matlab translator cloud computing
分类号 TP3 [自动化与计算机技术—计算机科学与技术]
  • 相关文献

参考文献16

  • 1J. Dean and S. Ghemawat, Mapreduce: Simplified data processing on large clusters, Communications of the ACM, vol. 51, no. 1, pp. 107-113, Jan. 2008.
  • 2T. White, Hadoop: The Definitive Guide, 2nd ed. O'Reilly Media / Yahoo Press, 2010.
  • 3J. Talbot, R. M. Yoo, and C. Kozyrakis, Phoenix++: Modular mapreduce for shared-memory systems, in Proc.of the Second International Workshop on MapReduce and Its Applications, New York, NY, USA: ACM, 2011, pp. 9- 16.
  • 4B. He, W. Fang, Q. Luo, N. K. Govindaraju, and T. Wang, Mars: A mapreduce framework on graphics processors, in Proc. of the 17th International Conference on Parallel Architectures and Compilation Techniques, New York, NY, USA: ACM, 2008, pp. 260-269.
  • 5J. Ekanayake, H. Li, B. Zhang, T. Gunarathne, S.-H. Bae, J. Qiu, and G. Fox, Twister: A runtime for iterative mapreduce, in Proc. of the 19th ACM Int. Symposium on High Performance Distributed Computing, New York, NY, USA: ACM, 2010, pp. 810-818.
  • 6T. Gunarathne, B. Zhang, T.-L. Wu, and J. Qiu, Portable parallel programming on cloud and hpc: Scientific applications of twister4azure, in Utility and Cloud Computing (UCC), 2011 Fourth 1EEE Int. Conf. on, Dec. 2011, pp. 97-104.
  • 7Y. Pan and J. Zhang, Parallel programming on cloud computing platforms: Challenges and solutions, KITCS/FTRA Journal of Convergence, vol. 3, no. 4, pp. 23-28, Dec. 2012.
  • 8A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, R Chakka, S. Anthony, H. Liu, R Wyckoff, and R. Murthy, Hive: A warehousing solution over a map-reduce framework, Proc. VLDB Endow., vol. 2, no. 2, pp. 1626-1629, Aug. 2009.
  • 9R. Lee, T. Luo, Y. Huai, E Wang, Y. He, and X. Zhang, Ysmart: Yet another sql-to-mapreduce translator, in Distributed Computing Systems (ICDCS), 2011 31st Int. Conf. on, June 2011, pp. 25-36.
  • 10A. Gilat, MATLAB: An Introduction with Applications, 4th ed. John Wiley & Sons, 2011.

同被引文献17

  • 1FANUCCI L,SAPONARA S, BACCHILLONE T, et al. sensing devices and sensor signal processing for remote monitoring of vital signs in CHF patients [ J 1. IEEE Transactions on Instrumentation and Measurement, 2013,62 (3) :553-569.
  • 2FADLULLAH Z M, FOUDA M M, KATO N, et al. To- ward intelligent machine to machine communications in smart grid [ J ]. IEEE Communications Magazine, 2011, 49(4) :60-65.
  • 3SHI Y, MORANT M. Muhistandard wireless transmis- Sion over SSMF and large-core POF for access and in- home networks [ J 1. IEEE Photonies Technology Let- ters, 2012,24(9) :736-738.
  • 4HOU R C ,BELQASMI F, GLITHO R H, et al. The de- sign and implementation of architectural components for the integration of the IP multimedia subsystem and wire- less actuator networks[ J]. IEEE Communications Mag- azine, 2011,49 (12) : 138-146.
  • 5XIE X Y, DENG D P. Design of embedded gateway software framework for heterogeneous networks inter- connection [ J ]. Electronics and Optoelectronics, 2011 (2) :306-309.
  • 6HOJIN P,ILWOO L,TAEIN H W,et al. Architecture of home gateway for device collaboration in extended home space[ J]. IEEE Transactions on Consumer Electronics, 2008,54 (4) : 1692 -1697.
  • 7JAEYONG H,SUNHEUM L,SUNHYUNG K,et al. De- velopment of home network gateway supporting both wire and wireless communication[ C]. 2nd International Con- ference on Future Generation Communication and Net- working, Hainan Island, 2008:86-89.
  • 8CHU H C, CHANG S L, LIAO Y H, et al. Design and implementation of heterogeneous wireless gateway [ C ]. IEEE International Conference on Systems, Man and Cy- bernetics, San Antonio, United states, 2009 : 2947-2952.
  • 9LIT T, REN J, TANG X C. Secure wireless monitoring and control systems for smart grid and smart home [ J ]. Wireless Communications,IEEE,2012,19(3) :66-73.
  • 10POSTOLACHE M, NEAMTU G, TROFIN S D. CAN - Ethernet gateway for automotive applications[ C ]. 17th International Conference on System Theory, Control and Computing, Sinaia, Romania,2013:422-427.

引证文献1

二级引证文献10

Tsinghua Science and Technology
2013年 第1期

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部