Dynamic Resource Allocation Using Virtual Machines and Parallel Data Processing in the Cloud

Authors

  • Bhushan YB Department of Computer Science and Engineering Sri Venkateswara Engineering College, Suryapet
  • Bhavani V Department of Computer Science and Engineering Sri Venkateswara Engineering College, Suryapet

Keywords:

Cloud Computing, Parallel Data Processing, Dynamic resource allocation, High-Throughput Computing, Loosely Coupled Applications

Abstract

The main enabling technology for cloud computing is virtualization which generalize the physical infrastructure and makes it easy to use and manage. Virtualization is used to allocate resources based on their needs and also supports green computing concept. Parallel data processing has emerged to be one of the killer applications for Infrastructure-as-a-Service (IaaS) clouds. The processing frameworks which are currently used have been designed for static, homogeneous cluster setups and disregard the particular nature of a cloud. The allocated compute resources may be inadequate for big parts of the submitted job and unnecessarily increase processing time and cost.

In this paper we are applying the concept of “SKEWNESS” to measure the unevenness in the multi- dimensional resource utilization of a server. By minimizing skewness, we can combine different types of workloads and improve the overall utilization of server resources and discuss the opportunities and challenges for efficient parallel data processing in clouds using “NEPHELE’S ARCHITECTURE”. Nephel’s architecture offers efficient parallel data processing in clouds. It is the first data processing framework for the dynamic resource allocation offered by today’s IaaS clouds for both, task scheduling and execution

References

Amazon Web Services LLC. Amazon Elastic Compute Cloud (Amazon EC2). http://aws.amazon.com/ec2/, 2009.

Amazon Web Services LLC. Amazon Elastic MapReduce. http://aws.amazon.com/elasticmapreduce/, 2009.

AmazonWeb Services LLC. Amazon Simple Storage Service. http://aws.amazon.com/s3/, 2009.

D. Battr´e, S. Ewen, F. Hueske, O. Kao, V. Markl, and D. Warneke. Nephele/PACTs: A Programming Model and Execution Framework for Web-Scale Analytical Processing. In SoCC ’10: Proceedings of the ACM Symposium on Cloud Computing 2010, pages 119–130, New York, NY, USA, 2010. ACM.

R. Chaiken, B. Jenkins, P.-A. Larson, B. Ramsey, D. Shakib,S. Weaver, and J. Zhou. SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets. Proc. VLDB Endow., 1(2):1265–1276, 2008.

M. Nelson, B.-H. Lim, and G. Hutchins, “Fast Transparent Migration for Virtual Machines,” Proc. USENIX Ann. Technical Conf., 2005.M.Young, The Techincal Writers Handbook. Mill Valley, CA:University Science, 1989.

N. Bobroff, A. Kochut, and K. Beaty, “Dynamic Placement of Virtual Machines for Managing SLA Violations,” Proc. IFIP/IEEE Int’l Symp.Integrated Network Management (IM ’07), 2007.

T. Wood, P. Shenoy, A. Venkataramani, and M. Yousif, “Black-Box and Gray-Box Strategies for Virtual Machine Migration,” Proc. Symp. Networked Systems Design and Implementation (NSDI ’07), Apr. 2007.

J. Dean and S. Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. In OSDI’04: Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation, pages 10–10, Berkeley, CA, USA, 2004. USENIX Association.

E. Deelman, G. Singh, M.-H. Su, J. Blythe, Y. Gil, C. Kesselman,G. Mehta, K. Vahi, G. B. Berriman, J. Good, A. Laity, J. C. Jacob,and D. S. Katz. Pegasus: A Framework for Mapping Complex Scientific Workflows onto Distributed Systems. Sci. Program.,13(3):219–237, 2005.

T. Dornemann, E. Juhnke, and B. Freisleben. On-Demand Resource Provisioning for BPEL Workflows Using Amazon’s Elastic Compute Cloud. In CCGRID ’09: Proceedings of the 2009 9 th IEEE/ACM International Symposium on Cluster Computing and the Grid, pages 140–147, Washington, DC, USA, 2009. IEEE Computer Society.

I. Foster and C. Kesselman. Globus: A Metacomputing Infrastructure Toolkit. Intl. Journal of Supercomputer Applications, 11(2):115– 128, 1997.

Downloads

Published

2024-02-26

How to Cite

Bhushan, Y., & Bhavani, V. (2024). Dynamic Resource Allocation Using Virtual Machines and Parallel Data Processing in the Cloud. COMPUSOFT: An International Journal of Advanced Computer Technology, 4(11), 2001–2005. Retrieved from https://ijact.in/index.php/j/article/view/350

Issue

Section

Original Research Article

Similar Articles

<< < 11 12 13 14 15 16 17 18 19 20 > >> 

You may also start an advanced similarity search for this article.