Abstract
Dynamic Multi-workflow Scheduling (DMWS) in cloud computing is a well-known combinatorial optimisation problem. It is a great challenge to tackle this problem by scheduling multiple workflows submitted at different times and meet user-defined quality of service objectives. Scheduling with user-defined budget constraints is becoming increasingly important due to cloud dynamics associated with on-demand provisioning, instance types, and pricing. To address the Budget-Constrained Dynamic Multi-workflow Scheduling (BC-DMWS) problem, a novel Cooperative Coevolution Genetic Programming (CCGP) approach is proposed. Two heuristic rules, namely VM Selection/Creation Rule (VMR) and Budget Alert Rule (BAR), are learned automatically by CCGP. VMR is used to allocate ready tasks to either existing or newly rented VM instances, while BAR makes decisions to downgrade VM instances so as to meet the budget constraint. Experiments show significant performance and success rate improvement compared to state-of-the-art algorithms.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arabnejad, V., Bubendorfer, K., Ng, B.: Dynamic multi-workflow scheduling: a deadline and cost-aware approach for commercial clouds. Future Gener. Comput. Syst. 100, 98–108 (2019)
AWS: Amazon EC2 on demand pricing (2022). https://aws.amazon.com/ec2/pricing/on-demand/
Blythe, J., Jain, S., et al.: Task scheduling strategies for workflow-based applications in grids. In: CCGrid 2005. IEEE International Symposium on Cluster Computing and the Grid, 2005, vol. 2, pp. 759–767. IEEE (2005)
Branke, J., Nguyen, S., Pickardt, C.W., Zhang, M.: Automated design of production scheduling heuristics: a review. IEEE Trans. Evol. Comput. 20(1), 110–124 (2015)
Braun, T.D., Siegel, H.J., et al.: A comparison of eleven static heuristics for mapping a class of independent tasks onto heterogeneous distributed computing systems. J. Parallel Distrib. Comput. 61(6), 810–837 (2001)
Burke, E.K., et al.: A classification of hyper-heuristic approaches. In: Gendreau, M., Potvin, J.Y. (eds.) Handbook of Metaheuristics. International Series in Operations Research & Management Science, vol. 146, pp. 449–468. Springer, Boston, MA (2010). https://doi.org/10.1007/978-1-4419-1665-5_15
Chakravarthi, K.K., Neelakantan, P., Shyamala, L., Vaidehi, V.: Reliable budget aware workflow scheduling strategy on multi-cloud environment. Cluster Comput. 25(2), 1189–1205 (2022)
Chen, W., Deelman, E.: Workflowsim: a toolkit for simulating scientific workflows in distributed environments. In: 2012 IEEE 8th International Conference on E-Science, pp. 1–8. IEEE (2012)
Escott, K.-R., Ma, H., Chen, G.: Genetic programming based hyper heuristic approach for dynamic workflow scheduling in the cloud. In: Hartmann, S., Küng, J., Kotsis, G., Tjoa, A.M., Khalil, I. (eds.) DEXA 2020. LNCS, vol. 12392, pp. 76–90. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59051-2_6
Escott, K.R., Ma, H., Chen, G.: A genetic programming hyper-heuristic approach to design high-level heuristics for dynamic workflow scheduling in cloud. In: 2020 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 3141–3148. IEEE (2020)
Escott, K.-R., Ma, H., Chen, G.: Transfer learning assisted GPHH for dynamic multi-workflow scheduling in cloud computing. In: Long, G., Yu, X., Wang, S. (eds.) AI 2022. LNCS (LNAI), vol. 13151, pp. 440–451. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-97546-3_36
Faragardi, H.R., Sedghpour, M.R.S., Fazliahmadi, S., Fahringer, T., Rasouli, N.: GRP-HEFT: a budget-constrained resource provisioning scheme for workflow scheduling in IaaS clouds. IEEE Trans. Parallel Distrib. Syst. 31(6), 1239–1254 (2019)
Gao, T., Wu, C.Q., Hou, A., Wang, Y., Li, R., Xu, M.: Minimizing financial cost of scientific workflows under deadline constraints in multi-cloud environments. In: Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing, pp. 114–121 (2019)
Jakobović, D., Jelenković, L., Budin, L.: Genetic programming heuristics for multiple machine scheduling. In: Ebner, M., O’Neill, M., Ekárt, A., Vanneschi, L., Esparcia-Alcázar, A.I. (eds.) EuroGP 2007. LNCS, vol. 4445, pp. 321–330. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-71605-1_30
Juve, G., Chervenak, A., Deelman, E., Bharathi, S., Mehta, G., Vahi, K.: Characterizing and profiling scientific workflows. Future Gener. Comput. Syst. 29(3), 682–692 (2013)
Koza, J.R.: Genetic programming as a means for programming computers by natural selection. Statist. Comput. 4(2), 87–112 (1994)
Koza, J.R., Koza, J.R.: Genetic Programming: on the Programming of Computers by Means of Natural Selection, vol. 1. MIT Press (1992)
Li, H., Wang, D., Xu, G., Yuan, Y., Xia, Y.: Improved swarm search algorithm for scheduling budget-constrained workflows in the cloud. Soft Comput. 26(8), 3809–3824 (2022). https://doi.org/10.1007/s00500-022-06782-w
Lin, J., Zhu, L., Gao, K.: A genetic programming hyper-heuristic approach for the multi-skill resource constrained project scheduling problem. Expert Syst. Appl. 140, 112915 (2020)
Liu, L., Zhang, M., Buyya, R., Fan, Q.: Deadline-constrained coevolutionary genetic algorithm for scientific workflow scheduling in cloud computing. Concurr. Comput. Pract. Exp. 29(5), e3942 (2017)
MacLachlan, J., Mei, Y.: Look-ahead genetic programming for uncertain capacitated arc routing problem. In: 2021 IEEE Congress on Evolutionary Computation (CEC), pp. 1872–1879. IEEE (2021)
Nguyen, S., Zhang, M., Johnston, M., Tan, K.C.: Automatic design of scheduling policies for dynamic multi-objective job shop scheduling via cooperative coevolution genetic programming. IEEE Trans. Evol. Comput. 18(2), 193–208 (2013)
Rizvi, N., Ramesh, D.: Fair budget constrained workflow scheduling approach for heterogeneous clouds. Cluster Comput. 23(4), 3185–3201 (2020). https://doi.org/10.1007/s10586-020-03079-1
Sahni, J., Vidyarthi, D.P.: A cost-effective deadline-constrained dynamic scheduling algorithm for scientific workflows in a cloud environment. IEEE Trans. Cloud Comput. 6(1), 2–18 (2015)
Shi, T., Ma, H., Chen, G., Hartmann, S.: Location-aware and budget-constrained service deployment for composite applications in multi-cloud environment. IEEE Trans. Parallel Distrib. Syst. 31(8), 1954–1969 (2020)
Tan, B., Ma, H., Mei, Y., Zhang, M.: A cooperative coevolution genetic programming hyper-heuristic approach for on-line resource allocation in container-based clouds. IEEE Trans. Cloud Comput. (2020)
Wu, C.Q., Cao, H.: Optimizing the performance of big data workflows in multi-cloud environments under budget constraint. In: 2016 IEEE International Conference on Services Computing (SCC), pp. 138–145. IEEE (2016)
Xiao, Q.Z., Zhong, J., Feng, L., Luo, L., Lv, J.: A cooperative coevolution hyper-heuristic framework for workflow scheduling problem. IEEE Trans. Serv. Comput. (2019)
Yang, Y., Chen, G., Ma, H., Zhang, M., Huang, V.: Budget and SLA aware dynamic workflow scheduling in cloud computing with heterogeneous resources. In: 2021 IEEE Congress on Evolutionary Computation (CEC), pp. 2141–2148. IEEE (2021)
Yu, Y., Ma, H., Chen, G.: Achieving multi-objective scheduling of heterogeneous workflows in cloud through a genetic programming based approach. In: 2021 IEEE Congress on Evolutionary Computation (CEC), pp. 1880–1887. IEEE (2021)
Zhang, F., Mei, Y., Nguyen, S., Zhang, M.: Collaborative multifidelity-based surrogate models for genetic programming in dynamic flexible job shop scheduling. IEEE Trans. Cybern. (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Escott, KR., Ma, H., Chen, G. (2023). Cooperative Coevolutionary Genetic Programming Hyper-Heuristic for Budget Constrained Dynamic Multi-workflow Scheduling in Cloud Computing. In: Pérez Cáceres, L., Stützle, T. (eds) Evolutionary Computation in Combinatorial Optimization. EvoCOP 2023. Lecture Notes in Computer Science, vol 13987. Springer, Cham. https://doi.org/10.1007/978-3-031-30035-6_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-30035-6_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-30034-9
Online ISBN: 978-3-031-30035-6
eBook Packages: Computer ScienceComputer Science (R0)