Abstract
Dynamic workflow scheduling (DWS) aims to allocate abundant cloud resources to process a large number of heterogeneous workflows in order to minimize total operation cost and the penalty for violating deadline constraints. Instead of using manually designed heuristics that cannot work effectively across different problem instances, we develop a new Genetic Programming Hyper-Heuristic (GPHH) algorithm to automatically design scheduling heuristics for a newly formulated deadline-constrained dynamic workflow scheduling in cloud (DCDWSC) problem. Different from previous works, our GPHH algorithm can design a pair of rules for Virtual Machine selection and task selection. A new dual-tree representation is proposed to jointly evolve the rule pair, enabling the algorithm to effectively control the inter-dependencies of the two rules. Experimental results show that our new algorithm can significantly outperform three baseline algorithms on a wide range of testing scenarios.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arabnejad, V., Bubendorfer, K., Ng, B.: Dynamic multi-workflow scheduling: a deadline and cost-aware approach for commercial clouds. Futur. Gener. Comput. Syst. 100, 98–108 (2019)
Armbrust, M., et al.: Above the clouds: a Berkeley view of cloud computing. Technical report (2009)
Chen, W., Deelman, E.: Workflowsim: a toolkit for simulating scientific workflows in distributed environments. In: 2012 IEEE 8th International Conference on E-Science, pp. 1–8. IEEE (2012)
Djigal, H., Feng, J., Lu, J., Ge, J.: IPPTS: an efficient algorithm for scientific workflow scheduling in heterogeneous computing systems. IEEE Trans. Parallel Distrib. Syst. 32(5), 1057–1071 (2020)
Escott, K.-R., Ma, H., Chen, G.: Genetic programming based hyper heuristic approach for dynamic workflow scheduling in the cloud. In: Hartmann, S., Küng, J., Kotsis, G., Tjoa, A.M., Khalil, I. (eds.) DEXA 2020. LNCS, vol. 12392, pp. 76–90. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59051-2_6
Escott, K.R., Ma, H., Chen, G.: A genetic programming hyper-heuristic approach to design high-level heuristics for dynamic workflow scheduling in cloud. In: 2020 IEEE Symposium Series on Computational Intelligence, pp. 3141–3148. IEEE (2020)
Faragardi, H.R., Saleh Sedghpour, M.R., Fazliahmadi, S., Fahringer, T., Rasouli, N.: GRP-HEFT: a budget-constrained resource provisioning scheme for workflow scheduling in IaaS clouds. IEEE Trans. Parallel Distrib. Syst. 31(6), 1239–1254 (2020)
Ismayilov, G., Topcuoglu, H.R.: Neural network based multi-objective evolutionary algorithm for dynamic workflow scheduling in cloud computing. Futur. Gener. Comput. Syst. 102, 307–322 (2020)
Rasouli Kenari, A., Shamsi, M.: A hyper-heuristic selector algorithm for cloud computing scheduling based on workflow features. Opsearch 58(4), 852–868 (2021). https://doi.org/10.1007/s12597-021-00508-6
Liu, J., et al.: Online multi-workflow scheduling under uncertain task execution time in IaaS clouds. IEEE Trans. Cloud Comput. 9(3), 1180–1194 (2019)
Liu, Y., Mei, Y., Zhang, M., Zhang, Z.: A predictive-reactive approach with genetic programming and cooperative coevolution for the uncertain capacitated arc routing problem. Evol. Comput. 28(2), 289–316 (2020)
O’Neill, M.: Riccardo Poli, William B. Langdon, Nicholas F. Mcphee: a field guide to genetic programming (2009)
Rizvi, N., Dharavath, R., Wang, L., Basava, A.: A workflow scheduling approach with modified fuzzy adaptive genetic algorithm in IaaS clouds. IEEE Trans. Serv. Comput. (2022). https://doi.org/10.1109/TSC.2022.3174112
Tan, B., Ma, H., Mei, Y., Zhang, M.: A cooperative coevolution genetic programming hyper-heuristics approach for on-line resource allocation in container-based clouds. IEEE Trans. Cloud Comput. 10(3), 1500–1514 (2022). https://doi.org/10.1109/TCC.2020.3026338
Topcuoglu, H., Hariri, S., Wu, M.Y.: Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans. Parallel Distrib. Syst. 13(3), 260–274 (2002)
Versluis, L., Iosup, A.: A survey of domains in workflow scheduling in computing infrastructures: community and keyword analysis, emerging trends, and taxonomies. Futur. Gener. Comput. Syst. 123, 156–177 (2021)
Wang, Z.J., et al.: Dynamic group learning distributed particle swarm optimization for large-scale optimization and its application in cloud workflow scheduling. IEEE Trans. Cybern. 50(6), 2715–2729 (2020)
Wu, Q., Ishikawa, F., Zhu, Q., Xia, Y., Wen, J.: Deadline-constrained cost optimization approaches for workflow scheduling in clouds. IEEE Trans. Parallel Distrib. Syst. 28(12), 3401–3412 (2017)
Xiao, J.-P., Hu, X.-M., Chen, W.-N.: Dynamic cloud workflow scheduling with a heuristic-based encoding genetic algorithm. In: Yang, H., Pasupa, K., Leung, A.C.-S., Kwok, J.T., Chan, J.H., King, I. (eds.) ICONIP 2020. LNCS, vol. 12533, pp. 38–49. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-63833-7_4
Xiao, Q.Z., Zhong, J., Feng, L., Luo, L., Lv, J.: A cooperative coevolution hyper-heuristic framework for workflow scheduling problem. IEEE Trans. Serv. Comput. 15(1), 150–163 (2022)
Xie, Y., Gui, F.X., Wang, W.J., Chien, C.F.: A two-stage multi-population genetic algorithm with heuristics for workflow scheduling in heterogeneous distributed computing environments. IEEE Trans. Cloud Comput. (2021). https://doi.org/10.1109/TCC.2021.3137881
Yang, Y., Chen, G., Ma, H., Zhang, M., Huang, V.: Budget and SLA aware dynamic workflow scheduling in cloud computing with heterogeneous resources. In: 2021 IEEE Congress on Evolutionary Computation, pp. 2141–2148 (2021)
Youn, C.H., Chen, M., Dazzi, P.: Cloud Broker and Cloudlet for Workflow Scheduling. Springer, Singapore (2017). https://doi.org/10.1007/978-981-10-5071-8
Yu, Y., Feng, Y., Ma, H., Chen, A., Wang, C.: Achieving flexible scheduling of heterogeneous workflows in cloud through a genetic programming based approach. In: 2019 IEEE Congress on Evolutionary Computation, pp. 3102–3109. IEEE (2019)
Zhang, F., Mei, Y., Nguyen, S., Zhang, M.: Correlation coefficient-based recombinative guidance for genetic programming hyperheuristics in dynamic flexible job shop scheduling. IEEE Trans. Evol. Comput. 25(3), 552–566 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yang, Y., Chen, G., Ma, H., Zhang, M. (2022). Dual-Tree Genetic Programming for Deadline-Constrained Dynamic Workflow Scheduling in Cloud. In: Troya, J., Medjahed, B., Piattini, M., Yao, L., Fernández, P., Ruiz-Cortés, A. (eds) Service-Oriented Computing. ICSOC 2022. Lecture Notes in Computer Science, vol 13740. Springer, Cham. https://doi.org/10.1007/978-3-031-20984-0_31
Download citation
DOI: https://doi.org/10.1007/978-3-031-20984-0_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20983-3
Online ISBN: 978-3-031-20984-0
eBook Packages: Computer ScienceComputer Science (R0)