Research on deep reinforcement learning based intelligent shop scheduling method

Zihui LUO; Chengling JIANG; Liang LIU; Xiaolong ZHENG; Huadong MA

doi:10.11959/j.issn.2096-3750.2022.00260

您当前的位置：

首页 >

文章列表页 >

Research on deep reinforcement learning based intelligent shop scheduling method

Theory and Technology | 更新时间：2024-06-05

- Research on deep reinforcement learning based intelligent shop scheduling method
- Chinese Journal on Internet of Things Vol. 6, Issue 1, Pages: 53-64(2022)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-3750.2022.00260
  CLC： TP18
- Published：30 March 2022，
  
  Published Online：2022-03，
- 稿件说明：
移动端阅览
ZIHUI LUO, CHENGLING JIANG, LIANG LIU, et al. Research on deep reinforcement learning based intelligent shop scheduling method. [J]. Chinese journal on internet of things, 2022, 6(1): 53-64.
DOI：

ZIHUI LUO, CHENGLING JIANG, LIANG LIU, et al. Research on deep reinforcement learning based intelligent shop scheduling method. [J]. Chinese journal on internet of things, 2022, 6(1): 53-64. DOI： 10.11959/j.issn.2096-3750.2022.00260.

摘要

工业物联网的空前繁荣为传统的工业生产制造模式开辟了一条新的道路。智能车间调度是整个生产过程实现全面控制和柔性生产的关键技术之一，要求以最大完工时间最小化分派多道工序和多台机器的生产调度。首先，将车间调度问题定义为马尔可夫决策过程，建立了一个基于指针网络的车间调度模型。其次，将作业调度过程看作是从一个序列到另一个序列的映射，提出了一种基于深度强化学习的车间调度算法。通过分析模型在不同参数设置下的收敛性，确定了最优参数。在不同规模的公共数据集和实际生产数据集上的实验结果表明，所提出的深度强化学习算法能够取得更好的性能。

Abstract

The unprecedented prosperity of the industrial internet of things (IIoT) has opened up a new path for the traditional industrial manufacturing model.Intelligent shop scheduling is one of the key technologies to achieve the overall control and flexible production of the whole production process.It requires an effective plan with a minimum makespan to allocate multiple processes and multiple machines for production scheduling.Firstly

the shop scheduling problem was defined as a Markov decision process (MDP)

and a shop scheduling model based on the pointer network was established.Secondly

the job scheduling process was regarded as a mapping from one sequence to another

and a new shop scheduling algorithm based on deep reinforcement learning (DRL) was proposed.By analyzing the convergence of the model under different parameter settings

the optimal parameters were determined.Experimental results on different scales of public data sets and actual production data sets show that the proposed DRL algorithm can obtain better performances.

关键词

工业物联网智能车间调度柔性生产深度强化学习车间调度方法

Keywords

IIoTintelligent shop schedulingflexible productiondeep reinforcement learningshop scheduling method

references

GILCHRIST A.Industry 4 . 0:The industrial internet of things[M]. Berkeley,CA: Apress, 2016.

VINYALS O, FORTUNATO M, JAITLY N . Pointer networks[J]. CoRR, 2015:abs/1506.03134.

LING Z X, TAO X Y, ZHANG Y ,et al. Solving optimization problems through fully convolutional networks:an application to the traveling salesman problem[J]. IEEE Transactions on Systems,Man,and Cybernetics:Systems, 2021,51(12): 7475-7485.

NAZARI M, OROOJLOOY A, SNYDER L V ,et al. Reinforcement learning for solving the vehicle routing problem[J]. CoR. 2018:abs/1802.04240.

BELLO I, PHAM H, LE Q V ,et al. Neural combinatorial optimization with reinforcement learning[C]// Proceeding of 5th International Conference on Learning Representations.Toulon, 2017: 1-13.

MNIH V, KAVUKCUOGLU K, SILVER D ,et al. Playing atari with deep reinforcement learning[J]. CoRR. 2013:abs/1312.5602.

ZHANG C, SONG W, CAO Z G ,et al. Learning to dispatch for job shop scheduling via deep reinforcement learning[EB]. 2020.

刘建伟, 高峰, 罗雄麟 . 基于值函数和策略梯度的深度强化学习综述[J]. 计算机学报, 2019,42(6): 1406-1438.

LIU J W, GAO F, LUO X L . Survey of deep reinforcement learning based on value function and policy gradient[J]. Chinese Journal of Computers, 2019,42(6): 1406-1438.

GAREY M R, JOHNSON D S, SETHI R . The complexity of flowshop and jobshop scheduling[J]. Mathematics of Operations Research, 1976,1(2): 117-129.

JOHNSON S M . Optimal two-and three-stage production schedules with setup times included[J]. Naval Research Logistics Quarterly, 1954,1(1): 61-68.

REZA HEJAZI S, SAGHAFIAN S . Flowshop-scheduling problems with makespan criterion:a review[J]. International Journal of Production Research, 2005,43(14): 2895-2929.

ZHANG J, DING G F, ZOU Y S ,et al. Review of job shop scheduling research and its new perspectives under Industry 4.0[J]. Journal of Intelligent Manufacturing, 2019,30(4): 1809-1830.

HARTMANIS J . Computers and intractability:a guide to the theory of np-completeness (Michael R.garey and David S.Johnson)[J]. Siam Review, 1982,24(1): 90-91.

ARTHANARY T S . An extension of two machine sequencing problem[J]. Opsearch, 1971(8): 10-22.

RUIZ R, VÁZQUEZ-RODRÍGUEZ J A, . The hybrid flow shop scheduling problem[J]. European Journal of Operational Research, 2010,205(1): 1-18.

TOSUN Ö, MARICHELVAM M K, TOSUN N . A literature review on hybrid flow shop scheduling[J]. International Journal of Advanced Operations Management, 2020,12(2): 156.

李颖俐, 李新宇, 高亮 . 混合流水车间调度问题研究综述[J]. 中国机械工程, 2020,31(23): 2798-2813,2828.

LI Y L, LI X Y, GAO L . Review on hybrid flow shop scheduling problems[J]. China Mechanical Engineering, 2020,31(23): 2798-2813,2828.

夏柱昌, 刘芳, 公茂果 ,等. 基于记忆库拉马克进化算法的作业车间调度[J]. 软件学报, 2010,21(12): 3082-3093.

XIA Z C, LIU F, GONG M G ,et al. Memory based Lamarckian evolutionary algorithm for job shop scheduling problem[J]. Journal of Software, 2010,21(12): 3082-3093.

REN T, WANG X Y, LIU T Y ,et al. Exact and metaheuristic algorithms for flow-shop scheduling problems with release dates[J]. Engineering Optimization, 2021: 1-17.

HIDRI L, ELKOSANTINI S, M MABKHOT M . Exact and heuristic procedures for the two-center hybrid flow shop scheduling problem with transportation times[J]. IEEE Access, 2018,6: 21788-21801.

HUNSUCKER J L, SHAH J R . Comparative performance analysis of priority rules in a constrained flow shop with multiple processors environment[J]. European Journal of Operational Research, 1994,72(1): 102-114.

CAMPBELL H G, DUDEK R A, SMITH M L . A heuristic algorithm for thenJob,mMachine sequencing problem[J]. Management Science, 1970,16(10): B-630.

CHEN W, HAO Y F . Genetic algorithm-based design and simulation of manufacturing flow shop scheduling[J]. International Journal of Simulation Modelling, 2018,17(4): 702-711.

ENGIN O, GÜÇLÜ A, . A new hybrid ant colony optimization algorithm for solving the no-wait flow shop scheduling problems[J]. Applied Soft Computing, 2018(72): 166-176.

LI X T, MA S J . Multiobjective discrete artificial bee colony algorithm for multiobjective permutation flow shop scheduling problem with sequence dependent setup times[J]. IEEE Transactions on Engineering Management, 2017,64(2): 149-165.

CUNHA B, MADUREIRA A M, FONSECA B ,et al. Deep reinforcement learning as a job shop scheduling solver:a literature review[C]// Hybrid Intelligent Systems. 2020.

LIU C L, CHANG C C, TSENG C J . Actor-critic deep reinforcement learning for solving job shop scheduling problems[J]. IEEE Access, 2020(8): 71752-71762.

HAN B A, YANG J J . Research on adaptive job shop scheduling problems based on dueling double DQN[J]. IEEE Access, 2020,8: 186474-186495.

WANG L B, HU X, WANG Y ,et al. Dynamic job-shop scheduling in smart manufacturing using deep reinforcement learning[J]. Computer Networks, 2021,190: 107969.

LUO S . Dynamic scheduling for flexible job shop with new job insertions by deep reinforcement learning[J]. Applied Soft Computing, 2020,91: 106208.

ZHANG C, SONG W, CAO Z G ,et al. Learning to dispatch for job shop scheduling via deep reinforcement learning[J]. Advances in Neural Information Processing Systems, 2020,33.

王凌, 潘子肖 . 基于深度强化学习与迭代贪婪的流水车间调度优化[J]. 控制与决策, 2021,36(11): 2609-2617.

WANG L, PAN Z X . Scheduling optimization for flow-shop based on deep reinforcement learning and iterative greedy method[J]. Control and Decision, 2021,36(11): 2609-2617.

LUO B, WANG S B, YANG B ,et al. An improved deep reinforcement learning approach for the dynamic job shop scheduling problem with random job arrivals[J]. Journal of Physics:Conference Series, 2021,1848(1): 012029.

TAILLARD E . Benchmarks for basic scheduling problems[J]. European Journal of Operational Research, 1993,64(2): 278-285.

Views

925

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Collaborative offloading computing scheme based on energy harvesting technology

Power optimization of superimposed pilot for downlink short-packet transmission in IIoT

Multi-data fusionaided indoor localization based on continuous action space deep reinforcement learning

Quality of service optimization algorithm based on deep reinforcement learning in software defined network

Intelligent routing strategy in the Internet of things based on deep reinforcement learning

Related Author

ZHAO Haodong

WANG Jun

ZHAO Haitao

WU Weihua

ZHOU Xingguang

XU Jing

XIA Wenchao

Jiaxuan YI

Related Institution

Nanjing University of Posts and Telecommunications

College of Communication and Information Engineering, Nanjing University of Posts and Telecommunications

Jiangsu Provincial Key Laboratory of Wireless Communications, Nanjing University of Posts and Telecommunications

School of Physics and Information Technology, Shaanxi Normal University

School of Electronic Information, Central South University

⁰