基于深度强化学习的智能车间调度方法研究

罗梓珲; 江呈羚; 刘亮; 郑霄龙; 马华东

doi:10.11959/j.issn.2096-3750.2022.00260

您当前的位置：

首页 >

文章列表页 >

基于深度强化学习的智能车间调度方法研究

理论与技术 | 更新时间：2024-06-05

- 基于深度强化学习的智能车间调度方法研究
- Research on deep reinforcement learning based intelligent shop scheduling method
- 物联网学报 2022年6卷第1期页码：53-64
- 作者机构：
- 作者简介：
  
  [ "罗梓珲（1996− ），男，北京邮电大学博士生，主要研究方向为工业物联网、边缘计算" ]
  [ "江呈羚（1997− ），女，北京邮电大学硕士生，主要研究方向为智能优化调度、深度强化学习" ]
  [ "刘亮（1982− ），男，北京邮电大学教授，主要研究方向为物联网、智能感知技术" ]
  [ "郑霄龙（1989− ），男，北京邮电大学副教授，主要研究方向为物联网、无线网络、普适计算" ]
  [ "马华东（1964− ），男，北京邮电大学教授，主要研究方向为多媒体系统与网络、物联网与传感网、视频理解与大数据分析" ]
- 基金信息：
  
  国家自然科学基金资助项目;The National Natural Science Foundation of China(62061146002);国家自然科学基金资助项目;The National Natural Science Foundation of China(61632008);国家自然科学基金资助项目;The National Natural Science Foundation of China(61921003);中央高校基本科研业务费资助项目;The Fundamental Research Funds for the Central Universities(2019XD-A14)
- DOI：10.11959/j.issn.2096-3750.2022.00260
  中图分类号： TP18
- 纸质出版日期：2022-03-30，
  
  网络出版日期：2022-03，
- 稿件说明：
移动端阅览
罗梓珲, 江呈羚, 刘亮, 等. 基于深度强化学习的智能车间调度方法研究[J]. 物联网学报, 2022,6(1):53-64.

ZIHUI LUO, CHENGLING JIANG, LIANG LIU, et al. Research on deep reinforcement learning based intelligent shop scheduling method. [J]. Chinese journal on internet of things, 2022, 6(1): 53-64.
罗梓珲, 江呈羚, 刘亮, 等. 基于深度强化学习的智能车间调度方法研究[J]. 物联网学报, 2022,6(1):53-64. DOI： 10.11959/j.issn.2096-3750.2022.00260.

ZIHUI LUO, CHENGLING JIANG, LIANG LIU, et al. Research on deep reinforcement learning based intelligent shop scheduling method. [J]. Chinese journal on internet of things, 2022, 6(1): 53-64. DOI： 10.11959/j.issn.2096-3750.2022.00260.

摘要

工业物联网的空前繁荣为传统的工业生产制造模式开辟了一条新的道路。智能车间调度是整个生产过程实现全面控制和柔性生产的关键技术之一，要求以最大完工时间最小化分派多道工序和多台机器的生产调度。首先，将车间调度问题定义为马尔可夫决策过程，建立了一个基于指针网络的车间调度模型。其次，将作业调度过程看作是从一个序列到另一个序列的映射，提出了一种基于深度强化学习的车间调度算法。通过分析模型在不同参数设置下的收敛性，确定了最优参数。在不同规模的公共数据集和实际生产数据集上的实验结果表明，所提出的深度强化学习算法能够取得更好的性能。

Abstract

The unprecedented prosperity of the industrial internet of things (IIoT) has opened up a new path for the traditional industrial manufacturing model.Intelligent shop scheduling is one of the key technologies to achieve the overall control and flexible production of the whole production process.It requires an effective plan with a minimum makespan to allocate multiple processes and multiple machines for production scheduling.Firstly

the shop scheduling problem was defined as a Markov decision process (MDP)

and a shop scheduling model based on the pointer network was established.Secondly

the job scheduling process was regarded as a mapping from one sequence to another

and a new shop scheduling algorithm based on deep reinforcement learning (DRL) was proposed.By analyzing the convergence of the model under different parameter settings

the optimal parameters were determined.Experimental results on different scales of public data sets and actual production data sets show that the proposed DRL algorithm can obtain better performances.

关键词

工业物联网智能车间调度柔性生产深度强化学习车间调度方法

Keywords

IIoTintelligent shop schedulingflexible productiondeep reinforcement learningshop scheduling method

references

GILCHRIST A.Industry 4 . 0:The industrial internet of things[M]. Berkeley,CA: Apress, 2016.

VINYALS O, FORTUNATO M, JAITLY N . Pointer networks[J]. CoRR, 2015:abs/1506.03134.

LING Z X, TAO X Y, ZHANG Y ,et al. Solving optimization problems through fully convolutional networks:an application to the traveling salesman problem[J]. IEEE Transactions on Systems,Man,and Cybernetics:Systems, 2021,51(12): 7475-7485.

NAZARI M, OROOJLOOY A, SNYDER L V ,et al. Reinforcement learning for solving the vehicle routing problem[J]. CoR. 2018:abs/1802.04240.

BELLO I, PHAM H, LE Q V ,et al. Neural combinatorial optimization with reinforcement learning[C]// Proceeding of 5th International Conference on Learning Representations.Toulon, 2017: 1-13.

MNIH V, KAVUKCUOGLU K, SILVER D ,et al. Playing atari with deep reinforcement learning[J]. CoRR. 2013:abs/1312.5602.

ZHANG C, SONG W, CAO Z G ,et al. Learning to dispatch for job shop scheduling via deep reinforcement learning[EB]. 2020.

刘建伟, 高峰, 罗雄麟 . 基于值函数和策略梯度的深度强化学习综述[J]. 计算机学报, 2019,42(6): 1406-1438.

LIU J W, GAO F, LUO X L . Survey of deep reinforcement learning based on value function and policy gradient[J]. Chinese Journal of Computers, 2019,42(6): 1406-1438.

GAREY M R, JOHNSON D S, SETHI R . The complexity of flowshop and jobshop scheduling[J]. Mathematics of Operations Research, 1976,1(2): 117-129.

JOHNSON S M . Optimal two-and three-stage production schedules with setup times included[J]. Naval Research Logistics Quarterly, 1954,1(1): 61-68.

REZA HEJAZI S, SAGHAFIAN S . Flowshop-scheduling problems with makespan criterion:a review[J]. International Journal of Production Research, 2005,43(14): 2895-2929.

ZHANG J, DING G F, ZOU Y S ,et al. Review of job shop scheduling research and its new perspectives under Industry 4.0[J]. Journal of Intelligent Manufacturing, 2019,30(4): 1809-1830.

HARTMANIS J . Computers and intractability:a guide to the theory of np-completeness (Michael R.garey and David S.Johnson)[J]. Siam Review, 1982,24(1): 90-91.

ARTHANARY T S . An extension of two machine sequencing problem[J]. Opsearch, 1971(8): 10-22.

RUIZ R, VÁZQUEZ-RODRÍGUEZ J A, . The hybrid flow shop scheduling problem[J]. European Journal of Operational Research, 2010,205(1): 1-18.

TOSUN Ö, MARICHELVAM M K, TOSUN N . A literature review on hybrid flow shop scheduling[J]. International Journal of Advanced Operations Management, 2020,12(2): 156.

李颖俐, 李新宇, 高亮 . 混合流水车间调度问题研究综述[J]. 中国机械工程, 2020,31(23): 2798-2813,2828.

LI Y L, LI X Y, GAO L . Review on hybrid flow shop scheduling problems[J]. China Mechanical Engineering, 2020,31(23): 2798-2813,2828.

夏柱昌, 刘芳, 公茂果 ,等. 基于记忆库拉马克进化算法的作业车间调度[J]. 软件学报, 2010,21(12): 3082-3093.

XIA Z C, LIU F, GONG M G ,et al. Memory based Lamarckian evolutionary algorithm for job shop scheduling problem[J]. Journal of Software, 2010,21(12): 3082-3093.

REN T, WANG X Y, LIU T Y ,et al. Exact and metaheuristic algorithms for flow-shop scheduling problems with release dates[J]. Engineering Optimization, 2021: 1-17.

HIDRI L, ELKOSANTINI S, M MABKHOT M . Exact and heuristic procedures for the two-center hybrid flow shop scheduling problem with transportation times[J]. IEEE Access, 2018,6: 21788-21801.

HUNSUCKER J L, SHAH J R . Comparative performance analysis of priority rules in a constrained flow shop with multiple processors environment[J]. European Journal of Operational Research, 1994,72(1): 102-114.

CAMPBELL H G, DUDEK R A, SMITH M L . A heuristic algorithm for thenJob,mMachine sequencing problem[J]. Management Science, 1970,16(10): B-630.

CHEN W, HAO Y F . Genetic algorithm-based design and simulation of manufacturing flow shop scheduling[J]. International Journal of Simulation Modelling, 2018,17(4): 702-711.

ENGIN O, GÜÇLÜ A, . A new hybrid ant colony optimization algorithm for solving the no-wait flow shop scheduling problems[J]. Applied Soft Computing, 2018(72): 166-176.

LI X T, MA S J . Multiobjective discrete artificial bee colony algorithm for multiobjective permutation flow shop scheduling problem with sequence dependent setup times[J]. IEEE Transactions on Engineering Management, 2017,64(2): 149-165.

CUNHA B, MADUREIRA A M, FONSECA B ,et al. Deep reinforcement learning as a job shop scheduling solver:a literature review[C]// Hybrid Intelligent Systems. 2020.

LIU C L, CHANG C C, TSENG C J . Actor-critic deep reinforcement learning for solving job shop scheduling problems[J]. IEEE Access, 2020(8): 71752-71762.

HAN B A, YANG J J . Research on adaptive job shop scheduling problems based on dueling double DQN[J]. IEEE Access, 2020,8: 186474-186495.

WANG L B, HU X, WANG Y ,et al. Dynamic job-shop scheduling in smart manufacturing using deep reinforcement learning[J]. Computer Networks, 2021,190: 107969.

LUO S . Dynamic scheduling for flexible job shop with new job insertions by deep reinforcement learning[J]. Applied Soft Computing, 2020,91: 106208.

ZHANG C, SONG W, CAO Z G ,et al. Learning to dispatch for job shop scheduling via deep reinforcement learning[J]. Advances in Neural Information Processing Systems, 2020,33.

王凌, 潘子肖 . 基于深度强化学习与迭代贪婪的流水车间调度优化[J]. 控制与决策, 2021,36(11): 2609-2617.

WANG L, PAN Z X . Scheduling optimization for flow-shop based on deep reinforcement learning and iterative greedy method[J]. Control and Decision, 2021,36(11): 2609-2617.

LUO B, WANG S B, YANG B ,et al. An improved deep reinforcement learning approach for the dynamic job shop scheduling problem with random job arrivals[J]. Journal of Physics:Conference Series, 2021,1848(1): 012029.

TAILLARD E . Benchmarks for basic scheduling problems[J]. European Journal of Operational Research, 1993,64(2): 278-285.

浏览量

925

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于能量收集技术的协作卸载计算方案

面向工业物联网场景下行短包传输的叠加导频功率优化

基于连续动作空间深度强化学习的多数据融合室内定位方法

基于深度强化学习的SDN服务质量智能优化算法

基于深度强化学习的物联网智能路由策略