Artificial Intelligence-Driven Customized Manufacturing Factory: Key Technologies, Applications, and Challenges
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
PROCEEDINGS OF THE IEEE, VOL. XX, NO. X, FEBRUARY 2020 1
Artificial Intelligence-Driven Customized
Manufacturing Factory: Key Technologies,
Applications, and Challenges
Jiafu Wan, Member, IEEE, Xiaomin Li, Hong-Ning Dai, Senior Member, IEEE, Andrew Kusiak, Life
Member, IEEE, Miguel Martı́nez-Garcı́a, Member, IEEE, and Di Li
Abstract—The traditional production paradigm of large batch
production does not offer flexibility towards satisfying the
requirements of individual customers. A new generation of
T HE Industry 4.0 initiative is advocating smart manu-
facturing as the industrial revolution leading to global
economic growth [1]–[4]. Many countries, corporations, and
smart factories is expected to support new multi-variety and
small-batch customized production modes. For that, Artificial research institutions have embraced the concept of Industry
Intelligence (AI) is enabling higher value-added manufacturing 4.0, in particular the United States, the European Union, and
by accelerating the integration of manufacturing and information East Asia [5]. Some industries have begun a transformation
communication technologies, including computing, communica- from the digital era to the intelligent era. Manufacturing repre-
tion, and control. The characteristics of a customized smart sents a large segment of the global economy, while the interest
factory are to include self-perception, operations optimization,
dynamic reconfiguration, and intelligent decision-making. The in smart manufacturing is expanding [6]. The progress in
AI technologies will allow manufacturing systems to perceive the information and communication technologies, for example, the
environment, adapt to the external needs, and extract the process Internet of Things (IoT) [7], [8], artificial intelligence (AI) [9],
knowledge, including business models, such as intelligent produc- [10], and big data [11], [12] for manufacturing applications,
tion, networked collaboration, and extended service models. has impacted smart manufacturing [13]. In the broad context
This paper focuses on the implementation of AI in cus-
tomized manufacturing (CM). The architecture of an AI-driven of manufacturing, customized manufacturing (CM) offers a
customized smart factory is presented. Details of intelligent value-added paradigm for smart manufacturing [14], as it
manufacturing devices, intelligent information interaction, and refers to personalized products and services. The benefits of
construction of a flexible manufacturing line are showcased. CM have been highlighted by multinational companies.
The state-of-the-art AI technologies of potential use in CM, i.e., Today, information and communication technologies are the
machine learning, multi-agent systems, Internet of Things, big
data, and cloud-edge computing are surveyed. The AI-enabled base of smart manufacturing [15], [16], and intelligent systems
technologies in a customized smart factory are validated with driven by AI are the core of CM [17]. With the development
a case study of customized packaging. The experimental results of AI technologies, new theories, models, algorithms, and
have demonstrated that the AI-assisted CM offers the possibility applications - towards simulating, extending, and enhancing
of higher production flexibility and efficiency. Challenges and human intelligence - are continuously developed. The progress
solutions related to AI in CM are also discussed.
of big data analysis and deep learning has accelerated AI
Index Terms—Customized Manufacturing; Artificial Intelli- to enter the 2.0 era [18]–[20]. AI 2.0 manifests itself as
gence; Industry 4.0; Smart Factory; Software-Defined Network.
a data-driven deep reinforcement learning intelligence [21],
network-based swarm intelligence [22], technology-oriented
hybrid intelligence of human-machine and brain-machine in-
I. I NTRODUCTION
teraction [23]–[25], cross-media reasoning intelligence [26],
This work was supported in part by the National Key R & D Program of [27], etc. Therefore, AI 2.0 offers significant potential to smart
China (Grant No. 2018YFB1700500), the Joint Fund of the National Natural manufacturing, especially, CM in smart factories [28].
Science Foundation of China and Guangdong Province (Grant No. U1801264),
and Macao Science and Technology Development Fund under Macao Funding Typically, AI solutions can be applied to several aspects
Scheme for Key R & D Projects (0025/2019/AKP). (Corresponding author: of smart manufacturing. AI algorithms can run the manu-
Hong-Ning Dai.) facturing of personalized products in a smart factory [29],
J. Wan and D. Li are with the School of Mechanical and Automotive
Engineering, South China University of Technology, Guangzhou, China (e- [30]. The AI-assisted CM is to construct smart manufacturing
mails: mejwan@scut.edu.cn, itdili@scut.edu.cn). systems supported by cognitive computing, machine status
X. Li is with the School of Mechanical Engineering, Zhongkai Uni- sensing, real-time data analysis, and autonomous decision-
versity of Agriculture and Engineering, Guangzhou, China (e-mail: lixi-
aomin@zhku.edu.cn). making [31], [32]. AI permeates through every link of CM
H.-N. Dai is with the Faculty of Information Technology, Macau University value chains, such as design, production, management, and
of Science and Technology, Macau SAR (email: hndai@ieee.org). service [33], [34]. Based on these insights of CM and AI,
A. Kusiak is with the Intelligent Systems Laboratory, Department of
Mechanical and Industrial Engineering, The University of Iowa, Iowa City, the focus of this paper is on the implementation of AI in the
USA (email: andrew-kusiak@uiowa.edu). smart factory for CM involving architecture, manufacturing
M. Martı́nez-Garcı́a is with the Dept. of Aeronautical and Auto- equipment, information exchange, flexible production line, and
motive Engineering, Loughborough University, UK (email: m.martinez-
garcia@lboro.ac.uk). smart manufacturing services.
Manuscript received xx; revised xx. The contributions of the research presented in this paper are
PROCEEDINGS OF THE IEEE, VOL. XX, NO. X, FEBRUARY 2020 2
as follows. processing/detection/assembly equipment, and storage,
• The architecture of the AI-assisted CM for smart factories all operating in a heterogeneous industrial network.
is developed by merging smart devices and industrial The Industrial IoT has progressed from the original
networks with big data analysis. industrial sensor networks to the Narrow Band-Internet
• The state-of-the-art AI technologies are reviewed and of Things (NB-IoT), LoRa WAN, and LTE Cat M1 with
discussed. increased coverage at reduced power consumption [38].
• The key AI-enabled technologies in CM are validated Edge computing units are deployed to improve system
with a prototype platform of a customized candy pack- intelligence. Cognitive technology ensures the context
aging line. awareness and semantic understanding of the industrial
• The challenges and possible solutions brought by the IoT [39]. Intelligent industrial IoT as the key technologies
introduction of AI into CM are discussed. is widely used for intelligent manufacturing.
The remainder of the paper is organized as follows. In Sec- • Dynamic reconfiguration. The concept of a smart factory
tion II, the relationship between the CM and AI is discussed. aims at the rapid manufacturing of a variety of products
The general architecture of AI-assisted CM is presented in in small batches. Since the product types may change
Section III. Section IV illustrates the implementation of AI dynamically, system resources need to be dynamically
in intelligent manufacturing equipment. The intelligent infor- reorganized. A multi-agent system [40] is introduced to
mation exchange process, flexible production line, and smart negotiate a new system configuration.
manufacturing services in the AI-assisted CM are proposed in • Massive volumes of data. An intelligent manufacturing
Section V and Section VI, respectively. A case study is pro- system includes interconnected devices generating data
vided in Section VII. The challenges and possible solutions to such as device status and process parameters. Cloud com-
the AI-assisted intelligent manufacturing factory are discussed puting and big data science make data analysis feasible
in Section VIII. Section IX concludes the paper. in failure prediction, active preventive maintenance, and
decision making.
II. C USTOMIZED M ANUFACTURING AND A RTIFICIAL • Deep integration. The underlying intelligent manufac-
I NTELLIGENCE turing entities, cloud platforms, edge servers, and up-
per monitoring terminals are closely connected. Data
This section first summarizes the characteristics of cus-
processing, control, and operations can be performed
tomized manufacturing in Section II-A and then discusses the
simultaneously in the Cyber-Physical Systems (CPS),
opportunities brought by AI-driven customized manufacturing
where the information barriers are broken down, thereby
in Section II-C.
realizing the deep integration of physical and information
environments.
A. Characteristics of customized manufacturing
Despite the progress made, manufacturing industry faces a B. Overview of AI technologies
number of challenges, some of which are: traditional mass- AI embraces theories, methods, technologies, and applica-
production is not able to adapt to the rapid production of tions to augment human intelligence. It includes not only AI
personalized products; and resource limitations, environmental techniques such as perception, machine learning (ML), deep
pollution, global warming, and an aging global population learning (DL), reinforcement learning, and decision making,
have become more prominent. Therefore, a new manufac- but also AI-enabled applications like computer vision, natural
turing paradigm to address these challenges is needed. The language processing, intelligent robots, and recommendation
customer-to-manufacture concept reflects the characteristics of systems, as shown in Fig. 1a. ML has outperformed traditional
customized production where a manufacturing system directly statistical methods in tasks such as classification, regression,
interacts with a customer to meet his/her personalized needs. clustering, and rule extraction [41]. Typical ML algorithms
The goal is to realize the rapid customization of personalized include decision tree, support vector machines, regression
products. The new generation of intelligent manufacturing analysis, Bayesian networks, and deep neural networks.
technology offers improved flexibility, transparency, resource As a subset of ML algorithms, DL algorithms have superior
utilization, and efficiency of manufacturing processes. It has performance than other ML algorithms. The recent success of
led to new programs, e.g., the Factory of the Future in DL algorithms mainly owes to three factors: 1) the availability
Europe [35], Industry 4.0 in Germany [1], and Made in of massive data; 2) the advent of computer capability achieved
China 2025 [36]. Moreover, the United States has accelerated by computer architectures and hardware, such as Graphic
research and development programs [37]. Processing Units (GPUs); 3) the advances in diverse DL
Compared with mass production, the production organi- algorithms such as a convolutional neural network (CNN),
zation of CM is more complex, quality control is more long short-term memory (LSTM) and their variants. Different
difficult, and the energy consumption needs attention. In from ML methods, which require substantial efforts in feature
classical automation, the production boundaries were rigid to engineering in processing raw industrial data, DL methods
ensure quality, cost, and efficiency. Compared with traditional combine feature engineering and learning process together,
production, CM has the following characteristics. thereby achieving outstanding performance.
• Smart interconnectivity. Smart manufacturing However, DL algorithms also have their disadvantages.
embraces a cyber-physical environment, e.g., First, DL algorithms often require a huge amount of data
PROCEEDINGS OF THE IEEE, VOL. XX, NO. X, FEBRUARY 2020 3
Knowledge
Knowledge Machine
Machinelearning
learning Computer
Computer Customized
Customized
graph
graphanalysis
analysis Customer
Customer After-sales
After-sales
vision
vision product
product manage-
manage- service
service
design
design ment
ment
Reinforcement
Reinforcement
learning
learning Natural
Naturallanguage
language
Manufact-
Manufact-
processing
processing Customized
Customized
AI
AI uring
uring
product
product
Market
Market
analysis
analysis
manage-
manage-
Deep
Deeplearning
learning manufacturing
manufacturing
foster
foster ment
ment
Perception
Perception
Speech
Speech Customized
Customized
recognition
recognition Manufacturing
Manufacturing
maintenance
maintenance
product
product …
…
Decision
Decisionmaking
making logistics
logistics
Recommendation
Recommendation Customized
Customizedmanufacturing
manufacturing
system
system Intelligent
Intelligentrobot
robot
(a) (b)
Fig. 1. The AI and customized manufacturing. (a) AI technologies include perception, machine learning, deep learning, reinforcement
learning, and decision making as well as AI-enabled applications like computer vision, natural language processing, intelligent robots,
and recommendation systems. (b) AI can foster customized manufacturing in the aspects: customized product design, customized product
manufacturing, manufacturing maintenance, customer management, logistics, after-sales service, and market analysis.
to train DL models to achieve better performance than other of cognitive capabilities, learning, and reasoning (e.g.,
ML algorithms. Moreover, the training of DL models requires analysis of order quantities, lead time, faults, errors, and
substantial computing resources (e.g., expensive GPUs and downtime). Product defects and process anomalies can
other computer hardware devices). Third, DL algorithms also be identified using computer vision and foreign object
suffer from poor interpretability, i.e., a DL model is like an detection. Human operators can be alerted to process
uncontrollable “black box”, which may not obtain the result deviations.
as predicted. The poor interpretability of DL models may 2) Facilitating predictive maintenance. Scheduled mainte-
prevent their wide adoption in industrial systems, especially in nance ensures that the equipment is in the best state.
critical tasks like fault diagnosis [42] despite recent advances Sensors installed on a production line collect data for
in improving the interpretability of DL models [43]. analysis with ML algorithms, including convolutional
neural networks. For example, the wear and tear of a
machine can be detected in real-time and a notification
C. AI-driven customized manufacturing
can be issued.
As AI technologies have demonstrated their potential in 3) Developing of smart supply chains. The variability and
areas such as customized product design, customized prod- uncertainty of supply chains for CM can be predicted
uct manufacturing, manufacturing management, manufactur- with ML algorithms. Moreover, the insights obtained can
ing maintenance, customer management, logistics, after-sales be used to predict sudden changes in customer demands.
service, and market analysis as shown in Fig. 1b, industrial In short, the incorporation of AI and industrial IoT
practitioners and researchers have begun their implementation. brings benefits to smart manufacturing. AI-assisted tools
For example, the work [44] presents a Bayesian network- improve manufacturing efficiency. Meanwhile, higher value-
based approach to analyze the consumers’ purchase behaviour added products can be introduced to the market.
via analyzing RFID data, which is collected from RFID-tags However, we cannot deny that AI technologies still have
attached to in-store shopping carts. Moreover, a deep learning their limitations when they are formally adopted to real-
method is adopted to identify possible machine faults through world manufacturing scenarios. On the one hand, AI and ML
analyzing mechanic data collected from the real industrial algorithms often have stringent requirements on computing
environments such as induction motors, gearboxes, and bear- facilities. For example, high-performance computing servers
ings [45]. equipped with GPUs are often required to fasten the training
Therefore, the introduction of AI technologies can poten- process on massive data [48] while exiting manufacturing fa-
tially realize the customized manufacturing. We name such cilities may not fulfill the stringent requirement on computing
AI-driven customized manufacturing as AI-driven CM. In capability. Therefore, the common practice is to outsource (or
summary, AI-driven CM has the following advantages [46], upload) the manufacturing data to cloud computing service
[47]. providers who can conduct the computing-intensive tasks.
1) Improved production efficiency and product quality. In Nevertheless, outsourcing the manufacturing data to the third
CM factories, automated devices can potentially make party may lead to the risk of leaking confidential data (e.g.,
decisions with reduced human interventions. Technolo- customized product design) or exposing private customer data
gies such as ML and computer vision are enablers to others. On the other hand, transferring the manufacturing
PROCEEDINGS OF THE IEEE, VOL. XX, NO. X, FEBRUARY 2020 4
Wireless link
Wired link
Router
Base station
Access point Logistics Customers
Suppliers Markets Computing center
Management
Smart
interation
Customer Visualization
Production
analysis
line
Edge Cloud
computing computing
Sensors
Smart Smart Sensors
Robots Motors devices services
AI models Data
Machines Maintenance Profits
Fig. 2. The architecture of AI-assisted customized manufacturing includes smart devices, smart interaction, AI layer, and smart services.
data to remote clouds inevitably leads to high latency, thereby devices, realize intelligent information interaction, and provide
failing to fulfill the real-time requirement of time-sensitive intelligent manufacturing services by merging AI technologies.
tasks. As shown in Fig. 2, an AI-assisted CM framework that
includes smart devices, smart interaction, AI layer, and smart
III. A RCHITECTURE OF AN AI-A SSISTED C USTOMIZED services. We then explain this framework in detail as follows.
M ANUFACTURING FACTORY 1) Smart devices: include robots, conveyors, and other ba-
This section first presents an AI-Assisted customized manu- sic controlled platforms. Smart devices serve as “the physical
facturing (AIaCM) framework in Section III-A and then gives layer” for the entire AIaCM. Specifically, different devices and
a brief comparison of the proposed AIaCM framework with equipment, such as robots and processing tools are controlled
the state-of-the-art literature in Section III-B. by their corresponding automatic control systems. Therefore, it
is crucial to meet the real-time requirement for the device layer
in an AIaCM system. To achieve this goal, ML algorithms can
A. AI-Assisted Customized Manufacturing Factory be implemented at the device layer in low power devices such
Different frameworks have been presented towards the as FPGAs. The interconnection of the physical devices, e.g.,
increased interactivity and resource management [49]–[51]. machines, conveyors, is implemented at the device layer [58],
Most studies have focused on information communica- [59] using edge computing servers.
tions [52] or big data processing [53]–[55]. So far, research 2) Smart interaction: links the device layer, AI layer, and
proposing generic AI-based CM frameworks is limited. Sys- services layer [60], [61]. It represents a bridge between differ-
tem performance metrics, e.g., flexibility, efficiency, scalabil- ent layers of the proposed architecture. The smart interaction
ity, and sustainability, can be improved by adopting AI tech- layer is composed of two vital modules. The first module
nologies such as ML, knowledge graphs, and human-computer includes basic network devices such as access points, switches,
interaction (HCI). This is especially true in sensing, inter- routers and network controllers, which are generally supported
action, resource optimization, operations, and maintenance by different network operating systems, or equipped with dif-
in a smart CM factory [56], [57]. Since cloud computing, ferent network functions. The basic network devices constitute
edge computing, and local computing paradigms have their the core of the network layer [62], [63]. Different from the first
unique strengths and limitations, they should be integrated module which is fixed or static, the second module consists
to maximize their effectiveness. At the same time, the cor- of the dynamic elements, including network/communications
responding AI algorithms should be redesigned to match the protocols, information interaction, and data persistent or tran-
corresponding computing paradigm. Cloud intelligence is re- sient storage. These dynamic elements are essentially infor-
sponsible for making comprehensive, time-insensitive analysis mation carriers to connect different manufacturing processes.
and decisions, while the edge and local node intelligence The dynamic module is running on top of the static one.
are applicable to the context or time-aware environments. In- AI is utilized in the prediction of wireless channels, op-
telligent manufacturing systems include smart manufacturing timization of mobile network handoffs, and control network
PROCEEDINGS OF THE IEEE, VOL. XX, NO. X, FEBRUARY 2020 5
TABLE I. Summary of most relevant state-of-the-art literature
Refs. Smart devices Smart interaction AI Smart services Pros Cons Applications
[49] X × X X Integration sensor with No edge computing Machine status monitor-
cloud services considered ing (primitive ML meth-
ods were used)
[50] X X X X Service-oriented smart No edge computing Milk production from
manufacturing considered buffalo pasture
[51] X X × × Integration CPS with No edge computing and Several cases from prod-
smart manufacturing in-depth analysis of AI uct design to manufac-
algorithms turing control
[52] X × × X Comprehensive consid- No AI as well as edge No specific application
eration of the entire in- computing considered
dustrial network
[54] X × X X Integration sensor with No edge computing Light gauge steel pro-
cloud services considered duction line
[55] X X X × Integration CPS with No edge computing Production line and fac-
AI considered tory management
[56] × × X × Diverse AI algorithms No consideration Cold spray additive
were used of smart devices, manufacturing,
interactions and augmented reality-
services guided inspection and
surface stress estimation
congestion. Recurrent Neural Networks (RNN) or Reservoir the critical issues such as the edge computing paradigm and
Computing (RC) are candidate solutions due to the advantages advanced AI technologies.
of them in analyzing temporal network data. In contrast, our AIaCM framework includes all the aspects
3) The AI layer: includes algorithms running at different in CM, including smart devices, smart interaction, AI tech-
computing platforms such as edge or cloud servers [54], nologies, and smart services. Meanwhile, our AIaCM frame-
[64]. The computing environment consists of cloud and edge work also considers the advent of edge computing, software-
computing servers running MapReduce, Hadoop, and Spark. defined networks, and advanced AI technologies. Moreover,
AI algorithms are adopted at different levels of computing we also present a full-fledged prototype to further demonstrate
paradigms in the AIaCM architecture. For instance, training a the effectiveness of the proposed framework (please refer to
deep learning model for image processing can be conducted Section VII for more details). The implementation details of
in the cloud. Then, edge computing servers are responsible for the AIaCM architecture are discussed next.
running the trained DL model and executing relatively simple
algorithms for specific manufacturing tasks.
4) Smart manufacturing services: include data visualiza- IV. I NTELLIGENT M ANUFACTURING D EVICES
tion, system maintenance, predictions, and market analysis. A. Edge computing-assisted intelligent agent construction
For example, a recommender system can provide customers
with details of CM products, and the information including the In the customized production paradigm, manufacturing de-
performance of a production line, market trends, and efficiency vices should be capable of rapid restructuring and reuse for
of the supply chain. small batches of personalized products [65], [66]. However,
it is challenging to achieve elastic and rapid control over the
massive manufacturing devices. The agent-based system was
B. Overview on state-of-the-art manufacturing methods considered a solution to this challenge [67], [68]. Agents can
Recently, substantial research efforts have been made to im- autonomously and continuously function in a collaborative
prove the interactivity and elasticity of exiting manufacturing system [69]. A multi-agent system can be constructed to
factories [49]–[57]. Table I summarizes most relevant state- take autonomous actions. Different types of agents have been
of-the-art literature. We can observe from Table I that most of constructed in [70]–[72].
the references only concentrate on a single or several aspects Although a single agent may have sensing, computing, and
in CM. For example, the work [49] presents a cloud man- reasoning capabilities, it alone can only accomplish relatively
ufacturing framework to analyze and process manufacturing simple tasks. Smart manufacturing may involve complex tasks,
data. Similarly, a cloud-based manufacturing equipment [54] is for instance, the image-based personalized product recogni-
proposed to provide users with on-demand services. However, tion, expected from the emerging multi-agent systems [73],
outsourcing manufacturing data to cloud services providers [74]. However, the multiple agents are deficient in processing
who are often owned by third parties can also bring the risks massive data. Recent advances in edge computing can meet
of leaking customers’ private data and exposing confidential this emerging need [75]–[77]. As shown in Fig. 3, a variety
manufacturing data (e.g., product design models). Despite of decentralized manufacturing agents are connected to edge
most of the aspects being considered, the work [50] ignores computing servers via high-speed industrial networks. The
PROCEEDINGS OF THE IEEE, VOL. XX, NO. X, FEBRUARY 2020 6
Motor
AI layer is-a
PLC is-a Conveying is-a Function is-a Computing
AI library
Data storage Computing facilities
Edge computing agent is-a
is-a is-a
Conveying_unit
Intelligent
Edge layer Warehouse
Cutting is-a ID_device 2
is-a is-a
is-a is-a
Edge computing node Agent Warehouse Working is-a
Management quality
Current
System ID_device 1
Agent is-a state
Neighbor
Working is-a
Agent layer is-a
quality
Current
Neighbor
state
Camera Motor
Conveyor belt Fig. 4. Manufacturing resources from the device function perspective.
The CM resources of a product can be mapped into computing,
cutting, conveying, and other functions.
Device layer
AGV
Robot Forklift
3D printer Robot arm
factory [78]. The above procedure can also optimize the
Fig. 3. Edge computing-assisted manufacturing devices. This archi- production line with better logistics while ensuring flexibility
tecture includes the device layer, agent layer, edge computing layer, and manufacturing efficiency.
and AI layer.
B. Manufacturing resource description based on ontology
edge computing assisted manufacturing agents embrace the Intelligent manufacturing will be greatly beneficial to the in-
device layer, agent layer, edge computing layer, and AI layer. tegration of distributed competitive resources (e.g., manpower
An agent is equipped with a reasoning module and a and diverse automated technologies), so that resource shar-
knowledge base, offering basic AI functionalities such as ing between enterprises and flexibility to respond to market
inferencing and computing. Moreover, with the support of changes are possible (i.e., CM). Therefore, in smart manu-
new communication technologies (e.g., 5G mobile networks facturing, it is imperative to realize dynamic configurations of
and high-speed industrial wired networks), all agents and edge manufacturing resources [79], [80]. CM can optimize lead time
computing servers can be interconnected. and manufacturing quality under various real-world constraints
Agents run on edge computing servers to guarantee low- of dynamic nature (resource and manpower limitations, market
latency services for data analytics. The agent edge servers are demand, etc.).
connected by high-speed industrial IoT to achieve low latency. There are several strategies in describing manufacturing
Generally, edge computing servers support a variety of AI resources, such as databases, object-oriented method [81], and
applications. the unified manufacturing resource model [82]. In contrast to
An example of such a system is a personalized product the conventional resource description methods, the ontology-
identification based on deep learning image recognition. First, based description is one of the most prominent methods. An
a multiple agent subsystem is constructed for producing per- ontology represents an explicit specification of a conceptual
sonalized products. Then, a single agent records image or model [83], by way of a classical symbolic AI reasoning
video data at different stages of the CM process. Next, the method (i.e., an expert system). Modeling an application
edge computing server runs the image recognition algorithms, domain knowledge through an expert system provides a con-
such as a convolutional neural network (CNN), R-CNN, Fast ceptual hierarchy that supports system integration and inter-
R-CNN, Faster R-CNN, YOLO, or Single Shot Detection operability via an interpretable way [84], [85].
(SSD), all of which have demonstrated their advantages in In our previous work [86], the device resources of smart
computer vision tasks. The identification results are rapidly manufacturing were integrated by the ontology-based inte-
transferred to the devices. When the single edge computing gration framework, to describe the intelligent manufacturing
server cannot meet the real-time requirements, the multiple resources. The architecture consisted of four layers, namely,
agent edge servers may work collaboratively to complete the the data layer, the rule layer, the knowledge layer, and the
specific tasks such as product identification. Indeed, during resource layer. The resource layer represented the entity of
the process, the master-slave or auction mode can be adopted intelligent manufacturing equipment (e.g., manipulators, con-
for coordination, according to the status analysis of each edge veyor belt, PLC), which was essentially the field device.
server. The knowledge layer was essentially the information model
Additionally, with the help of edge computing, it is possible composed of intelligent devices, which was integrated into the
to establish a quantitative energy-aware model with a multi- domain knowledge base through the OWL language [87]. The
agent system for load balancing, collaborative processing rule layer was used to gather the intelligent characteristics of
of complex tasks, and scheduling optimization in a smart intelligent equipment, such as decision-making and reasoning.
PROCEEDINGS OF THE IEEE, VOL. XX, NO. X, FEBRUARY 2020 7
The data layer included a distributed database for real-time Wireless link Information extraction process
Wired link
data storing, and the relational database was used to associate Edge computing node Data Information knowledge
the real-time data.
Due to the massive amount of data generated from man- AI
ufacturing devices, it is nearly impossible to consider all Computing facilities
the manufacturing device resources. Thus, it is important to
construct a new manufacturing description model to realize
Sensors
the reconfiguration of various manufacturing resources. In this Sensors
Sensors
Sensors
model, the resources can be easily adjusted by running the
model. Therefore, ontology modeling is conducted on a device
and related attributes of an intelligent production line in CM.
The manufacturing resources are mapped to different functions
Sensors
with different attributes. For instance, the time constraint of
a product manufacturing is divided into a number of time
slots with consideration of features of processes and devices.
Then, the CM resources of a product can be mapped into Fig. 5. Intelligent sensing based on edge AI computing. Sensor nodes
collect ambient data while edge computing notes can preprocess and
computing, cutting, conveying, and other functions with the cache the collected data, which can be further transferred to remote
limited time slot, as shown in Fig. 4. Next, a customized cloud servers for in-depth data analysis.
product can be produced by different devices with different
time constraints. Accordingly, a product can be represented
by ontology functions. Especially, the sensing parameters can be adjusted in a flex-
Meanwhile, after making a reasonable arrangement of dif- ible monitoring subsystem in the manufacturing environment,
ferent manufacturing functions at different time slots, a DL according to different application requirements and the task
algorithm can forecast time slots of working states. The time priority. To achieve a rapid response high priority system, the
slots of working states are important for the reconfiguration of edge AI servers should have access to the sensing data, and
manufacturing resources. Therefore, in actual applications, a capability to categorize the status of the CM environment.
different attribution of a device and customized products can This can be done by processing the data features through ML
be employed as a constraint condition. classification algorithms such as logistic regression, SVM, and
classification trees. When the data is out of the safety range, a
C. Edge Computing in Intelligent Sensing certain risk may exist in the manufacturing environment. For
The concept of ubiquitous intelligent sensing is a corner- instance, if an anomalous temperature event would happen in
stone of smart manufacturing in the Industry 4.0 framework. the CM area, the edge server could drive the affected nodes to
Numerous research studies have been conducted in monitoring increase their temperature sensing frequency, in order to obtain
manufacturing environments [88]–[90]. Most published results more environmental details and to make proactive forecasts
adopt a precondition-sensing system that only accepts a static and decisions.
sensing parameter. Obviously, this results in inflexibility and The environmental sensing data delivery is another im-
the sensing parameters are difficult to be adjusted to fulfill portant component in CM. With the development of smart
different requirements. Second, although some studies claim manufacturing, a sensing node not only performs sensing but
dynamic parameter tuning, the absence of a prediction function also transmits the data. With the proliferation of massive
is still an issue. Existing environment sensing (monitoring) sensing data, sensor nodes have been facing more challenges
cannot adjust the sensing parameters in advance to achieve a from the perspectives of data volume and data heterogeneity.
more intelligent manufacturing response. In order to collect environment data effectively, it is needed
As shown in Fig. 5, the manufacturing environment intel- to introduce new AI technologies. The sensor nodes can
ligent sensing based on the edge AI computing framework realize intelligent routing and communications by adjusting
includes two components: sensors nodes and edge computing the network parameters, assigning different network loads and
nodes [91]. Generally, smart sensor nodes are equipped with priorities to different types of data packets. With this optimized
different sensors, processors, and storage and communication sensing transfer strategy, the AI methods can make adequate
modules. The sensors are responsible for converting the phys- forecasts with reduced bandwidth usage.
ical status of the manufacturing environment into digital sig- Discussion. We present intelligent manufacturing devices
nals, and the communication module delivers the sensing data from edge computing-assisted intelligent agent construction,
to the edge server or remote data centers. The edge computing manufacturing resource description based on ontology, and
servers (nodes) include the stronger processing units, larger edge computing in intelligent sensing. It is a challenge to
memories, and storage space. These servers are connected upgrade the existing manufacturing devices to improve the
to different sensors nodes and deployed in approximation interoperability and the inter-connectivity. Retrofitting instead
to the devices, with the provision of the data storage and of replacing all the legacy machines may be an alternative
smart computing services by running different AI algorithms. strategy in this regard. The legacy manufacturing equipment
Meanwhile, the edge computing servers are interconnected can be connected to the Internet by additively mounting sen-
with each other to exchange information and knowledge. sors or IoT nodes in approximation to existing manufacturingPROCEEDINGS OF THE IEEE, VOL. XX, NO. X, FEBRUARY 2020 8
routers, switches). Consequently, industrial networks cannot
Coordination node
AI models be adapted to dynamic and elastic network environments,
especially in customized manufacturing. The software-defined
Computing center
networking (SDN) technology can separate the conventional
network into the data plane and the control plane. In this
manner, SDN can achieve flexible and efficient network
AI models
control for industrial networks. It has been reported that a
software-defined industrial network can increase the flexibility
SDN controller of a dynamical network system while decreasing the cost of
constructing a new network infrastructure [99].
SDN router
The introduction of AI technologies to SDN can further
bestow network nodes with intelligence. As demonstrated in
Control flow
Data center Fig. 6, AI technologies are introduced into traditional SDN
Data flow so as to form a novel software-defined industrial network
Fig. 6. Software-defined industrial networks consist of network (SDIN). The proposed SDIN contains a number of mapping
coordinated nodes, SDN routers, SDN controllers, data centers, and network nodes, SDIN related devices, data centers, and cloud
cloud computing servers which can support intensive computing tasks computing servers to support intensive computing tasks of
of AI algorithms. AI algorithms. Manufacturing devices are connected by their
communication modules, and they are mapped to different
network terminal nodes. On the SDIN level, key devices
devices [92], [93]. Moreover, monitors can be attached to
such as coordinated nodes and SDIN controllers construct the
existing machinery to visualize the monitoring process. It is
SDIN layer. First of all, coordinated nodes are linked with the
worth mentioning that retrofitting strategies may apply for the
ordinary nodes, and deliver network control messages from
sensing or monitoring scenarios while they are not suitable or
other SDN devices. Second, the SDN routers are the key
less suitable for the cases requiring to make active actions
devices that realize the separation of data flow and control
(like control or movement). Furthermore, a comprehensive
flow of the entire manufacturing network. In addition, the
plan should be made in advance rather than arbitrarily adding
SDIN controller is directly connected to the AI server, and
sensors to the existing production line [94]. Retrofitting strate-
the AI server provides network decisions directly to the SDN
gies also have their limitations, such as a limited number of
controller.
internal physical quantities can be monitored in a retrofitted
In the network information process, AI algorithms, such as
asset with respect to a newly-designed smart machine.
deep neural networks, reinforcement learning, SVM, and other
ML algorithms can be executed in a server according to the
V. I NTELLIGENT I NFORMATION I NTERACTION I N A
state of the network devices, such as load information, commu-
S MART FACTORY
nication rate, received signal strength indicator, and other data.
In the CM domain, the information exchange system needs Then, the AI server returns the optimized results to the SDN
to fulfill the dynamic adjustment of network resources so controller, and the results are divided into different instructions
as to produce multiple customized products in parallel. In for different network devices in the light of a specific CM task.
order to obtain optimal strategies, many studies have focused Following the above steps, the SDN controllers send a set of
on this topic, and proposed insightful algorithms as well as instructions to the routers and the coordination nodes. Finally,
strategies [95]. However, there are still two open issues: a network terminals readjust the related parameters, (e.g., com-
network framework to dynamically adjust network resources, munication bandwidth, transmitted powers) to complete the
and the end-to-end (E2E) data delivery. In this section, we data communication process.
present software-defined industrial networks and AI-assisted Intelligent optimization algorithms (e.g., ant colony or
E2E communication to tackle these two challenges. particle swarm optimization) can find optimal data transfer
strategies – based on the network parameters provided by
A. Software-defined industrial networks the SDIN, or given by the constraints of data interaction.
Industrial networks are a crucial component in CM, and cus- These algorithms can adjust the latency and energy consump-
tomized product manufacturing groups can be understood as tion requirements. Thus SDIN can improve the information
subnets. Via an industrial network (consisting of base stations, management processes within a CM industry framework,
access points, network gateways, network switches, network reducing the cost of dynamically adjusting or reconfiguring
routers, and terminals), the CM equipment and devices are network resources. Moreover, it can improve and propel the
closely interconnected with each other and can be supported whole manufacturing intelligence. Additionally, by adopting
by edge or cloud computing paradigms [96]. Taking full an AI-assisted SDIN, the production efficiency can be further
advantage of AI-driven software-defined industrial networks, improved.
and relevant networking technologies is an important method
to achieve intelligent information sharing in CM [97], [98]. B. End-to-End communication
In conventional industrial networks, network control func- End-to-end (E2E) or device-to-device communication be-
tions have been fixed at network nodes (e.g., gateways, tween manufacturing entities is a convenient communicationPROCEEDINGS OF THE IEEE, VOL. XX, NO. X, FEBRUARY 2020 9 strategy in industrial networks [100], [101]. E2E communica- by the AI-based method (e.g., naı̈ve Bayes). Next, an improved tion provides communication services with lower latency and hybrid MAC is constructed on top of the CSMA and TDMA. higher reliability, as compared to a centralized approach [102]. TDMA and CSMA schemes deal with the periodic and aperi- With effective information interaction via E2E communication, odic data flows of the E2E communications. The size of this the entire system can achieve full connectivity. In the context proposed mechanism can be adjusted in accordance with the of CM, data transmission with different real-time constraints AI-optimized results of a real application. has become a critical requirement [103]. The E2E industrial The network routing is also another key component of communication approach optimizes the usage of network E2E communications. The key node of the routing path resources (e.g., network access and bandwidth allocation) plays an important role in the E2E communications as well. through data communication of varying latency [104], [105]. However, the performances of routing key nodes are impacted Meanwhile, in order to realize the E2E communication in the by the workload; for instance, the amount of forwarded data. industrial domain, a hybrid E2E communication network – Similarly, AI plays a significant role in the routing layer. The based on the AI technology and SDIN – is here constructed predicted state parameters, such as communication rate and by exploiting different media, communication protocols, and network loads of key nodes, can be obtained by using historical strategies. The hybrid E2E-based communication mechanism data from the network node status by algorithms, such as deep with the AI assistance can be divided into three layers: the neural networks or deep reinforcement learning (e.g. deep Q- physical layer, the media access controlling (MAC) layer, and learning). the routing layer. In the physical layer, according to the advantages and VI. F LEXIBLE M ANUFACTURING L INE disadvantages of the involved communication technologies, A flexible manufacturing production line realizes customiza- different communication media include optical fiber [106], tion. AI-driven production line strategies and technologies, network cable [107], and wireless radio [108]. Generally, in- such as collective intelligence, autonomous intelligence, and dustrial communications can be divided into wired or wireless cross-media reasoning intelligence, have accelerated the global communications. On the one hand, wired communications typ- manufacturing process. Therefore, the subjects of cooperative ically exhibit high-stability and low-latency. A representative operation between multiple agents, dynamic reconfiguration case is an industrial Ethernet, which is based on a common of manufacturing, and self-organizing scheduling based on Ethernet and runs improved Ethernet protocols, such as Ether- production tasks are presented in this section. CAT [109], EtherNet/IP [110], and Powerlink [111]. On the other hand, wireless networks have been adopted in applica- tions with relatively high flexibility [112], [113]. Nowadays, A. Cooperative multiple agents an increasing number of mobile elements have been incor- Cooperation among multiple agents is necessary to dynam- porated in manufacturing systems; therefore, wireless media ically construct collaborative groups for the completion of has been widely exploited in mobile communications [114]. customized production tasks [118]. As discussed in Section IV, Conventional strategies on fixed and static industrial networks multiple agents with edge computing provide a better option may not fulfill the emerging requirements on flexible network than a single device to build a collaborative operation to realize configurations. The AI and related technologies, such as deep CM [78], [119]. Therefore, by combining the edge computing- reinforcement learning, optimization theory and game theory, assisted intelligent agents and different AI algorithms, a novel can play significant roles in improving the communication cooperative operation can be constructed as shown in Fig. 7. efficiency in the physical layer, e.g., determining the optimal The strategy of cooperative operation by multiples agents can communication between wired and wireless networks while be divided into the order of submission, task decomposition, achieving a good balance between network operational cost cooperative group, and subgroup assignment. and network performance. The working process of a flexible manufacturing produc- In the MAC layer, different devices have different require- tion line can be described as follows. First, according to ments for E2E communications according to their specific the customers’ requirements, the CM product orders are is- functions. Although many different MAC protocols have been sued to the manufacturing system through the recommender proposed (e.g., CSMA–Carrier Sense Multiple Access) [115], system. After receiving the product orders, the AI-assisted CDMA–Code Division Multiple Access) [116], TDMA–Time task decomposition algorithms take the product orders as the Division Multiple Access) [117] and their improved versions, input, the device working procedure as the output, and the these methods still lack flexibility, and do not fulfill the product manufacturing time as a constraint; these algorithms emerging requirements of industrial applications. Generally, are mainly executed at the remote cloud server. A product industrial E2E communications can be divided into two cat- order can be divided into multiple subtasks, which are sent to egories: periodic communications and aperiodic communica- all the agents via the industrial network. After the negotiation, tions. Similarly, AI plays an important role in the MAC layer. agents return the answers to the edge server, which handles the An example is a hybrid approach that combines the CSMA and working subtasks according to corresponding conditions and TDMA, with an intelligent optimization method, to improve constraints. Next, the AI-assisted cost-evaluation algorithm the efficiency of the E2E communication. In particular, the two calculates the cost of a producing group (i.e., cooperative categories of communication requirements (high and low real- manufacturing group) from the historical data. Then, the edge time or periodic and aperiodic communications) are classified agents intelligently select suitable device agents to finish the
PROCEEDINGS OF THE IEEE, VOL. XX, NO. X, FEBRUARY 2020 10
Working step
Product Edge agent
Users order
tep
i n gs g
rk in
Wo merg Fine group
Management
Computing server Wo
rk
me ing AI methods
Product task rgi ste
ng p
Customers
Process libary
Process analysis Edge agent Fine group
Order submmission Task decomposition Main cooperative group Sub cooperative group
Fig. 7. Cooperative multiple agents. The strategy of cooperative multiple agents can be divided into i) the order of submission, ii) task
decomposition, iii) cooperative group and iv) subgroup assignment.
product order after considering the whole cooperative group B. Dynamic reconfiguration of manufacturing Systems
performances, such as producing time and product quality.
With the scientific development of the industrial market
Moreover, the edge agents send the selection result to the
and manufacturing equipment, different industrial devices
device agents, which are chosen to take part in the producing
present different performance requirements representing mul-
order. The main cooperative group is constructed based on the
tiple function trends [121]. For instance, the latest Computer
working steps.
Numerical Control (CNC) machine tool can complete a wide
The main cooperative group may not be well suited for real range of tasks, from lathing to milling functions. On the
applications, especially for complicated CM tasks. Therefore, other hand, a dedicated manufacturing line does not meet
an AI-based method for constructing a suitable-size coop- new industrial requirements, especially for customized produc-
erative subgroup is an important step for dealing with the tion [122]. The trend today is towards reconfiguration and re-
mentioned problem. A possible strategy is to use cognitive ap- programmability of manufacturing processes [123]. Although
proaches such as the Adaptive Control of Thought—Rational several studies have investigated the problem and presented
(ACT-R) model [120]. These subtasks cooperative groups can meaningful results [124], [125], most of them lack intelligent
be mapped to the digital space (i.e., edge agent) and form design to fulfill the emerging requirements of dynamic recon-
even lower level subgroups, all interconnected by the conveyor, figuration of manufacturing systems, especially for customized
logistics systems, and industrial communication systems. Each manufacturing. In particular, the work [124] focuses on the
subgroup can delegate the same edge agent, to provide the communications between agents while [125] investigates the
management and customers with manufacturing services. The relationship between manufacturing flexibility and demands.
characteristics of the subgroups are partly derived from the Thus, AI technologies have seldom been adopted in these
process constraints and the physical constraints of the plant. studies. At present, ontology (as shown in Section IV) of-
In principle, the higher the constrains the deeper the task tree fers insights into dynamic reconfiguration of manufacturing
will expand, from more abstract tasks to particular atomic resources [102], [126].
targets achievable by the present devices. This structure can A schematic of the dynamic reconfiguration process based
be replicated with a probabilistic graphical model or with a on the ontology inference is shown in Fig. 8. Each customized
fuzzy tree. product invokes several processing procedures. First, a person-
After all the agents have been assigned with subtasks, alized product manufacturing-related device (such as cutting,
they form two level-cooperative groups. The formation of materials handling device) is selected by ontology reasoning
these cooperative groups is beneficial to resource manage- based on the device function. Then, the second selection of the
ment. Then, according to the manufacturing task attributes, devices involved in the manufacturing is finished according
multiple agents complete the producing task. During this to ontology results with respect to the related manufacturing
period, the corresponding device agents send their status data process, the manufacturing time, manufacturing quality, and
to edge servers timely, and the manufacturing process can other parameters of a device. Finally, a CM production line
be monitored by analyzing these data in the entire system. is constructed. Specifically, when the production line receives
In contrast to the AI-driven cooperative operation between a production task, the raw material for a specific type of
multiple agents, conventional methods often rely on human products is delivered from an autonomous warehouse. Then,
operators who participate in the whole process or computer- the production line completes the manufacturing tasks in the
assisted operators also requiring human interventions. These process sequence. Furthermore, when one of the manufac-
methods inevitably result in huge operational expenditure. turing devices breaks down during the process, automaticPROCEEDINGS OF THE IEEE, VOL. XX, NO. X, FEBRUARY 2020 11
Feature 1 Algorithm 1
Computing AI
AGV Machine Monitor Machine Robot Robot Task set plaform methods
Knowledge abstraction 1 Feature Algorithm
Function Function Function Function Function Function
Production Task1 Task2 Task3 Task4
Device 2 Device Device Device Device Device tasks
Re
Re
3
pla
Period slots Slot Slot Slot Slot Slot Slot
pla
e c
e c
Device Device Device Device Idle Device Device
Idle Idle Busy
Idle Idle Busy Idle
Idle
Broken down
Idle
Device Device Device Device Device Idle Busy Idle Busy
Idle Busy Busy
Idle
Ontology inference Heavy load
Busy
Idle Busy Busy
Idle Busy Busy
Idle Busy
Fig. 8. Dynamic reconfiguration of manufacturing resources: 1)
device selection by ontology reasoning based on the device function;
2) CM production line is constructed; 3) automatic switching to other
Busy Busy
Idle Idle Busy
Idle Idle Busy
Idle
devices from heavy-load devices or broken devices.
Busy
Idle Idle Busy
Idle Idle Busy
Idle Idle
switching of the related machining equipment by ontology
inference is conducted. Meanwhile, the reasoning mechanism Fig. 9. Self-organization of schedules of multiple production tasks
reflects the reconstruction function of a flexible production of consists of three steps: task analysis, task decomposition and task
execution.
the production line.
The presented approach leads to optimal process planning
and functional reconstruction. Besides, it shows the strengths
of ontology modeling and reasoning. In practice, only ontology
and constraints need to be established according to the above to the manufacturing requirements and self-conditions (e.g.,
description. According to Jena syntax1 , the corresponding API manufacturing time and quality). These results are broadcasted
interface can be invoked to meet the task requirements of this to other agents, including different servers. Next, the edge
scenario. In the future, other AI algorithms are expected to be agents update the working state of the idle device agent in
integrated with ontology inference. the corresponding time slot. These procedures are repeated
until the new task steps are allocated within a certain or fixed
time. Lastly, multiple agents finish the scheduling of the new
C. Self-organizing Schedules of Multiple Production Tasks production task in a self-organization manner.
Product orders generally have stochastic and intermittent
characteristics as the arrival time of orders is usually un- Self-organization of schedules with multiple agents and
certain [127]. This may result in having to share produc- time slots can effectively complete simultaneous production
tion resources among multiple tasks. Therefore, creating self- tasks using a flexible production. Furthermore, production
organizing schedules with a time slot based on multiple agents line efficiency is improved. Consequently, all manufacturing
for multiple production tasks is paramount [71]. The mech- resources, including different devices and subsystems, are
anism of self-organizing schedules for multiple production more intelligent to finish the multiple production tasks au-
tasks can be divided into three steps: task analysis, task tonomously. In contrast, conventional methods often require
decomposition, and task execution. huge human resources in scheduling and planning production
As shown in Fig. 9, in terms of initialization, when a new tasks [128]. Despite the recent advances in computer-aid meth-
production task is processed by the multi-tasking production ods [129], they still require substantial human interventions
line, the new production tasks are divided into multiple steps and cannot meet the flexible requirements.
by an AI-based method executed at the cloud. Additionally, However, we have to admit that AI-driven self-organization
according to the process lead time, the producing period can of schedules does not get rid of humans in the loop of
be decomposed into time slots of different lengths. Moreover, the entire production process. The main goal of AI-driven
for one working step in a time slot, edge agents select all idle methods is to save unnecessary human resource consumption
device agents by comparing the mapping relationship between and mitigate other operational expenditures. In this manner,
working steps and device agent functions. This processed human workers can concentrate on planning and optimizing
time slot information is then broadcasted to all the agents the overall production procedure instead of conducting te-
simultaneously. Then, idle device agents choose the working dious and repetitive tasks. Meanwhile, an appropriate human
step by price bidding or negotiating with others, according intervention is still necessary when full automation is not
1 Jena syntax defines a set of rules, principles, and procedures to specify
achievable or is partially implemented. In this sense, AI-driven
the semantic web framework of Apache Jena (https://jena.apache.org/getting methods can also assist human workers to give intelligent
started/index.html). determinations.You can also read
