Towards a Statistical Model Checking Method for Safety-Critical Cyber-Physical System Verification

Hindawi
Security and Communication Networks
Volume 2021, Article ID 5536722, 12 pages
https://doi.org/10.1155/2021/5536722

Research Article
Towards a Statistical Model Checking Method for Safety-Critical
Cyber-Physical System Verification

 Jian Xie ,1,2,3 Wenan Tan,1,2,3 Bingwu Fang ,2,4 and Zhiqiu Huang1,2,3
 1
 College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, China
 2
 Key Laboratory of Safety-Critical Software, Nanjing University of Aeronautics and Astronautics, Nanjing, China
 3
 Collaborative Innovation Center of Novel Software Technology and Industrialization, Nanjing, China
 4
 College of Information Engineering, Anhui Finance and Trade Vocational College, Hefei, China

 Correspondence should be addressed to Bingwu Fang; bingwufang@163.com

 Received 10 February 2021; Revised 2 April 2021; Accepted 16 April 2021; Published 18 May 2021

 Academic Editor: Weizhi Meng

 Copyright © 2021 Jian Xie et al. This is an open access article distributed under the Creative Commons Attribution License, which
 permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

 Safety-Critical Cyber-Physical System (SCCPS) refers to the system that if the system fails or its key functions fail, it will cause
 casualties, property damage, environmental damage, and other catastrophic consequences. Therefore, it is vital to verify the safety
 of safety critical systems. In the community, the SCCPS safety verification mainly relies on the statistical model checking
 methodology, but for SCCPS with extremely high safety requirements, the statistical model checking method is difficult/infeasible
 to sample the extremely small probability event since the probability of the system violating the safety is very low (rare property).
 In response to this problem, we propose a new method of statistical model checking for high-safety SCCPS. Firstly, with the
 CTMC-approximated SCCPS path probability space model, it leverages the maximum likelihood estimation method to learn the
 parameters of CTMC. Then, the embedded DTMC can be derived from CTMC, and a cross-entropy optimization model based on
 DTMC can be constructed. Finally, we propose an algorithm of iteratively learning the optimal importance sampling distribution
 on the discrete path space and an algorithm to check the statistical model of verifying the rare attribute. Eventually, experimental
 results show that the method proposed in this paper can effectively verify the rare attributes of SCCPS. Under the same sample
 size, comparing with the heuristic importance sampling methods, the estimated value of this method can be better distributed
 around the mean value, and the related standard deviation and relative error are reduced by more than an order of magnitude.

1. Introduction continuously changing behavior in their physical layer is
 intertwined with the discrete changing behavior in their
Safety-Critical Cyber-Physical System (SCCPS) is charac- decision control layer. Their state spaces are infinite as well.
terized with high safety and high reliability and are widely It increases the difficulty and brings severe challenges to the
used in fields closely related to the national economy and safety analysis and verification of SCCPS. However, the
people’s livelihoods, such as aerospace, nuclear industry, traditional model checking has the problem of state space
public transportation, finance, and medical care. Once the explosion, and it is difficult to effectively verify it [11].
execution of such system fails, it will deeply threaten the safety With the execution path of the sampling system, Sta-
of human’s life and property [1–3]. Therefore, it is vital to tistical Model Checking (SMC) uses statistical analysis
analyze and verify the safety and reliability of safety-critical techniques to approximate the probability that the target
systems, and it is of great significance to the design and system meets the sequential logic attributes and can provide
development of safety-critical systems. Indeed, it has attracted arbitrarily small error limits [12–14]. Because SMC does not
wide attention from researchers and has extensively grown as need to analyze the complex logic inside the target system to
a prominent research topic in the community [4–7]. verify the timing logic properties of the system, it can ef-
 Essentially, SCCPS is a kind of complex cyber-physical fectively avoid the complexity of the system and the ex-
fusion system [8–10]. For this kind of systems, the plosion of the state space [15, 16]. Therefore, SMC is the

2 Security and Communication Networks

most effective solution to verify the timing properties of (i) The probability that M satisfies the attribute φ:
complex SCCPS [12, 17–19]. However, for SCCPS requiring Pr(M⊨π)
extremely high safety, the probability of occurrence of the (ii) Whether the probability of M satisfying the attribute
negative events of its safety attributes and the probability of φ is higher than or equal to the threshold θ:
system failures are extremely low. It is infeasible for SMC to M⊨Pr(≥θ) (φ)
sample extremely low probability events. Thus, how to use
SMC to verify the extremely secure SCCPS is an urgent In SMC, it first simulates the execution of the system
problem to be solved [20, 21]. model M to extract a random execution path ω. Then, the
 To date, verification of the SMC rare attributes mainly BLTL model detector is used to determine whether ω sat-
relies on the importance sampling method. For CTMC and isfies the attribute φ, and a certain number of samples will be
DTMC random models, Reijsbergen et al. [22] and Barbot generated after multiple simulations. It further leverages the
et al. [23] utilized the heuristic methods to obtain an statistical method to perform statistical analysis on the
importance sampling distribution to complete the attribute samples to assess the probability of the system model M
verification of the two models, respectively. Clarke and satisfying the attribute φ, as well as give the confidence
Zuliani [24] proposed the cross-entropy minimization interval or the estimated error margin. Let I(ω) represent the
importance sampling-based SMC method to verify the output result of the BLTL model detector. If ω⊨π, I(ω) � 1;
safety properties of the Stateflow/Simulink model system. otherwise, it is 0. I(ω) is a Bernoulli random variable, so the
Zuliani et al. [17] used the SMC method in his study [24] to behavior of M can be modeled by the Bernoulli distribution
verify the secure attribute of the discrete-time SHS. The with a parameter p:
methods proposed by Clarke and Zuliani assume that the Pr(I(ω) � 1) � p,
distribution of the system path space is an exponential (1)
distribution. By simply increasing the failure rate of the Pr(I(ω) � 0) � 1 − p.
system parameters, several paths that satisfy the rare at-
 The parameter p represents the probability that the
tributes are extracted at one time to calculate the optimal
 model M satisfies the BLTL attribute φ. With the Bernoulli
parameters for the exponential distribution to obtain an
 distribution, we note that p � E[I(ω)], var[I(ω)] � p ×
importance sampling distribution [25]. J´egourel et al. [26]
 (12212p). Since the value of p is unknown, the goal of SMC is
leveraged the cross-entropy minimum optimization
 to estimate the value of p.
method in the random model of a random guardian
 SMC can be divided into two categories: hypothesis
command system, which can approximate the path dis-
 testing and parameter estimation. The hypothesis testing is
tribution of the system by increasing the number of
 used to determine whether the probability of the system
commands (number of parameters), to obtain an impor-
 satisfying the temporal logic attribute is greater than or equal
tance sampling distribution in the random model. How-
 to a given threshold, which is a qualitative result, while the
ever, the optimal importance sampling distribution
 parameter estimation is a quantitative result to represent the
obtained with the aforementioned methods is not from the
 approximate probability of the system satisfying the tem-
distribution family of the system path space, but essentially
 poral logic attribute. SMC qualitative algorithms include the
is a heuristic importance sampling method. Thus, the
 single sampling plan (SSP) algorithm [27], the sequential
verification results are only rough approximation.
 probability ratio test (SPRT) algorithm [27], and the
 In this paper, we propose a method with the SCCPS path
 Bayesian hypothesis test (BHT) algorithm [18]. SMC
space to construct a cross-entropy optimization model and
 quantitative algorithms mainly include the approximate
use an iterative learning method to obtain an optimal im-
 probabilistic model checking (APMC) [28] algorithm and
portance sampling distribution from the parameterized
 the Bayesian interval estimation testing (BIET) algorithm
distribution cluster of the path space. It can ensure that the
 [18]. Kim et al. [29] conducted an empirical evaluation on
optimal importance sampling distribution is from the spatial
 the performance and applicability of the four algorithms
distribution family in the SCCPS path, and the iterative
 (i.e., SSP, SPRT, BHT, and BIET).
learning method can ensure that the distribution evenly
covers the unsafe path distribution area. As evaluated in our
experiments, the accuracy and efficiency of the rare attribute
verification are significantly improved. 2.2. Safety Requirement Specification. In this paper, we use
 Bounded Linear Temporal Logic (BTCL) as our specification
 language. BLTL restricts Linear Temporal Logic (LTL) with
 time bounds on the temporal operators. Formally, the syntax
2. Background of BLTL is given as
2.1. Statistical Model Checking. Statistical Model Checking φ ⩴ x ∼ v|(φ 1) ∨ φ 2|(φ 1 ∧ φ 2)|φ 1φ 1 ∪ t φ 2, (2)
(SMC) can be simply described as follows: given a system
model M and system properties φ described by the bounded where ∼∈ { ≤ , ≥ , �}, x ∈ SV (the finite set of state variables),
linear temporal logic (BLTL) [18], it uses the Monte Carlo v ∈ R, t ∈ R≥0 , and ∨, ∧, and are the usual Boolean con-
sampling, model checking, and statistical analysis techniques nectives. The formulas x ∼ v is called the atomic proposi-
to qualitatively/quantitatively verify the following two tions (AP). The formula φ{_1} ∪ t φ{_2} will return true if and
questions: only if φ{_2} is true and φ{_1} will hold within the time t. The

Security and Communication Networks 3

operators ◇t and □t can be defined as follows by using the (1) If φ is of the form x ∼ v, σ k ⊨φ if σ k ⊨φ since sk+I �
 ∪ t operator: ◇tφ � True ∪ tφ, which required φ to hold true sk+I and tk+I � tk+I by using [17] for i � 0.
within time t (true). □tφ �¬◇t¬φ requires φ to hold true up (2) If φ is of the form φ1 ∨φ2 ,
to time t.
 The semantics of BLTL formulas [28, 30, 31] is defined ⎪
 ⎧ iff σ k ⊨φ1 or σ k ⊨φ2 ,
 ⎪
 ⎪
with respect to system traces (or executions). A trace is a ⎨
 σ k ⊨φ1 ∨φ2 ⎪ iff σ k ⊨φ1 or σ k ⊨φ2 , (3)
sequence σ � (s0 , t0 ), (s1 , t1 ), . . ., where si is the state of the ⎪
 ⎪
 ⎩
system at the represented time ti . The pair (si , ti ) expresses iff σ k ⊨φ1 ∨φ2 ,
the fact that the system moved to state si+1 after having spent
ti time units in state si . If the trace σ satisfies the property φ,
we write σ⊨φ. The trace suffix of σ starting at k ∈ N is by induction hypothesis as #(φ1 ∨φ2 ) ≥ #(φ1 ) and
denoted by σ k , and σ 0 denotes the full trace σ. #(φ1 ∨φ2 ) ≥ #(φ2 ). The proof is similar to φ1 and
 The semantics of BLTL for a trace σ k is defined as φ1 ∩ φ2 .
follows: (3) If φ is of the form φ1 ∪ t φ2 , σ k ⊨φ1 ∪ t φ2 if the fol-
 lowing three conditions are satisfied:
 (i) σ k ⊨x ∼ v, iff x ∼ v holds true in state sk
 ∼
 (ii) σ k ⊨φ1 ∧φ2 , iff σ k ⊨φ1 and σ k ⊨φ2 (a′ ). 0≤l

4 Security and Communication Networks

 standard Wiener process defined in the real 3.1. SCCPS Path Space Model
 number field. It assumes that ∀l ∈ L, f(l, ·), and
 g(l, ·) are bounded and Lipschitz continuous; 3.1.1. Model Representation. To avoid the complexity of the
 dynamic evolution of SCCPS, SMC does not pay attention
 (vi) G: E ⟶ 2X is to assign a guardian condition to
 to the structure of SCCPS, but focus on sampling the
 each discrete transition, satisfying the following
 execution path of SCCPS. The behavior of SCCPS evolving
 conditions:
 over time can be characterized by the path of the system.
 ∗∗
 ∀e � (l, l′ ) ∈ E, G(e) denotes a measurable According to the execution semantics of SCCPS, the ex-
 subset of z Inv (l) ecution path generation process of SCCPS can be described
 ∗∗
 ∀l ∈ L, G(e): e � (l, l′ ) ∈ E, l′ ∈ L is a dis- as follows: in the current control mode li , the continuous
 joint subset of z Inv (l) variable xi evolves according to the SDE. When the evo-
 lution of xi satisfies the guardian condition (xi ∈ G(li , li+1 )),
 (vii) R: E × X ⟶ P(X) is a reset mapping. P(X)
 it migrates to the next control mode li+1 and the initial value
 represents a set of probability measures defined on
 of xi+1 is determined by the random reset kernel R. The
 X, and continuous variables are reset according to
 residence time of li is ti � inf t ∈ R>0 , xi (t) ∉ Inv(li ) . ti is a
 the probability distribution.
 random variable, and its value depends on the SDE of li and
 According to the definition, the SCCPS hybrid state the initial values xi (0) and Inv (li ). According to the
space is L × X, and (l, x) ∈ L × X represents the hybrid state. generation process of the SCCPS execution path, the next
The continuous dynamics of SCCPS evolves according to the state of SCCPS depends on the current state and the related
SDE in the current control mode. However, the discrete residence time of the current state. Therefore, the execution
dynamics refers to migrating one control mode to another path of the SCCPS can be regarded as that it is generated in
control mode with the guardian condition on the discrete the continuous-time Markov process in the hybrid state
transition, when the continuous variable cannot reach the space. As the residence time of li is longer, the probability of
boundary of the invariant. migration from li is higher. It can further presume that the
 Let xl (t) be the SDE solution of the initial state xl (0); residence time of li obeys the exponential distribution, and
τ(l) � inf t ∈ R>0 , xl (t) ∉ Inv(l) means that, in the control the continuous-time Markov process then becomes CTMC.
mode l, the first time that the evolution of a continuous Let Gl denote the guard condition set of all edges starting
variable violates the invariant, that is, the first time of exiting from l:
the control mode l. Gl � G(e): e � l, l′ ∈ E, l′ ∈ Loc , (4)
 SCCPS execution semantics: a random execution of where G(e) ∈z Inv (l) and G(ei ) ∩ G(ej ) � ∅, i ≠ j. In l, the
SCCPS is defined as a random process (l(t), x(t)) ∈ L × X in time for the continuous variable evolving to satisfying the
the SCCPS state space. If there is a stop-time sequence T0 � conditions of each guard is τ 1 , τ 2 , . . . , τ |Gl |. Then, the resi-
0 < T1 < T2 < · · · that makes ∀k ∈ N, where dence time in l is tl � min τ 1 , τ 2 , . . . , τ |Gl | . Supposing
 (i) (l0 , x0 ) ∈ L × X indicates the initial state of SCCPS. τ 1 , τ 2 , . . . , τ |Gl |, respectively, obey the exponential distribu-
 (ii) t ∈ (Tk , Tk+1 ), l(t) � l(Tk ) is a const, and x(t) is a tion of parameters λl,l′ , l′ ∈ L, (l, l′ ) ∈ E , then the resi-
 continuous solution of the SDE dence time tl in l obeys the exponential distribution of
 dx (t) � f(l(Tk ), x(t))dt + g(l(Tk ), x(t))dBt ; parameters l′ ∈Loc,(l,l′ )∈E λl,l′ . With this assumption, the ex-
 • Tk+1 � Tk + τ(l(Tk )); ecution path of SCCPS can be generated by the CTMC
 random process.
 • the probability distribution of x(Tk+1 ) is determined
 by the reset map R(ek , x(T−k+1 )), where
 ek � (l(Tk ), l(Tk+1 )) ∈ E and x(T−k+1 ) � limt⟶ Definition 1. SCCPS path generation model: the path
 Tk+1x(t). generation model on the SCCPS state space is defined as
 CTMC � (S, s0 , λ), where
 SCCPS path: a SCCPS execution path is defined as an
infinite sequence σ � ((l0 , x0 ), t0 ), ((l1 , x1 ), t1 ), . . . from the (i) S � L represents the discrete state set of SCCPS
initial state (l0 , x0 ), where (li , xi ) ∈ L × X represents the • s0 ∈ L denotes the initial state of SCCPS
SCCPS state. ti ∈ R≥0 means the time that transitions the
 • Migration rate function λ: S × S ⟶ R≥0 , and all
state (li , xi ) to the next state (li+1 , xi+1 ).
 migration rate function values form the migration
 rate matrix λ
3. Our Approach It can be seen from this definition that when the
 CTMC structure is known, its behavior is controlled by
In this section, we present our proposed method with the the migration rate matrix λ, whose value comes from
SCCPS path space to construct a cross-entropy optimization SCCPS. The value of λ is estimated with the maximum
model and use an iterative learning method to obtain an likelihood method according to simulating the execution
optimal importance sampling distribution from the pa- of SCCPS to obtain the time samples of the state
rameterized distribution cluster of the path space. transition.

Security and Communication Networks 5

3.1.2. Algorithm of Learning Model Parameters. The rarity of 3.2. Method of Sampling Rare Attributes. In the path space of
the path does not necessarily imply that the conversion rate the high-safety SCCPS, it is difficult to obtain samples
between two adjacent discrete states is low, and the rarity of satisfying the rare attributes, which makes the SMC infea-
the safety attributes in the path space does not necessarily sible. To address this challenge, we propose a method for
imply that the optimal parameters in the parameter space are sampling the rare attributes. It uses the cross-entropy
rare. Based on this observation, this section introduces our method to learn an optimal-importance sample distribution
approach of leveraging the maximum likelihood estimation from the path space of the SCCPS. With this sample dis-
method to estimate the migration rate of two adjacent tribution, it is easy to obtain the samples that satisfy the rare
discrete states of SCCPS and obtain the migration rate attributes. Thus, the convergence of the SMC can be
matrix λ. With the simulation operation of each discrete accelerated. The importance sampling distribution is cor-
state of SCCPS, the discrete state is sampled to migrate to the rected by the likelihood ratio weighting to ensure that the
next discrete state time; we then use the maximum likeli- SMC verification result is unbiased.
hood estimation to obtain an estimate of λ.
 For the state si ∈ S, we simulate executing the SDE in the
running state si to obtain the migration time 3.2.1. Zero-Variance Importance Sampling Distribution.
tk (k � 1, . . . , N) samples of the adjacent state sj . Assuming The basic idea of the importance sampling method [33, 34] is
that the migration time between si and sj obeys the expo- to change the probability density distribution of random
nential distribution of the parameter λij , then the likelihood variables, so as to obtain the samples of extremely small
function of λij can be obtained: probability events with a higher probability. We now present
 N
 the SMC method based on the importance sampling. Let
 L λij � λij e− λij tk
 , (5) f(ω) be the true distribution of path ω, and let g(ω) be the
 k�1 importance sampling distribution, and g(ω) can obtain the
 samples of the extremely small probability events with a
and its log likelihood function is as follows: higher probability when g(ω) ≠ 0 and f(ω) ≠ 0. In the case
 N N of verifying the extremely small probability events, it is
 ln L λij � ln λij − λij tk . (6) difficult to sample from f(ω) to meet the requirements, but
 k�1 k�1 the importance sampling method is to sample from g(ω).
 The probability p � Ef [I(ω)] satisfying the system attribute
 We further take the derivative of λij with the log-
 can be described as
likelihood function and make it equal to 0, and its estimated
value can be resolved, λ ij � (1/N) N k�1 tk . With f(ω)
 p � Ef [I(ω)] � I(ω)f(ω)dω � I(ω) g(ω)dω
E(λ ij ) � (1/N) N
 k�1 E( ) � (1/λ ), it can be seen that the
 λ ij ij g(ω)
estimated value is an unbiased estimate of λij . The estimated
variance is � I(ω)W(ω)g(ω)dω � Eg [I(ω)W(ω)],
 N N
 1 ⎠ � 1 Var t � 1 ,
 Var λ ij � Var⎛
 ⎝ t ⎞ (9)
 N k�1 k N2 j�1
 k
 Nλ2ij
 where W(ω) � (f(ω)/g(ω)) is the likelihood ratio, and
 (7) g(ω) is for the importance sampling. We leverage the
 likelihood ratio to correct the weighting to ensure that the
but the estimated variance is biased, and the variance will be
 estimated value of p is unbiased. We then randomly sample
decreased as the samples increase.
 In most cases, it is difficult to obtain a clear expression N independent execution paths ωi , i ∈ {1, . . . , N} from the
for the random execution of SCCPS. However, what the importance distribution g(ω) and obtain the unbiased
safety concerned is the accessibility analysis of discrete estimate:
states. The discrete state set S and its transitions can capture 1 N
all necessary information. Therefore, we derive the DTMC �
 p I ωi W ωi , (10)
 N i�1
from the SCCPS path generation model to represent the path
space of SCCPS. The value of DTMC’s migration probability and estimated variance
matrix P: S × S ⟶ [0, 1] can be obtained from the mi-
gration rate matrix λ of the SCCPS path generation model. 1 2 2 2
 ] �
 Varg [p E I (ω)W (ω) − p , (11)
For two states si and sj ∈ S, N g

 λij for p, respectively.
 ⎧
 ⎪
 ⎪
 ⎪
 ⎪ , s i ≠ sj , The efficiency and accuracy of importance sampling rely
 ⎨ λi
 P s i , sj � ⎪ (8) on the selection of the distribution g(ω). If the selection is
 ⎪
 ⎪ inadequate, the importance sampling method is difficult to
 ⎪
 ⎩ 1, s i � sj , effectively achieve the acceleration effect and may play a
 decelerating effect. The key problem of importance sampling
where λi � sj ∈Sλij . is to find a density function for the optimal sampling

6 Security and Communication Networks

probability to minimize the estimated variance. With for- the second term. Let D(λ) � Ω g∗ (ω)Inf(ω, λ)dω ; the
mula (10) returning 0, it can obtain the following formula: minimization problem of formula (13) is equivalent to the
 I(ω)f(ω) maximization problem of formula (14):
 g∗ (ω) � , (12)
 p maxλ g∗ (ω)ln f(ω, λ)dω � maxλ I(ω)f(ω)ln f(ω, λ)dω
 Ω Ω
where g∗ (ω) is a zero-variance importance sampling dis- � maxλ E[I(ω)ln f(ω, λ)].
tribution, which means that extracting only one sample from
the zero-variance importance sampling distribution can be (15)
used to calculate its estimated value, that is, any sample is an Solving the optimization problem of formula (14)
unbiased estimate of its mean. However, the zero-variance requires sampling from the true distribution f(ω).
importance sampling distribution depends on the true value However, in the case of rare attribute verification, it is
p, and the value of p is unknown. Therefore, it is impossible difficult to sample from f(ω) to the path sample that
to sample from g∗ (ω). This paper proposes to use the cross- satisfies the rare attribute. By using importance again, the
entropy method to find an approximate optimal importance sampling method samples from the distribution f(ω, μ)
sampling distribution closest to g∗ (ω) from the parame- and the selection of parameter μ should be able to in-
terized distribution family of the sample path space, so as to crease the probability of the path that meets the rare
reduce the SMC variance and accelerate the convergence of attribute. Therefore, the optimization problem of formula
the SMC algorithm. (14) can be re-formed as
 f(ω)
 maxλ I(ω) f(ω, μ)ln f(ω, λ)dω
3.2.2. Cross-Entropy Optimization Model. This section is to Ω f(ω, μ)
obtain the optimal importance sampling distribution by
minimizing the cross entropy between the two probability (16)
 � maxλ I(ω)W(ω, μ)f(ω, μ)ln f(ω, λ)dω .
distributions. According to the definition of cross entropy Ω
[35], this section provides the definition of cross entropy for
the SCCPS path space. � maxλ Eμ [I(ω)W(ω, μ)ln f(ω, λ)].

 Among them, the likelihood ratio function
Definition 2. Cross entropy for the SCCPS path space: the W(ω, μ) � (f(ω)/f(ω, μ)). In formula (16), the optimal
cross entropy between two probability measures f(ω) and solution of its optimization problem λ∗ can be estimated by
f′ (ω) for the SCCPS path space Ω is as follows: the path sample, and the sample mean is replaced by the
 f(ω) expectation Get the estimated value of λ∗
 CE f(ω), f′ (ω) � f(ω)ln d . (13)
 Ω f′ (ω) ω N
 1
 The cross entropy is used to assess the similarity of two λ ∗ � argmaxλ I ωi W ωi , μ ln f ωi , λ , (17)
 N i�1
probability distributions. The value of cross entropy is
smaller, and f(ω) and f′ (ω) are more similar, i.e., where ω1 , ω2 , . . . , ωN is a sample from the distribution
CE(f(ω), f′ (ω)) � 0 if and only if f(ω) � f′ (ω). f(ω, μ).
 According to Definition 2, the construction of the cross-
entropy optimization model on the SCCPS path space is
given below. Assume that the original distribution f(ω) of 3.3. Algorithm of Verifying the Cross-Entropy Safety. In
the SCCPS path ω comes from the parameterized distri- Section 3.1, we provide a DTMC-based method to ap-
bution family f(ω, θ) , The cross-entropy optimization proximate the SCCPS path space. SMC mainly considers
method is used to select a distribution f(ω, λ∗ ), λ∗ ∈ θ in the system execution path ω � s0 , s1 , . . . , sk (k > 0) within a
the parameterized distribution family, λ∗ ∈ θ and the op- bounded time T, where k is a random variable to rep-
timal distribution g∗ (ω) have the smallest cross-entropy. resent the number of state transitions, and its value varies
This optimization problem can be described for with ω. Let ⟨l, m⟩ denote two adjacent and ordered state
 g∗ (ω) pairs in ω, S(ω) represent the set of ordered state pairs in
 minλ CE g∗ (ω), f(ω, λ) � minλ g∗ (ω)ln d ω, n(ω)
 lm represent the number of transitions from state l to
 Ω f(ω, λ) ω state m in ω, and n(ω) represent the number of occur-
 l
 rences of the state l in ω; then, the probability measure
 � minλ g∗ (ω)ln g∗ (ω)dω function of path ω under system parameter p can be
 Ω
 formulated as
 − g∗ (ω)ln f(ω, λ)dω . n(ω)
 Ω f(ω, p) � ιinit s0 plm lm . (18)
 (14) ⟨l,m⟩∈S[ω]

 The first term of formula (13) has nothing to do with λ Substituting f(ωi , λ) of formulas (16) with (17), we
and minimizing cross entropy is equivalent to maximizing obtain

Security and Communication Networks 7

 1 N ⎜ (ω ) ⎟
 maxp I ωi W ωi , μ ⎛
 ⎜
 ⎝lntinit s0 + nlm i ln plm ⎞
 ⎟
 ⎠s.t. plm � 1, (19)
 N i�1 m∈S
 ⟨l,m⟩∈S(ωi )

and formula (18) can be transformed by the Lagrangian
multiplier method into the following optimization problem:

 N
 ⎜ (ω ) ⎟
 maxp I ωi W ωi , μ ⎛
 ⎜
 ⎝ln ιinit s0 + nlm i ln plm ⎞
 ⎟ ⎝ p − 1⎞
 ⎠ + ]i ⎛ lm
 ⎠, (20)
 i�1 ⟨l,m⟩∈S(ωi ) m∈S

 (j)
 (j) (j) (j) ω 
where ]i is the Lagrangian multiplier. Taking the derivative (j+1) (j) N
 i�1 I ωi W ωi , p nlm
 i

 plm � αplm +(1 − α) (j)
 ,
of formula (19) to plm and making it equal to 0, the solution (j) (j) (j) ωi 
 N
 i�1 I ωi W ωi , p nl
can be
 (ω i ) α ∈ (0, 1).
 N
 i�1 I ωi W ωi , μ nlm
 plm � , (21) (23)
 (ω i )
 N
 i�1 I ωi W ωi , μ n l
 The smoothing strategy can retain important but not yet
where ωi (1 ≤ i ≤ N) is the sample path from the distribution effective parameters. Iterative formula (21) and smoothing
f(ω, μ), and f(ωi ) represents the true probability distri- formula (22) can jointly ensure that approximately uniform
bution of the SCCPS path. sampling is obtained from the path set of events satisfying
 With formula (20), it indicates that the estimated value the minimal probability.
of the optimal solution relies on the initial distribution The selected initial distribution f(·; p(0) ) should be able
f(ω, μ). However, the distribution of f(ω, μ) is generally to produce some paths that satisfy the event with minimal
far from the optimal distribution. Therefore, in order to probability in the first iteration, that is, the selected pa-
reduce the influence of the initial distribution f(ω, μ) on rameter p(0) should be able to increase the probability of
the optimal importance sampling distribution, this paper occurrence of the extremely small probability events.
proposes the iterative solution in the path space. Through Therefore, in this paper, we set the initial parameter p(0) to a
the iteration, the algorithm can explore a wider path uniform distribution, and the uniform distribution can
space, so as to obtain a better approximate optimal quickly obtain the sample path that satisfies the extremely
solution. small probability event. The condition for stopping the it-
 Let the initial distribution parameter be u � p(0) , and an eration can be that the coefficient of variance or the distance
iterative formula can be obtained from formula (20): between two iteration parameter vectors are not higher than
 a certain constant or the maximum number of iterations. For
 (j) (j) (j) ω
 (j)
 example, given any small positive number ϵ > 0, if ‖p(j) −
 (j+1) N
 i�1 I ωi W ωi , p nlm
 i

 plm � , (22) p(j − 1)‖ < ϵ is satisfied, the iteration will be stopped. To
 (j)
 (j) (j) ω facilitate the comparison, we limit the maximum number of
 N
 i�1 I ωi W ωi , p(j) nl i
 iterations in the experiment. To sum up, Algorithm 1
where N is the number of samples per iteration, presents the description of the importance sampling dis-
 (j) (j) (j)
W(ωi , p(j) ) � (f(ωi )/f(ωi , p(j) )) represents the like- tribution learning algorithm, which iteratively solves the
 (j)
lihood ratio of the nth iteration, and ωi is the ith sample approximate optimal importance sampling distribution in
 (j) the SCCPS path space of the attributes for being verified.
path sampled from the distribution f(ωi , p(j) ).
 Usually, only a few state transitions can be seen in each Regardless of sample acquisition time and BLTL model
simulated execution. During each iteration, some param- checking time, the time complexity of Algorithm 1 is
eters do not work in the path that satisfies the extremely O(jmax |p|N). Since the optimized objective function is
small probability event. Formula (21) will set these pa- convex, there is a unique optimal solution. If Algorithm 1
rameter values to zero so that these parameters will not can converge, it must converge to the vicinity of the unique
work in all subsequent iterations. As a result, the iterative optimal solution [36]. Since the number of samples in each
algorithm converges too prematurely to detect a wider iteration is limited, the convergence is probabilistic but not
parameter space. To avoid this situation, this paper adopts a necessarily monotonic. By simply limiting the maximum
smoothing strategy to temporarily reduce the importance number of iterations jmax , the algorithm can be guaranteed
of inoperative parameters in the iteration instead of simply to be terminated with 100% probability. For the proof of
setting them to zero. The smoothing strategy is to weight convergence of cross-entropy optimization, please refer to
current iteration value and the parameters of the previous [37]; thus, a formal proof of convergence is not provided in
iteration: this paper. In experiments, we observe that the parameters

8 Security and Communication Networks

are convergent. Once the parameters converge, the last set of Aeronautics and Space Administration [39]. The two hor-
 that
simulated samples is used to estimate the probability p izontal tails on the two side of the aircraft’s fuselage are
SCCPS satisfies the safety attribute with the optimal im- controlled by two elevators, respectively. Each elevator has
portance sampling distribution. Algorithm 2 describes the two independent hydraulic actuators. In the normal oper-
verification process of the safety verification algorithm. ation process, each elevator is positioned by its corre-
 sponding external actuator, and its internal actuator can be
 used when the external actuator does not work. The two
4. Experiment and Analysis external actuators are driven by two independent hydraulic
To evaluate the effectiveness and performance of the Cross- circuits, and the two internal actuators are both connected to
Entropy Safety Verification Algorithm (CESVA) method the third hydraulic circuit. The system should ensure that
proposed in this paper, we apply CESVA to a fault-tolerant only one set of actuators (i.e., external or internal) locates the
controller for an aircraft elevator system (FTC4AE), that is, a elevator at any given time. If the external actuator or its
Stateflow/Simulink hybrid system modeling case from corresponding hydraulic circuit fails, the system will activate
MATLAB. It introduces the randomness in terms of the fault the internal actuator. If the fault still exists, the external
injection and simulates with MATLAB to obtain the system actuator will be shut down and eventually isolated. The fault
execution path. Path checking is realized by the BLTL model in the hydraulic circuit may be temporary, and if the fault is
detector of Plasma-Lab [38]. In the experiment, the rare cleared, the hydraulic circuit can always be restored to the
attributes of FTC4AE is verified with the CESVA method, online state. The control logic of the system is implemented
which is further compared with the Heuristic Importance in the form of a state flow diagram, while the hydraulic
Sampling (HIS) method [17]. actuators and elevators are modeled by using Simulink.
 According to modifying the Stateflow/Simulink model,
 we add random faults into three hydraulic circuits. Setting
4.1. Validity Measurement of Experimental Results. In the the fault model with an out-of-bounds’ reading of circuit
case of nonrare attribute verification, the confidence interval pressure, we model the fault injection as three independent
is used to assess the accuracy of various methods, while in Poisson processes. When the hydraulic circuit fails, the
the case of rare attribute verification, the relative error of circuit will stay in the fault state for one second. Then, the
sampling is used to assess the accuracy of the estimation: pressure reading will restore to its normal value, and the
 fault state will be terminated. In our experiments, the being
 ������ ���
 ]
 Var[p 1 estimated safety attribute is the probability that, within 25
 ) �
 RE(p ≈ , (24) seconds, the horizontal tails will not respond to the control
 ]
 E[p Np
 inputs in the duration of 1 second.
where E[p ] is replaced by the current estimated value p , We estimated the probability of the BLTL formula φ:
Var[p ] � (1/N − 1) N i�1 (I(σ i )W(σ i , μ, λ ∗
 ) − 
 p ) 2
 .
 Skewness is a measure of assessing the skewing direction φ � F25 G1 H1 fail ∨ H3 fail ∧ H2 fail , (26)
and degree of data distribution and is the characteristic
number that characterizes the degree of asymmetry of the where H1 and H3 represent the hydraulic circuit that drives
probability distribution density curve with respect to the the external actuator, while H2 represents the hydraulic
average. Skewness is defined as the third-order standardized circuit that drives the internal actuator.
moment of the sample, and the skewness of the normal In the experiment, the failure rate of the three hydraulic
distribution is 0, and its estimator is evenly distributed circuits is set to 0.001, and the failure repair rate is 1. With
around the mean: the two parameters, the parameter v in Algorithm 1 can be
 calculated. It still is difficult to obtain samples that satisfy the
 3
 N N N
 
 i�1 p − (1/N) j�1 pj attribute φ with the previous parameters. Therefore, to
 ) �
 skew(p (3/2)
 . ensure that the obtained samples can satisfy the attribute φ,
 (N − 1)(N − 2) (Var[p ])
 the initial failure rate is set as 0.1 and the fault repair rate is
 (25) set as 1. According to these two parameters, the initial
 The negative skewness means that the distribution is left- parameter of iteration p(0) in Algorithm 1 can be calculated.
tailed. At this time, the data on the left of the mean are less In order to assess the performance of verifying the rare
than the data on the right. Intuitively, the tail on the left is attributes with the CESVA method, 20 iterations of Algo-
longer than the tail on the right. In contrast, the positive rithm 1 are performed. In each iteration, the number of
skewness means that the distribution is right-tailed. The data samples is N � 104, the smoothing factor α � 0.2, and the
on the right of the mean is less than the left. Intuitively, the total number of required samples is 2.0 × 105 .
tail on the right is longer than the tail on the left. Figure 1 shows the change trend of the failure rate
 parameters during the 20 iterations of the CESVA method.
 At the beginning of the iteration, the parameters converge
4.2. Experiment and Analysis on a Fault-Tolerant Controller rapidly. When the parameters are close to their optimal
for the Aircraft Elevator System. The fault-tolerant controller values, the convergences of their values slow down with
for an aircraft elevator system is a part of a large Simulink random fluctuations. From the 16th iteration, the failure rate
model of HL-20 rescuers developed by the National parameters start to converge to the stable values. From the

Security and Communication Networks 9

6 10–10
5

4 10–11
3

2 10–12
1

0
 10–13
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
 λ1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
 λ2
 λ3 Figure 2: Distribution of estimated values of CESVA during 20
 iterations.
 Figure 1: Convergence of parameters during 20 iterations.

perspective of the parameter convergence trend, it seems 1.0
that the value of the failure rate parameter increases with the
increasing iteration times. It indicates that the proportion of
sampling the paths satisfying the rare attribute is gradually 0.1
increasing.
 Figure 2 illustrates the distribution of the estimated
values of the CESVA method during the iterations. The 0.01
estimated value gradually converges from the 17th iteration.
Figure 3 presents the distribution of the relative error of the
CESVA method during the iterations. The relative error 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
gradually converges from the 16th iteration. Finally, the
 Figure 3: Distribution of relative error of CESVA during 20
probability estimated value of the security attribute φ is iterations.
1.682 × 10− 12 , and the value of the relative error is 0.01.
 In order to verify the statistical performance of the
CESVA method, 100 experiments were carried out under the the Stateflow/Simulink model, Clarke and Zuliani [24]
above parameters, and 2.0 × 105 samples were used in each proposed the SMC method of cross-entropy minimization
experiment. Compared with the performance of the HIS importance sampling to verify its safety properties.
method under the same sample size, Table 1 shows the mean, Zuliani et al. [17] further used the SMC method in paper
skewness, and statistical indicators such as standard devi- [24] to verify the safety properties of a class of discrete-
ation (likelihood ratio standard deviation), relative error, time SHS. The method proposed by Clarke and Zuliani
and sample size for each experiment. As presented in Ta- [24] assumes that the distribution of the system path space
ble 1, with the same sample size, the estimated values of the is exponential distribution. By simply increasing the
CESVA method are more closely distributed around the failure rate of the system parameters and calculating the
mean value, and the likelihood is over 10 times less than the optimal parameters of the exponential distribution with
standard deviation and relative error, when comparing the paths satisfying the rare attributes extracted at one
against the HIS method. Although the true probability is time, an importance sampling distribution can be ob-
unknown, statistical indicators such as the standard devi- tained. J´egourel et al. [26] used a random guardian
ation, skewness, and relative error of the likelihood ratio command to the importance sampling distribution. This
illustrate that the true probability and the mean are very model can approximate the path distribution of the system
close. by increasing the number of commands (the number of
 parameters) and uses the minimized cross-entropy
5. Related Work method to obtain an importance sampling distribution in
 the random model. However, the optimal importance
The verification of the rare attribute for SMC mainly in- sampling distribution obtained by the above method does
cludes the importance sampling method, the importance not come from the distribution family of the system path
splitting method, and the statistical learning method. space, and these methods actually belong to the heuristic
 The importance sampling method is an effective importance sampling method.
method to solve the verification of rare attributes. For the The importance segmentation method [34] is a method
CTMC and DTMC random models, Reijsbergen et al. [40] of reducing the estimated variance. Based on the importance
and Barbot et al. [23] leveraged the heuristic methods to segmentation method, J´egourel et al. [33] proposed the
obtain an importance sampling distribution to complete SMC algorithm for the verification of small probability
the attribute verification of the two types of models. For events. The key idea is to decompose the system logic

10 Security and Communication Networks

 Input: N, the number of samples per iteration.
 Input: v, the true path distribution parameter of SCCPS.
 Input: p(0) , the initialization parameter.
 Input: jmax , the maximum number of iterations.
 Output: p∗ Optimal parameters.
 (1) Function learningAlg (N, v, p(0) jmax )
 (2) j � 0;
 (3) while j < jmax do
 (4) A � 0, B � 0, i � 1
 (5) while i ≤ N do
 (6) generate a path ωi according to the pdf f(., p(j) )
 (7) if ωi ⊨φ then (ωi )
 (8) Wi � ⟨l,m⟩∈S(ωi )(vlm /plm )nlm ;
 (ω )
 (9) A � A + Wi nlm i ;
 (ω )
 (10) B � B + Wi nl i ;
 (11) i � i + 1;
 (j+1) (j)
 (12) plm � αplm + (1 − α)(A/B);
 (13) j�j+1
 (14) return p(j− 1)

 ALGORITHM 1: Importance sampling distribution learning algorithm.

 Input: NI S, The number of samples.
 Input: v, the true path distribution parameter of SCCPS.
 Input: p∗ , the optimal parameters calculated by Algorithm 1.
 Output: p , Probability of SCCPS meeting safety attributes.
 (1) Function verifyingAlg (N, v, p(0) jmax )
 (2) A � 0, i � 1
 (3) while i ≤ N do
 (4) generate a path ωi according to the pdf f(., p(j) )
 (5) if ωi ⊨φ then (ωi )
 (6) Wi � ⟨l,m⟩∈S(ωi )(vlm /plm )nlm :
 (7) A � A + Wi ;
 (8) i�i+1
 (9) return (A/NIS )

 ALGORITHM 2: Safety verification algorithm.

Table 1: Comparison of statistical performance between CESVA Applying statistical learning methods to SMC is also an
and HIS. important research direction. Du et al. [19] proposed a
 learning SMC framework based on support vector machine-
 Standard
Algorithm Mean Skewness
 deviation
 Relative error based two classifiers. It uses cost-sensitive and resampling
 12 − 14 methods to solve the unbalanced data learning problem of
CESVA 1.687×10− 0.029 1.853×10 0.011
 12 support vector machines and implements predicting and
HIS 1.986×10− 1.264 2.654×10− 13 0.133
 assessing the probability of occurrence of small-probability
 events with a relatively small number of samples. However,
attributes into embedded attributes, which makes its this method cannot obtain rare attribute samples. For the
probability easier to be estimated and reduces the number of low-probability attributes of hardware circuits with multiple
sample paths required by verification. To improve the failure regions, Kumar et al. [41] assumed that the system
performance, the attributes need to be decomposed into failure distribution is a Gaussian mixture model, thus
multiple levels with different probabilities. During the de- proposed to use the variational Bayes method to learn an
composition process, copying or eliminating paths depend optimal importance sampling distribution from the
on their intermediate behavior. When the decomposition is Gaussian mixture model. However, the optimal importance
over, an estimated probability that the attribute is satisfied sampling distribution is not a distribution family from the
can be obtained. The importance segmentation method is system path space. Kalajdzic et al. [42] proposed an SMC
essentially heuristic and depends on the model, but lacks the method based on the principle of feedback control. This
support of theoretical results. method learns a model of a cyber-physical fusion system by

Security and Communication Networks 11

using importance sampling to estimate the system state and Ministry of Industry and Information Technology Research
importance division to control the system. So it can infer the Project (NJ2019006).
probability that the system satisfies the given attributes.
 The method proposed in this paper starts from the
SCCPS path probability space, constructs a cross-entropy
 References
optimization model, and uses an iterative learning method [1] N. A. Tanner, J. R. Wait, C. R. Farrar, and H. Sohn, “Structural
to obtain an optimal importance sampling distribution from health monitoring using modular wireless sensors,” Journal of
the parameterized distribution clusters of the path space. It Intelligent Material Systems and Structures, vol. 14, no. 1,
ensures that the optimal importance sampling distribution pp. 43–56, 2003.
can come from the distribution family in the path probability [2] S. K. Kampf, M. Salazar, and S. W. Tyler, “Preliminary in-
space of SCCPS. And, the iterative learning method ensures vestigations of effluent drainage from mining heap leach
that the distribution can evenly cover the unsafe path dis- facilities,” Vadose Zone Journal, vol. 1, no. 1, pp. 186–196,
tribution area. Therefore, the accuracy and efficiency of the 2002.
rare attribute verification can be improved significantly. [3] G. Chunpeng, Z. Liu, J. Xia, and F. Liming, “Revocable
 identitybased broadcast proxy re-encryption for data sharing
 in clouds,” IEEE Transactions on Dependable and Secure
6. Conclusion Computing, vol. 99, 2019.
SMC has been successfully applied to SCCPS safety attri- [4] L. Yu and J.-p. Wang, “Review of the current and future
 technologies for video compression,” Journal of Zhejiang
bute verification and has become the most effective solu-
 University Science C, vol. 11, no. 1, pp. 1–13, 2010.
tion, but rare attribute verification is still a challenge for
 [5] H.-h. Xu and J. Zhu, “Aniterative approach to Bayes risk
SMC. To be able to extract samples satisfying the rare decoding and system combination,” Journal of Zhejiang
attributes from SCCPS, CTMC is used to construct the University SCIENCE C, vol. 12, no. 3, pp. 204–212, 2011.
probability space model of the execution path of SCCPS [6] O. Déniz, M. Castrillón, J. Lorenzo, L. Antón, M. Hernandez,
given with the probability measure of the random execu- and G. Bueno, “Computer vision based eyewear selector,”
tion path as well as the parameterized probability distri- Journal of Zhejiang University Science C, vol. 11, no. 2,
bution function family, to construct the cross-entropy pp. 79–91, 2010.
iterative model. According to the iteratively learning from [7] D. Theodoridis, Y. Boutalis, and M. Christodoulou, “Direct
finding the approximate optimal importance sampling adaptive regulation of unknownnonlinear systems with
distribution in the SCCPS path probability space, the ef- analysis of themodel order problem,” Journal of Zhejiang
ficient sampling of rare attribute samples in SCCPS is University Science C, vol. 12, no. 1, pp. 1–16, 2011.
achieved. With the evaluating experiments, the experi- [8] X.-c. Zhou, H.-b. Shen, and J.-p. Ye, “Integrating outlier
mental results show that, for the verification of rare at- filtering in large margin training,” Journal of Zhejiang Uni-
tributes, comparing against the heuristic importance versity Science C, vol. 12, no. 5, pp. 362–370, 2011.
sampling method with the same number of samples, the [9] I. Prigogine, Order through Fluctuation: Self-Organization and
estimated value of our method is better distributed around Social System, pp. 93–134, Addison-Wesley, London, UK,
 1976.
the mean, and the standard deviation and relative error are
 [10] C. Ge, W. Susilo, Z. Liu, J. Xia, P. Szalachowski, and F. Liming,
reduced by more than an order of magnitude. Based on the
 “Secure keyword search and data sharing mechanism for
method proposed in this paper, combining with the current cloud computing,” IEEE Transactions on Dependable and
mainstream SMC method to develop an adaptive SMC tool Secure Computing, vol. 5, 2020.
is set as the future work. [11] Z. Wu, Y. An, Z. Wang et al., “Study on zoelite enhanced
 contact-adsorption regeneration-stabilization process for
Data Availability nitrogen removal,” Journal of Hazardous Materials, vol 156,
 2008 in press.
The data used to support the findings of this study are [12] H. L. S. Younes, “Error control for probabilistic model
available from the corresponding author upon request. The checking,” in Lecture Notes in Computer Science,
authors apply CESVA to a fault-tolerant controller for an E. A. Emerson and K. S. Namjoshi, Eds., pp. 142–156,
aircraft elevator system (FTC4AE) that is a State-flow/ Springer, Berlin, Germany, 2006.
Simulink hybrid system modeling case from MATLAB. [13] K. G. Larsen, “Statistical model checking, refinement
 checking, optimization, . . . for stochastic hybrid systems,” in
Conflicts of Interest Lecture Notes in Computer Science, pp. 7–10, Springer, Berlin,
 Germany, 2012.
The authors declare that they have no conflicts of interest. [14] Q. Wang, P. Zuliani, S. Kong, S. Gao, E. M. Clarke, and
 “ SReach, “SReach: a probabilistic bounded delta-reachability
Acknowledgments analyzer for stochastic hybrid systems,” Computational
 Methods in Systems Biology, vol. 9308, pp. 15–27, 2015.
This work was supported by the National Key Research and [15] S. Gorini, M. Quirini, A. Menciassi, G. Permorio, C. Stefanini,
Development Program of China (no.2018YFB1003900), and P. Dario, “A novel sma-based actuator for a legged en-
National Natural Science Foundation of China (no. doscopic capsule,” in First IEEE/RAS-EMBS International
61772270), Key Laboratory of Safety-Critical Software Conference on Biomedical Robotics and Biomechatronics,
(Nanjing University of Aeronautics and Astronautics), and pp. 443–449, Pisa, Italy, February 2006.

12 Security and Communication Networks

[16] U. Rizvi, Combined Multiple Transmit Antennas and Multi- checking,” in Lecture Notes in Computer Science, pp. 143–159,
 Level Modulation Techniques, Stockholm, Sweden, Europe, in Springer, Berlin, Germany, 2014.
 Swedish, 2006. [34] G. Jiang and M. C. Fu, “Importance splitting for finite-time
[17] P. Zuliani, C. Baier, and E. M. Clarke, “Rare-event verification rare event simulation,” IEEE Transactions on Automatic
 for stochastic hybrid systems,,” in Proceedings of the ACM Control, vol. 63, no. 6, pp. 1670–1677, 2018.
 International Conference on Hybrid Systems: Computation & [35] D. P. Kroese, T. Taimre, and Z. I. Botev, Handbook of monte
 Control, pp. 217–226, ACM, Quebec, Canada, April 2012. carlo methods, John Wiley & Sons, Hoboken, NJ, USA, 2013.
[18] P. Zuliani, A. Platzer, and E. M. Clarke, “Bayesian statistical [36] P.-T. de Boer, D. P. Kroese, S. Mannor, and R. Y. Rubinstein,
 model checking with application to stateflow/simulink veri- “A tutorial on the cross-entropy method,” Annals of Opera-
 fication,” Formal Methods in System Design, vol. 43, no. 2, tions Research, vol. 134, no. 1, pp. 19–67, 2005.
 pp. 338–367, 2013. [37] A. Costa, O. D. Jones, and D. Kroese, “Convergence properties
[19] D. Du, B. Cheng, and J. Liu, “Statistical model checking for of the cross-entropy method for discrete optimization,”
 rare-event in safety-critical system,” Journal of Software in Operations Research Letters, vol. 35, no. 5, pp. 573–580, 2007.
 Chinese, vol. 26, no. 2, pp. 305–320, 2015. [38] B. Boyer, K. Corre, A. Legay, and S. Sedwards, “PLASMA-lab:
[20] L. Sweeney, Uniqueness of simple demographics in the U.S. a flexible, distributable statistical model checking library,” in
 population, Technical Report No. LIDAP-WP4, Carnegie Proceedings of the 10th International Conference on Quanti-
 Mellon University, Pittsburgh, PA, USA, 2000. tative Evaluation of Systems, pp. 160–164, Buenos Aires,
[21] ISO, “Steels-classification-part 1: classification of steels into Argentina, August 2013.
 unalloyed and alloy steels based on chemical composition,” [39] M. V. Stringfellow, N. G. Leveson, and B. D. Owens, “Safety-
 Technical Report ISO 4948-1, ISO, Geneva, Switzerland, 1982. driven design for software-intensive aerospace and automo-
[22] D. Reijsbergen, P. de Boer, W. R. W. Scheinhardt, and tive systems,” Proceedings of the IEEE, vol. 98, no. 4,
 B. R. Haverkort, “Rare event simulation for highly dependable pp. 515–525, 2010.
 systems with fast repairs,” in Proceedings of the Seventh In- [40] D. Reijsbergen, P. de Boer, W. R. W. Scheinhardt, and
 ternational Conference on the Quantitative Evaluation of B. R. Haverkort, “Rare event simulation for highly dependable
 Systems, pp. 251–260, IEEE, Williamsburg, VA, USA, Sep- systems with fast repairs,” Perform. Evaluation, vol. 69, no. 7-
 tember 2010. 8, pp. 336–355, 2012.
[23] B. Barbot, S. Haddad, and C. Picaronny, “Coupling and [41] J. A. Kumar, S. N. Ahmadyan, and S. Vasudevan, “Efficient
 importance sampling for statistical model checking,” Tools statistical model checking of hardware circuits with multiple
 and Algorithms for the Construction and Analysis of Systems, failure regions,” IEEE Transactions on Computer-Aided Design
 vol. 7214, pp. 331–346, 2012. of Integrated Circuits and Systems, vol. 33, no. 6, pp. 945–958,
[24] E. M. Clarke and P. Zuliani, “Statistical model checking for 2014.
 cyber-physical systems,” Automated Technology for Verifica- [42] K. Kalajdzic, C. J´egourel, A. Lukina et al., “Feedback control
 tion and Analysis, vol. 6996, pp. 1–12, 2011. for statistical model checking of cyber-physical systems,” in
[25] University, Citing Electronic Sources of Information, Uni- Proceedings of the leveraging applications of FormalMethods,
 versity of Sheffield Library, Howard, UK, 2001, http://www. verification and Validation: foundational techniques - 7th
 shef.ac.uk/library/libdocs/hsl-dvc1.pdf. international Symposium, ISoLA 2016, Imperial, Corfu,
[26] C. J´egourel, A. Legay, and S. Sedwards, “Command-based Greece, October 2016.
 importance sampling for statistical model checking,” Theo-
 retical Computer Science, vol. 649, pp. 1–24, 2016.
[27] H. L. S. Younes and R. G. Simmons, “Statistical probabilistic
 model checking with a focus on time-bounded properties,”
 Information and Computation, vol. 204, no. 9, pp. 1368–1409,
 2006.
[28] T. H´erault, R. Lassaigne, F. Magniette, and S. Peyronnet,
 “Approximate probabilistic model checking,” in Lecture Notes
 in Computer Science, pp. 73–84, Springer, Berlin, Germany,
 2004.
[29] Y. J. Kim, M. Kim, and T. Kim, “Statistical moHaifa, Israeldel
 checking for safety critical hybrid systems: an empirical
 evaluation,” in proceedings of the 8th international haifa
 verification conference on hardware and software: verification
 and testing, pp. 162–177, Haifa, Israel, November 2012.
[30] G. Agha and K. Palmskog, “A survey of statistical model
 checking,” ACM Transactions on Modeling and Computer
 Simulation, vol. 28, no. 1–6, pp. 6–39, 2018.
[31] A. Legay and M. Viswanathan, “Statistical model checking:
 challenges and perspectives,” International Journal on Soft-
 ware Tools for Technology Transfer, vol. 17, no. 4, pp. 369–376,
 2015.
[32] J. Hu, J. Lygeros, and S. Sastry, “Towards a theory of stochastic
 hybrid systems,” Hybrid Systems: Computation and Control,
 vol. 337, pp. 160–173, 2000.
[33] C. J´egourel, A. Legay, and S. Sedwards, “An effective heuristic
 for adaptive importance splitting in statistical model