Fast Configuration Change Impact Analysis for Network Overlay DCNs - Lizhao You, Hao Tang, Jiahua Zhang, Xiao Li Huawei Technologies ...

APNET’20, August 3-4, 2020

 Fast Configuration Change Impact
 Analysis for Network Overlay DCNs
 Lizhao You, Hao Tang, Jiahua Zhang, Xiao Li
 Huawei Technologies

 youlizhao@huawei.com

 HUAWEI TECHNOLOGIES CO., LTD.

Network Overlay DCNs

  ToR switches: VxLAN tunnel endpoints and
 distributed gateways
 Delta Configurations
  Layer 2: bridge-domain interface (BDIF)

 PE Router
  Layer 3: virtual BDIF (VBDIF)
 FW,
 LB,  Distributed protocol: BGP EVPN
 …
 Border Leaf
 & Spine
  Policies: ACLs, micro-segmentation (MCS)

 ToR Leaf

 B C D
  To deploy/update services, users design the VPC
 A
 network, and specify policies through the controller
 Server
  Delta configurations are generated, and pushed to
 the fabric
 DCN Fabric

 APNET’20, August 3-4, 2020 Page 2

Problem: All-Pair Reachability Changes
 Before Deployment

 B C Delta B C
 Changes
 Expected
 Delta Configurations
 A A
 Unexpected
 PE Router
 FW,
 LB,
 …
 Border Leaf
 All-Pair Reachability Matrix All-Pair Reachability Matrix
 & Spine

 ToR Leaf
  Re-computing full matrix is not scalable

 A
 B C D  Incremental: find possibly changed pairs, and re-
 Server compute their reachability only
  Existing incremental approaches do not address the

 DCN Fabric new challenges introduced by network overlay DCNs

 APNET’20, August 3-4, 2020 Page 3

Challenges #I: Model Expressiveness
 Tunnel Header  Feature 1: tunneling with overlay/underlay traffic
  Feature 2: packet rewrites for multiple tunnels
  Feature 3: MCS, group definitions are distributed
  A: 1.1.1.0/24, 1.1.2.0/24  Group ID: 10
 Packet Rewrite  B: 1.1.3.0/24  Group ID: 10
  C: 1.1.4.0/24  Group ID: 20
  Policy on groups:
 Cross-VRF static route: two tunnels
  SRC ID 10, DST ID 20, SRC PORT 80, DENY
 Border Leaf
 VRF2
 Existing BDD-based EC approaches (APV[ICNP’13, APKeep[NSDI’20])
 VRF1

 and TBV-based EC approaches (VeriFlow[NSDI’13], DeltaNet[NSDI’17])

 ToR Leaf
 • Current EC computation does not support tunneling
 • Extensive packet rewrites  Performance degradation

 A B C D
 • The TBV model relies on IP rules instead of Group ID rules
 VPC1/VRF1 VPC2/VRF2 • Conversion  Excessive IP rules
 • Invalid: some SRCs may not reach DSTs
 APNET’20, August 3-4, 2020 Page 4

Challenges #2: Completeness of Indexing Methods

 Indexing method (e.g., TenantGuard [NDSS’17])
 Add a static route

 B1
 E
 B2  Compute a reachable path
  Associate the visited device with the reachable pair
  B1  {(A,B), (A,C)}

 L1 L2
  Any changes on device B1
 L3 L4
  Re-compute (A,B) and (A,C)
 A B C D

Device-level association is coarse-grained and inefficient Can not find new reachable pairs
• E.g., E only changes (A,C), no need to re-compute (A,B) • E.g., B2 adds a static route for A  D, and
• Interface-level association may be preferable B has no association with (A,D) initially

 APNET’20, August 3-4, 2020 Page 5

Our Solutions

 BDD Predicate Model Fine-Grained Indexing Table

 1 2

 3

 Waypoint to find relevant (new) reachable pairs

 We follow the control-plane verification approach:
 protocol simulation  reachability analysis

 APNET’20, August 3-4, 2020 Page 6

Port-Predicate Model
Symbolic Packet & Boolean Formula (Predicate)
 VxLAN header

 Outer IP GroupID VNI Inner IP HDs
 HDs Aggregation of allowed space of ACLs and FIB
 64bits 16bits 24bits 64bits

 1
 ACL 
 ∩ ( ) ∩ ACL Forwarding ports of devices
 i-th bit: 1
 i-th bit: 0 
 Underlay or Overlay: general if
i-th bit: 0 or 1 ∪ = universal quantifier then
 if ∀ , v == Overlay Packet
Each IP/Prefix = ( 1 , … , 168 )
 else
 Underlay Packet
 Header Space = ( 1 , … , )
 (union, intersection, difference) Rewrite: erase-and-set

 All Boolean formulas are represented by BDD existence quantifier == erase ∃ , v ∩ 

 APNET’20, August 3-4, 2020 Predicate quantifiers improve
 Pageour
 7 model expressiveness

All-Pair Reachability Matrix

 Forwarding Graph DFS for each starting endpoint
 B C
 A
Start A If2 End
 If1 B
 If1 A

 If2

 B All-Pair Reachability Matrix
 If1=(L1,NVE), If2=(L2,NVE)
 Endpoints
 A

 If1

 If2

 Indexing Table
 B

 APNET’20, August 3-4, 2020 Page 8

Change-Impact Analysis

Step 1: Changed Interface Detection Step 2: Waypoint Computation Step 3: Comparison
 Changed
 Interfaces
 Backward Forward DEL MOD ADD

 Indexing

 Old forwarding graph New forwarding graph Reachable
 Pairs

 Comparison

 Reachable Pairs

 DEL MOD ADD DEL MOD ADD

 APNET’20, August 3-4, 2020 Page 9

Preliminary Evaluation
 Implementation Synthesized Datasets (similar to real networks)
  Based on Batfish*  Base dataset (20Leafs ~ 100Leafs)  Different service update cases
  Support Huawei CLI/YANG  Inter-VPC traffic  Case A/B: ADD SUBNET/ADD VPC
  BGP EVPN protocol  Intra-VPC (inter-subnet) traffic  Case C: ADD cross-VRF Static Routes
  JavaBDD  1 Leaf has 20 endpoints (subnets)  Case D: MOD MCS

 Case D: increase is
 Case C
 mostly due to waypoint.
 To be optimized.

 Less than 25s
 for 2k EPs (4 mill. Pairs)
 6X
 Cases A-C: increase
 is due to modeling,
 detection, comparison 2X

 Metric: new graph modeling, detection, waypoint computation and comparison time
 APNET’20, August
* https://github.com/batfish/batfish 3-4, 2020 Page 10

Conclusion

 We are the first to design and demonstrate an incremental configuration verifier
 for network overlay DCNs.

 We design a fast incremental verification algorithm that leverages fine-grained
 indexing and waypoint computation methods to find all-pair reachability changes.

 In the future, we will further explore new forwarding features: policy-based
 routing, firewall zone policies, NAT policies, etc.

 APNET’20, August 3-4, 2020 Page 11

Thank you
 www.huawei.com

Copyright©2011 Huawei Technologies Co., Ltd. All Rights Reserved.
The information in this document may contain predictive statements including, without limitation, statements regarding the
future financial and operating results, future product portfolio, new technology, etc. There are a number of factors that
could cause actual results and developments to differ materially from those expressed or implied in the predictive statements.
Therefore, such information is provided for reference purpose only and constitutes neither an offer nor an acceptance. Huawei
may change the information at any time without notice.