Bridging the Governance Gap: Interoperability for blockchain and legacy systems - WHITE PAPER DECEMBER 2020 - weforum.org
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Centre for the Fourth Industrial Revolution Bridging the Governance Gap: Interoperability for blockchain and legacy systems WHITE PAPER DECEMBER 2020
Cover: Unsplash/Terry
Inside: Unsplash/chuttersnap; Unsplash/Luke Ellis; Unsplash/RenRan; Unsplash/MarkusSpiske;
Unsplash/ScottWebb; Unsplash/AndersJilden; Unsplash/ChristianHolzinger
Contents
3 1 Introduction
4 1.1 Why do legacy systems and DLT solutions need to embrace each other?
7 1.2 Systemic concerns and how integration with DLT can help
8 1.3 Limitations of DLT and challenges of integration
9 2 Definitions
11 3 Interoperability – fundamental pillars
13 4 The importance of an open‑source, decentralized data validation middleware for enterprise systems
15 4.1 Contribute to or join a large open‑source community building on a common interoperability
framework
16 4.2 Adopt an oracle network that operates across all blockchain environments
18 4.3 Use decentralization to validate the integrity of data inputs and provide highly available, tamperproof
data exchange
20 4.4 Demand oracles that support access to authenticated data sources and credentials management
capabilities
22 4.5 Use crypto assets to provide for crypto‑economic guarantees
24 4.6 Leverage oracle networks with marketplaces and reputation frameworks to identify high‑quality
node operators
26 4.7 Build on a generalized oracle network with multiple security layers for defence in depth
27 5 Addressing legal and data privacy concerns
29 6 ‘Blockchain Abstraction Layer’: A bridge between blockchains and legacy systems built on interoperability
fundamentals
32 7 Conclusion: What does an ideal integrated system look like?
35 Contributors
36 Appendix: Reviewers
37 Further reading
39 Endnotes
© 2020 World Economic Forum. All rights
reserved. No part of this publication may
be reproduced or transmitted in any form
or by any means, including photocopying
and recording, or by any information
storage and retrieval system.
Bridging the Governance Gap: Interoperability for blockchain and legacy systems 2
December 2020 Bridging the Governance Gap:
Interoperability for blockchain and legacy systems
1 Introduction
Smart contracts can unlock the hidden value of
legacy digital systems based on interoperability
with the capabilities of DLT systems.
Bridging the Governance Gap: Interoperability for blockchain and legacy systems 3
In the past 10 years, blockchain and distributed Multiple reports analysing the blockchain/DLT
ledger technologies (DLT) have generated adoption by organizations have pointed out that
tremendous interest and activity from developers, blockchain integration with other systems (e.g.
enterprises, venture capitalists, regulators and users other blockchains or other non‑DLT information
alike. The innovation of blockchain technology leads systems) is one of the crucial challenges. Early
to an important question about how legacy digital experiments on interoperability have demonstrated
systems, operated by enterprises, governments and DLT‑to‑legacy integration to be useful in many use
institutions, will be affected. Presently, the answers cases (e.g. reinsurance) in establishing trust among
to this question have varied from one extreme (“all multiple parties. This white paper intends to identify
legacy systems will be replaced”) to the other (“DLT the foundational pillars for legacy system‑DLT
is too slow and unproven to actually replace any system interoperability as well as efforts made in
working legacy system”). However, the eventual this direction, and recommends the adoption of
answer may lie somewhere in between, where the a holistic industry standard that can accelerate
utility of select legacy systems is upgraded by DLT the development and implementation of these
integration wherever appropriate, and DLT solutions “interoperability bridges” across geographic regions
witness a growth in enterprise adoption.1 and disparate systems.
1.1 Why do legacy systems and DLT solutions need to
embrace each other?
Blockchain and distributed ledger technologies fault tolerance etc. It has been referred to as one
(DLTs) are backend infrastructure that store and of the features of “blockchain trilemma”: ensuring
transfer data among parties within a shared ledger blockchain decentralization, scalability and security
without the need for traditional intermediaries. Since imply trade‑offs, at least in the short term. To
the ledger is redundantly processed and updated by a certain extent, the mechanisms for ensuring
a decentralized network of computers instead of a decentralization at different blockchain layers may
centrally managed server, users can generally trust conflict with security and scalability.
the integrity of the data and computations without
the need for trusted authorities. The objective is to While simple smart contracts can be created
use a blockchain as a shared ledger for exchanging within isolated blockchain environments to store
value between disparate entities in order to achieve data or execute transactions between users, they
greater efficiency, heightened transparency and a stand to provide more value to enterprise and
reduction in complex reconciliation. other ecosystems when connected to data and
systems outside of the blockchain. For instance, if
One of the most discussed innovations in blockchain an insurance service provider wishes to automate
technology is smart contracts – conditional business the dispersal of flight insurance claims, it could do
logic (if x event happens, then execute y action) so via a smart contract. But for the smart contract
that is executed on the blockchain. These also to execute, it needs accurate information on the
allow the creation of digital assets where the smart scheduled and actual departure times of the flights,
contract uses embedded logic by defining how as well as the ability to settle in fiat currencies on
those digital tokens function (e.g. represent voting traditional payment rails. Since blockchains do
rights or a stake in protocol revenue). Since smart not natively generate this information or provide
contracts operate on a blockchain, in general no connections to traditional systems, a piece of
counterparty or external entity can tamper with the infrastructure called an “oracle” must be adopted to
terms, execution or outcome, providing the user connect the smart contract to external resources.
with technologically enforced guarantees that the Additionally, since a smart contract cannot execute
contract will be fairly honoured.2 Given the nature of on software outside of the blockchain, the oracles
these guarantees, smart contracts are being looked provide the smart contract with inputs and also
at as a new form of multiparty business automation. take the outputs and execute them as actions on
However, it should be noted that smart contracts are external systems.
vulnerable to exploitation if their code is not audited
well for security, fault tolerance (even when nodes go Oracles serve as an external source of data for DLT
down) and privacy. systems. They connect the external world to the
self‑contained world of DLTs, acting as middleware
As part of their underlying design, blockchains for data and transaction sharing between
create strong security guarantees at the expense environments in a secure and authoritative manner.
of being inherently disconnected from any Information shared by oracles is digitally signed and
network or system in the outside (off‑chain) hence is considered non‑repudiable (assurance
world. This decentralized consensus design that the signature cannot be denied by the party
runs on other inherent costs such as rewards for who signed it). Since smart contracts are executed
validating transactions on‑chain and ensuring on DLT in a deterministic fashion (i.e. transactions
Bridging the Governance Gap: Interoperability for blockchain and legacy systems 4
that execute exactly as written because they are to retrieve external data, validate it and deliver it
verified by all nodes with high fault tolerance), the to the intended entity. Each step of the process
built‑in ability to communicate with the external requires important considerations to ensure the
world becomes difficult to establish without a loss desired qualities of the blockchain (e.g. being
of trust. Oracles become this entity that signs tamper‑resistant, permissionless and immutable)
claims about the state of the external world. Since are not lost when expanding to off‑chain data
DLT systems were built intentionally to be detached and systems. This requires an oracle to be able
from the external world and its trusted third parties, to source data from high‑quality application
it is crucial that the links (i.e. oracles) have high programming interfaces (APIs), show proof of the
integrity. Decentralization, economic incentives, use origin of the data, maintain secure and reliable
of trusted execution environments etc. are some of data delivery, provide economic guarantees to
the ways in which oracle integrity is ensured on a incentivize trusted oracle services and possibly
varying basis (based on the end use case). Some of even provide additional cryptographic techniques
these approaches are discussed in this paper. to keep the data itself private. Some oracles
are physical or tangible devices that measure
Oracles are not blockchains themselves but real‑world values, such as temperature or whether
are secure blockchain middleware that a shipment has arrived safely. Other oracles are
operates partly on the blockchain (“on‑chain”) intangible, and comprise only code. For instance,
and partly outside of the blockchain (“off‑chain”) the recording of external product prices on a
asynchronously. The main goal of an oracle is blockchain is facilitated by oracles.
FIGURE 1 Data flow in a typical DLT‑legacy interoperability framework through oracles
World’s data Decentralized oracle Any blockchain Decentralized oracle
Legacy systems
sources network network network
01011 01010 11100 01101
Quality data Data delivery Crypto-economic Crypto-economic Data delivery Quality data
Origin proofs Origin proofs
inputs and data validation guarantees Data privacy guarantees and data validation outputs
Key considerations
An enterprise’s legacy systems could generally Let’s explore an example to understand why such
connect, bidirectionally, to blockchain networks interoperability is beneficial.
using an oracle. This model is essential if smart
contracts are to have a significant global impact The Government of India launched an ambitious
on business process efficiency and transparency, programme called Pradhan Mantri Fasal Bima
particularly because enterprise smart‑contract Yojana (translated as Prime Minister’s Crop
use cases generally require access to high‑quality Insurance Scheme; abbreviated as PMFBY) in 2016
off‑chain data and traditional business infrastructure to provide farmers with insurance coverage and
in order to run end‑to‑end. financial support for any losses due to unforeseen
crop failures arising out of natural calamities/pests/
In order to ensure communication between diseases and to encourage them to adopt modern
off‑chain legacy systems and on‑chain smart technology and innovative farming methods. Most
contracts and to be able to expand the utility of of the insurance premium is covered by the central
smart contracts beyond just DLT applications, and state governments, while the farmer pays a
adopting an approach towards building maximum of 2% of the total premium annually.
secure middleware solutions (developed using The large scale of the programme is evident from
open‑source technology platforms) that anyone its broad coverage and the type of stakeholders
is able to connect to and build on as needed, involved, as reported by the government in the
along with governance models that harmonize Kharif cropping season (June–October) of 2019.3
inter‑system communication, is required.
Bridging the Governance Gap: Interoperability for blockchain and legacy systems 5
FIGURE 2 Highlights of Pradhan Mantri Fasal Bima Yojana
Total farmers covered 18 million Stakeholders / implementational agencies
Crop area insured 28 million hectares Central govt. Insurance cos. Weather cos.
Total premium $2 billion State govt. Banks Farmers
Source: The Prime Minister’s Total insurance Insurance Agri research Drone / remote
12+
companies involved regulator institutions sensing cos.
Crop Insurance Scheme as
on October 2019
In accordance with the operational guidelines, to collect and process data, which often have
a variety of external stakeholders are brought highly varying requirements in terms of privacy,
together to generate/collect data on crop yield security, transaction speed and human intervention
and weather, conduct crop‑cutting experiments (usually mandated by governmental regulations).
to verify actual yield, undertake remote sensing to Apart from these systems, the central government
collect large‑scale data on crop impact, working operates a National Crop Insurance Portal as the
with insurance companies to underwrite the central IT system that interacts with all of the other
risk, and banks/governments to provide farmers infrastructure and external data sources required.
with credit and welfare transfers. All of these The following schematic shows a summary of the
stakeholders and many other subcontracted many different types of data sources that are used
agencies operate a variety of technical systems in the operationalization of PMFBY:
FIGURE 3 Various legacy IT systems being used in PMFBY and their data
Pay-Gov:
payment gateway
Premium, claim
remittances
Mahalanobis IT system of
National Crop insurance
Forecast Centre company
Dought assessment, Crop-cutting
sown area reduction/ mid-term/preventive/
correction sowing/ individual/
National localized claims
Crop Insurance
Portal
ISRO satellite
IMD’s automatic Digital land
data integration
weather stations record systems
Crop health, vegetation and rain gauges of state govts.
index, sowing and Real-time weather Digital land records,
Source: https://pmfby.gov. harvesting activity
in/pdf/Revised_Operational_
parameters code-mapping
tracking
Guidelines.pdf
Bridging the Governance Gap: Interoperability for blockchain and legacy systems 6
1.2 Systemic concerns and how integration with
DLT can help
Given the scale of the scheme and the multiplicity of – Information security: Keeping digital
stakeholders involved, several concerns are identified information secure solely through traditional
and addressed by the implementational agencies: methods may not be very effective, especially
against rogue system administrators and
– Transparency and accountability: As a single points of failure. DLT generally makes it
taxpayer‑funded programme, transparency and difficult to break in and alter records (especially
accountability become increasingly important for on large public networks such as Bitcoin
the government and regulators. An advantage and Ethereum), which may further incentivize
blockchain brings over legacy databases is stakeholders to provide honest services to
its ability to record the first use of data (in a maintain a strong reputation.
transaction or publication) on the blockchain,
record it immutably (so there is no ability to The crop insurance programme serves as an apt
change it later or, at least, if changes are made, case to highlight the current deficiencies most
they are publicly viewable) and allow open legacy systems face when dealing with multiparty
verification by all participants. The “nodes” of business processes. While a blockchain should
the blockchain can be spread among various not necessarily be deployed merely for digitizing
government (including ombudsman bodies) and coordinating data, if an organization’s
and civil‑society stakeholders in a permissioned objective is to automate business processes in
blockchain network. Generally, assuming a decentralized and disintermediated manner
the integrity of data coming from the various for various reasons, then a blockchain‑based
agencies is to be trusted, this network can serve architecture becomes imperative. Crop insurance,
as a trustworthy, fault‑tolerant, common data as depicted above, requires a number of entities
store for participants. to share and verify information (about estimates
of crop damage, historical yield, local weather
– Reducing corrupt behaviour using smart information, estimated loss of production,
contracts: DLT‑based smart contracts can be neighbouring farms and their yield data etc.). In
programmed to automatically transfer welfare most cases, the interests of these entities are
benefits to farmers based on whether the data not totally aligned, which is often by design,
relayed by legacy systems and/or stakeholders to ensure that mechanisms for checks and
satisfy preset conditions (e.g. a certain amount of balances work. However, the data relating to the
rainfall occurred). This largely eliminates human farmer, crop damage and yield history must be
discretion in settling claims and improves the agreed upon by all parties as it is integral to the
claim settlement time and cost via data‑driven use case. Disintermediation using blockchain
automation (as projected in Figure 1). allows consensus on data to be reached by all
parties (meaning, in this case, claims are settled
– Data validation through oracle systems: only when all parties agree on the extent of the
Since the programme depends on collecting crop damage). Additionally, blockchain also
data from multiple independent agencies, provides the capability of maintaining a high
concerns about the data’s validity and reliability level of data integrity (e,.g,. censorship‑resistant
are always present. Using oracles that draw data and tamper‑proof record‑keeping) and has
from multiple sources to process, validate and a built‑in high technical fault tolerance. As a
relay information from external systems to be result, well‑audited smart contracts can infuse
stored as immutable records on a blockchain the business processes with new capabilities
can help ensure that key datasets are securely to evaluate insurance claims faster and more
transmitted from authentic sources and validated accurately, and deliver benefits to the intended
against tampering before the execution of any recipients more quickly. A report by the Food
agreements occur. However, there still need to be and Agricultural Organization (FAO) outlines
protections for vulnerabilities with regards to data the opportunities of smart contracts and
sources being compromised. (This concept is legacy‑DLT interoperability in agriculture
explained in depth later in this white paper.) insurance and micropayments.
Bridging the Governance Gap: Interoperability for blockchain and legacy systems 7
FIGURE 4 FAO Report: Blockchain for Agriculture (2019)
Agricultural insurance systems in the Asia-Pacific region range from major public-sector programmes
in India and the Philippines through to public-private partnerships in China and the Republic of Korea
and finally to the purely private markets encountered in Australia and New Zealand and the non-formal
private mutual and community-based crop and livestock initiatives in Bangladesh, India and Nepal.
Low-cost agricultural insurance schemes are increasingly viewed as mechanisms for providing social
protection to the increasing numbers of people affected by floods or droughts and helping to lessen
the impacts they suffer as a result of such events. However, despite the multiple benefits, the rate of
adoption of insurance products by the rural poor still remains relatively low. The mechanisms that are
in place to validate claims and to effect payouts are still time-consuming and this is one of the reasons
for index-based insurance not being chosen as the first risk-mitigation strategy by smallholder farmers.
Index insurance based on smart contracts can automate and greatly simplify the process, thereby
facilitating instant payouts to the insured in the case of adverse weather incidents. Automatic data
Source: http://www. feeds provide continuous and reliable hyperlocal data to the contract, thereby eliminating the need for
fao.org/3/CA2906EN/ on-site claim assessment by the surveyor.
ca2906en.pdf
1.3 Limitations of DLT and challenges of integration
In addition to the above opportunities and benefits, it – Unclear regulation: As discussed in later
is important to consider the limitations of blockchain sections, one of the interoperability framework
and DLT and the trade‑offs that need to be made. pillars suggests using crypto‑economic
guarantees in relation to digital assets. In many
– Resource consuming and limited scalability: jurisdictions, clear regulatory standards are yet
A blockchain spends more resources on to be implemented for the use of such assets,
consensus protocols (computation and particularly in cross‑border transactions.
participating nodes) to reduce the risk of
double‑spend attack (more prominent in Blockchain technology is currently evolving, as is
permissionless public blockchains). Also, on the regulatory understanding of its wide‑ranging
large public blockchain networks, transaction implementations. This paper assumes that
verification takes time,; therefore, throughput readers understand the technology’s general
(the number of transactions being validated capabilities and limitations in detail, and it is not
and added to the ledger per second) is intended to be a framework to assess whether
currently lower than traditional systems. a specific use case is suitable for blockchain.
Technology researchers have been working Instead, once readers have already established
towards scalability solutions for permissionless that blockchain is desirable for their specific use
blockchains, although it is unclear when these case and business processes, this paper aims to
will be achieved. Permissioned blockchain spotlight the role of blockchain, smart contracts
networks generally do not have scalability or and oracles in accelerating the automation of such
resource consumption challenges. processes. It highlights how an oracle‑powered
abstraction layer introduced on top of legacy
– Data dissemination risk: Enterprises are systems and blockchains can improve the
concerned with unintended data dissemination integrity of the interoperability among systems.
risk on blockchains. Some of these concerns The recommendations made in the paper thus
can be partially or fully addressed through comprise fundamental pillars in building such an
private side chains (e.g. Constellation used in interoperability bridge between legacy and DLT
Quorum network)4 or advanced cryptography systems, so that all of the systems can be held to
techniques (e.g. zero‑knowledge proofs), but the same standard in terms of security and integrity.
may be costly to implement in the current
state. Baseline Protocol also tries to address
this concern by using zero‑knowledge proofs
to allow enterprises to prove that each
counterparty used the same common set of
records and functions within a shared business
process without internal data leaving their
respective databases.
Bridging the Governance Gap: Interoperability for blockchain and legacy systems 8
2 Definitions
Clear definition of blockchain terminology
is crucial to understanding and conveying
ideas precisely.
Bridging the Governance Gap: Interoperability for blockchain and legacy systems 9
Many of the terms used in this white paper are well – Since smart contracts run on the blockchain,
understood but tend to have context‑dependent they allow tamperproof (barring 51% attack) and
interpretations. The following list describes the automated execution of the parameters once
frequently used terms and articulates the context data is received
in which they should be interpreted within the text,
unless otherwise indicated. – Represent deterministic automation of a
business process involving multiple parties/
Blockchain network users; neither party can default on its obligations
– Put simply, a blockchain consists of a linked and get away with it or tamper with the process
chain that stores auditable data in units called once the smart contract is executed
blocks, where each block contains the data, its
own hash value (a unique cryptographic value) Oracles
and a pointer to the hash of the previous block – Oracles link between the physical world and
the blockchain – middleware infrastructure
– Transactions are stored as immutable records that connects blockchains and any external
in a distributed stateful ledger (timestamped off‑chain system – responsible for reading/
record‑keeping in chronological order) writing data from/to another legacy system
and blockchain
– On a blockchain network, transactions are
automatically validated and stored without the – Any number of oracles (nodes) can be
need for a centralized administrator combined to make up a decentralized oracle
network, which can aggregate the responses
– There are three type of blockchain networks:5 of each node in any manner (mean, median,
mode, weighted etc.) to ensure there is no
– Permissionless: Permissionless networks are single point of failure in the delivery of data,
those that anyone can join at any time, such improving data integrity – and thus providing
as Bitcoin or Ethereum liveness guarantees (that every input is
guaranteed to generate an output) and data
– Permissioned private: permissioned private manipulation protection
networks consist of a consortium of finite
and well‑defined entities that deploy, run and – May also have more advanced functions such
maintain all of the nodes. Generally, these as providing cryptographic proofs (verifiability of
networks are developed, and even maintained, the data), hardware modifications (using trusted
by a blockchain service provider. Examples: execution environments (TEE) to compute
the IBM Food Trust transactions), computation (producing privacy,
scalability) etc.
– Permissioned public: with permissioned
public network, entities initiate a network Distributed ledger technology (DLT)
and allow certain parties to join, if they – An overarching term that encompasses the
meet certain requirements, such as being family of blockchain technologies, including both
authenticated and compliant with regulations. permissionless and permissioned blockchains,
In these networks, the consortium is smart contracts and oracles. It also includes
self‑sufficient and does not need to rely on non‑blockchain architectures such as directed
a vendor. Examples: Alastria in Spain, led acyclic graphs
by an association of over 500 members; the
European Blockchain Services Infrastructure Abstraction layer
(EBSI) in Europe led by the European – Abstraction, as a computer science concept,
Union; and LACChain in Latin America and is an approach to preserving information that
the Caribbean, led by the Inter‑American is relevant in a given context and forgetting
Development Bank and its partners in the information that is irrelevant in that context.
programme; and the blockchain network of It is a process of ignoring temporal details or
the Energy Web Chain by the Energy Web attributes in the study of systems to focus
Foundation (EWF) consortium attention on details of greater importance.
For the limited context in this paper, an oracle
Smart contracts network that sits between all blockchains and
– Self‑executing agreements that are triggered legacy systems to pass data between the two
based on predefined and agreed events without seamlessly and securely is being referred to as
human intervention an abstraction layer
Bridging the Governance Gap: Interoperability for blockchain and legacy systems 103 Interoperability –
fundamental pillars
Approaches to develop legacy-DLT
interoperability should be open, universal and
emerge bottom-up from users rather than
being imposed as a top-down standard.
Bridging the Governance Gap: Interoperability for blockchain and legacy systems 11Interoperability in the DLT space is a broad concept. case also exists, whereby on‑chain information or
Much research has been undertaken towards some type of command from a smart contract is
enabling blockchain‑to‑blockchain interoperability, sent to an external system that uses it for further
which is implemented largely through inter‑ledger processing or to act in the real world.
transactions. These transactions are intended
to allow for digital asset exchange between DLT Ultimately, the full power of legacy‑DLT
platforms, with no data or assets from non‑DLT interoperability is unlocked when abstraction is
systems entering into the overall system.6 This used to blur the boundaries between the DLT
paper, in contrast, focuses on the exchange of data and the legacy environments.7 In the legacy‑DLT
and dependent transactions between DLT systems context, abstraction goes beyond blockchain
and legacy systems. middleware for data retrieval and exchange. It also
includes security properties. This level of confidence
The primary and predominant reason for DLT‑legacy that cross‑network (i.e. across legacy and DLT
interoperability is to enable smart contracts networks) processes will execute as written with a
(on‑chain) to use an oracle to fetch information from high degree of security and reliability will allow new
a legacy system (off‑chain), format it, validate it and use cases to be experimented with. These solutions
store it on the blockchain where it can be used to are referred to here, in this paper, as middleware
trigger some type of agreement. The reverse use (explained in more detail in the next section).
FIGURE 5 Connecting smart contracts with external legacy systems through interoperability
standards and legal frameworks
Decentralized apps Middleware
Legacy sytems
using smart contracts built on
Interoperability
standards
$ €
¥
Legal
framework
As represented in the figure above, the basic rules – Use a generalized middleware that can satisfy a
for interoperability and legal principles should wide variety of security and performance needs
be established if middleware solutions are to be across all IT systems
developed that will allow data/transaction exchange
between legacy and DLT solutions in compliance Further, it is important to mention that an approach
with existing jurisdictional laws. And as these for interoperability should emerge from the relevant
solutions can be developed by different players on stakeholders and users after experimenting with
different technologies, the underlying interoperability various models. Imposing an interoperability
standards should be flexible and comprehensive to standard from the top down may create negative
work across legacy systems and DLT solutions, and effects on the blockchain trilemma as it may
should ideally have the following characteristics: reduce decentralization by accepting (centralized)
data coming from outside. This would also create
– Supported by a large open‑source community security loopholes. When standards are used to
building on a secure middleware force interoperability, they may lock all market
players into an inferior technology, as Jean Tirole
– Support most blockchain networks and legacy underlined in a ground‑breaking article.8 Accordingly,
systems globally this paper outlines approaches (strategic pathways)
that organizations can adopt to arrive at an
– Embed protocols and technologies to verify ecosystem‑driven interoperability solution rather than
data integrity and counterparty performance prescribing a definitive set of standards.
Bridging the Governance Gap: Interoperability for blockchain and legacy systems 124 The importance of an
open‑source, decentralized
data validation middleware
for enterprise systems
Enterprises running large legacy systems
are the key to accelerating the adoption of
DLT-based smart contracts.
Bridging the Governance Gap: Interoperability for blockchain and legacy systems 13The fast pace of ongoing technology developments Enterprise systems can support the development
in the DLT space may overwhelm any enterprise of new or innovative services in an ecosystem if
or government agency. At times, it may seem like those systems can interact with other platforms
the existing system is already becoming outdated. and networks. This stands true for legacy‑to‑DLT
However, doing a fundamental revamp or even interoperability as well. However, enterprises and
replacing an existing system with a new system governments may innovate with DLT‑related services
may be an expensive and impractical strategy. at a slower pace if they are unable to connect legacy
As a general matter, any organization that is not and DLT systems securely and effectively. The ability
developing a roadmap on how to integrate future to easily integrate with any existing and/or future
DLT capabilities risks falling behind on innovation blockchain network and maintain strong guarantees
that could potentially be relevant for its industry. of integrity and determinism will provide governments
and enterprises with the ability to build across
A sensible approach that helps to mitigate and interact with counterparties operating in any
technology risk and switching costs is to adopt blockchain environment. Furthermore, organizations
an open‑source secure blockchain middleware, or regions may use different blockchain networks,
wherever the use of blockchain is relevant, that and integration and interoperability among them will
provides legacy systems with universal access to be valuable.
current and future blockchain environments without
needing to restructure or rebuild any mission‑critical This reality presents an immediate need for
internal backend infrastructure. In addition to open‑source blockchain middleware solutions that
universal connection, blockchain middleware might are able to connect information and transactions
provide some legacy systems with an “in‑house” across different blockchain and legacy systems.
method of developing higher security and reliability By serving as a universal communication layer,
standards onto their existing systems using this middleware also acts as a common software
various validation and filtering techniques such as repository where blockchain developers can share
decentralization (redundant confirmation of the same resources such as documentation, technical
data point in a trust‑less manner), reputation systems walkthroughs and information explaining how other
(filter oracles based on performance quality) and users can interact with their system. A standard,
technologically enforced financial incentives/penalties open medium for blockchains and systems to
(stake capital to back the quality of oracle services). connect with one another without permissions
However, such decentralized oracles may run the risk avoids integration bottlenecks such as requiring
of majority attacks (where more than 51% of oracle permission from administrators or needing
nodes collude to provide incorrect data), single two development teams to develop specific
source failure (in the case of a single source of data) documentation for each new use case.
and failure of on‑chain data confidentiality where
additional technological considerations are not taken In the coming sections, this paper attempts to
into account. The sections below discuss some of provide strategic pathways towards these goals of
the mechanisms to manage these risks. building interoperability between legacy and DLT
systems. Summarized, the proposed pathways are
as follows:
FIGURE 6 Summary highlight of the interoperability pathways recommended in the paper
- Contribute to or join a large open-source community, building common interoperability frameworks
Capacity-building
- Develop integration capability for all blockchains and DLT platforms (both public and private)
- Decentralization to improve security of data and systems
Data validation - Leverage oracle networks with reputation frameworks for identifying high-quality nodes
- Use middleware with access to authenticated data sources and credentials management capabilities
Build security and - Use crypto assets to provide economic and security guarantees
guarantees - Build on a generalized oracle network with multiple security layers for defence in depth
Bridging the Governance Gap: Interoperability for blockchain and legacy systems 144.1 Contribute to or join a large open‑source community
building on a common interoperability framework
Much like the internet, collectively adopting a time/resources as every single integration requires
standard open network benefits everyone by communicating with a select few administrators to
creating much larger network effects, which in first get permission to integrate and then to write/
turn creates more value for each individual user. edit specific documentation.
Doing so requires a permissionless environment
that anyone can build on, access and use in Second, open‑source networks are far better
whatever form they need for their specific use case. suited to interoperating among multiple different
Interoperability cannot reach its full potential without stakeholders as no one party gains an unfair
being an open network, as it will hit a development advantage from owning the IP or sole development
choke point if all connection points must rely rights. Users feel more assured by being able
on a single permissioned middleware. A closed to see and verify the codebase they are using
network may also be subject to political stand‑offs to secure large amounts of value, knowing their
whereby certain enterprises will not join consortium competitor doesn’t have a special backdoor/
networks operated by competitors or will refuse privileged access into it. This approach also allows
to write documentation for certain competitors, for easy modifications and improved security, as
thereby limiting access or causing delays. Such open‑source code means more developers can
an approach will likely result in a fragmented, work on the same codebase, pruning any bugs and
unscalable system of many different intranets, expanding capabilities that benefit everyone equally.
instead of the one global internet we all enjoy today.
However, such an open approach doesn’t have to
A standard open‑source framework provides come at the expense of regulatory frameworks, as
several advantages. First, blockchain developers governments can build those on top as different
need to provide only one set of documentation pluggable solutions to ensure compliance based on
describing how other systems can talk to their their own local laws. In much the same way thatas
blockchain using secure middleware. This allows the internet is a single network that is modular and
any external system to operate on that blockchain can be adapted to fit certain legal requirements, an
(via the middleware) in a permissionless manner by open interoperability network can simultaneously
simply following the documentation. In contrast, support multiple legal frameworks without
closed‑source middleware is extremely difficult to cross‑dependencies on other localities.
scale and requires an extensive amount of extra
Bridging the Governance Gap: Interoperability for blockchain and legacy systems 15Strategies to adopt open‑source culture and community
Cultivate solutions based on an Use tools and platforms to build Facilitate government adoption of
open‑source technology that is open‑source communities of developers, time‑tested open‑source software that
generalized and flexible enough to researchers, enterprises, users, has been written and improved upon by
accommodate current and future governments, node operators and other a global user base of researchers and
needs and is permissionless in terms relevant actors to participate in open developers. Mission‑critical software
of integration. This means it must be discussions and express their unique systems become more functional when
upgradeable and not enforce a particular needs. Write documentation on how external applications can build additional
development framework or technology to interact with the middleware, which features for them and more secure when
solution on users. then allows everyone to connect to their using a global pool of talent to contribute
systems easily and uniformly. and review the codebase.
Case study: Governments joining the open‑source movement
Governments are increasingly using open‑source development practices to write their software. For example, the number of
government employees using Github (a collaborative platform to write software projects) has continued to increase. Linux, an
open‑source operating system, powers many government servers around the world, including the US Army, which has the largest
installed base for Red Hat Linux. Recently, the Government of India put out the codebase of its contact tracing app Aarogya Setu,
which has more than 130 million users, in open source.
FIGURE 7 Government organizations on GitHub
500
400
300
200
100
0
2009 2010 2011 2012 2013 2014
4.2 Adopt an oracle network that operates across all
blockchain environments
To fast‑track the ability for: (1) data providers to share Having blockchain middleware running across
data and services across all of the various blockchain all blockchain networks allows enterprises to
environments; and (2) legacy systems to operate on synchronize blockchain applications, legacy tools
any chain, limiting the upfront set‑up requirements and internal databases into one trackable and
and costs is vital. If data providers and legacy reliable operation. It also provides government
systems have to spend time and allocate resources entities running legacy digital infrastructure with
to integrate separately with each new blockchain the ability to rapidly scale up adoption of public
environment, they are likely to be very slow to bring programmes and deploy compliance standards
their data and services into blockchain ecosystems. across different blockchains through a single
gateway. andofingingThe collective use of a
Multiparty business processes may be difficult to common middleware by users from different
transfer on to DLT platforms unless the various blockchains, enterprises and governments reduces
distributed counterparties can all support multiple the costs of providing and accessing data and
different blockchain environments, allowing for services for everyone – achieved via shared financial
accommodation to partners’ preferred platforms. support of the node operators running interoperability
If certain blockchains are not available, there infrastructure. This benefits blockchains through
will be gaps in operations that affect the ability the cultivation of more data‑rich environments for
of other systems to properly interact: e.g. the developing applications that interface with legacy
supply‑chain solution can’t easily talk to the infrastructure, while equally benefiting legacy systems
finance system, which then interferes with the that can now provide data and services to users
trade finance network. across all kinds of DLT networks.
Bridging the Governance Gap: Interoperability for blockchain and legacy systems 16Strategies to integrate multiple DLT/network capability
Have an oracle network that can run Implement open‑source blockchain Use that same abstraction layer to allow
natively on each blockchain so that middleware that sits as an abstraction layer enterprises to connect to all of the different
oracle services are subject only to the above all of the blockchains, providing a chains and other legacy systems from a
throughput and security of a blockchain. universal gateway for data providers/node single integration. This vastly reduces the
This allows blockchains to customize their operators to transact with all chains from a integration work with external systems
oracle solution to fit that blockchain. single framework. This reduces friction for and consolidates internal operations by
data providers to set up on different chains, ensuring that all of the companies’ systems
lowering the costs for everyone. are synced up together.
FIGURE 8 Case study: Interoperability among multiple blockchains – Cosmos and Polkadot
Case study: Interoperability among multiple blockchains –
Cosmos and Polkadot
Parachains
Polkadot and Cosmos are two projects working on
interoperability between blockchains.
Relay
Polkadot connects several chains together in a single chains
network, allowing them to process transactions in parallel and
exchange data between chains with security guarantees. It
uses a network of heterogeneous blockchain shards called Bridge
parachains. These chains connect to and are secured by the
Polkadot Relay Chain. They can also connect with external
networks via bridges.
(Source: https://polkadot.network/Polkadot‑lightpaper.pdf)
Cosmos’s architecture is based on a “hub‑and‑spoke” system
whereby a series of different blockchain chains connect to a
“central” hub by means of inter‑blockchain communication.
The goal of Cosmos is to break the barriers between
blockchains by allowing them to transact with each other. This
vision is achieved through a set of open‑source tools such
as Tendermint (a byzantine fault tolerant [BFT] consensus
algorithm), the Cosmos SDK and IBC, which allows building
interoperable blockchain applications. Note that Cosmos is an
open‑source community project.
(Source: https://cosmos.network/intro)
Bridging the Governance Gap: Interoperability for blockchain and legacy systems 174.3 Use decentralization to validate the integrity
of data inputs and provide highly available,
tamperproof data exchange
Decentralization is one of the core tenets of security with the defined rules. This provides network users
for blockchain applications, wherein each node in with deterministic computation (computation which,
the decentralized network is financially incentivized given an input, will always produce the same
to independently validate each transaction based output), data integrity and liveness guarantees
on a common set of protocol rules. In a sufficiently for the transactions they send to the network for
decentralized network with proper incentives for processing.
honest reporting, the network’s consensus will
overpower any transactions that do not comply
FIGURE 9 Two‑level distribution of data sources and nodes for oracles
Data source 1
Node 1
Data source 2
ORACLE Node 2
contract
Data source 3
Node 3
source: https://link. Data source 4
smartcontract.com/
whitepaper
Decentralization of oracles should occur at both the multiple data sources), the inputs and outputs of a
node and data source levels (as shown in Figure digital agreement can become as tamper‑resistant
2), without compromising the quality of any one as the smart contract itself.
component (e.g. incorporating low‑quality data), to
ensure there are multiple layers of redundancies. For governments, this becomes especially
Decentralized oracle networks remove any single imperative where citizen services and claims can
point of failure in the delivery of data to prevent the be verified through an independent validation
smart contract from relying on any one single node system that records their request on a DLT, and the
or source of truth. governments/enterprises servicing that request are
bound by non‑repudiation (assurance that entities
For further guarantees about the integrity of data, cannot deny any information they have signed and
especially in situations of a single data source, provided). When combining decentralization with
specialized cryptographic proofs can be provided high‑quality nodes filtered through a reputation
that verify the authenticity of the data point’s origin. system (described below), citizens can receive
This reassures users that external data came stronger guarantees on certain government services
directly from a particular web server and was not by having them validated by a decentralized
tampered with en route, as only the specific web network, while governments can reduce manual
server can provide a unique signature proving its and complex coordination processes in relation to
origin. By combining this cryptographic technique information and asset transfers by offloading them
with decentralization (e.g. multiple nodes and to a reliable decentralized network of oracles.
Bridging the Governance Gap: Interoperability for blockchain and legacy systems 18Strategies to develop decentralized data validation
Apply the security model of Aggregate data from multiple high‑quality Select a multitude of independent
decentralization to oracles relaying data sources, retrieving the same data high‑quality oracle node operators that
data between on‑chain and off‑chain multiple times asynchronously and/or have been security reviewed and meet
systems. Decentralized oracle networks incorporating dispute resolution processes the collective standards set out by the
use multiple independent/Sybil‑ resistant to safeguard against a single data source stakeholders involved, such as a strong
nodes (mechanisms preventing or single API call being the sole arbiter of performance history of delivering reliable
nodes creating multiple identities from truth for a smart contract. This approach services, known registered entities run by
influencing the network) to validate the provides multiple layers of redundancy experts such as leading DevOps, and any
same off‑chain data point, providing and ensures that if a data provider or node other consideration deemed important.
liveness and preventing a single oracle goes offline or fails to deliver the data, the
from corrupting the data. overall data feed is still reliable.
Case study: Crypto price referencing using decentralized oracle networks: Chainlink and MakerDAO
Chainlink, a framework for decentralized oracles, supports decentralized oracle networks that feed tamper‑proof price reference data
to smart contracts on‑chain. Similar to blockchains, which use decentralized computation to create data integrity, oracle networks
such as Chainlink employ the same decentralization model to aggregate price data and store it on‑chain, ensuring it is highly
accurate, available and resistant to manipulation. Using a decentralized network of security‑reviewed oracle nodes and data sourced
from off‑chain data providers, Chainlink maintains an on‑chain price feed that updates every time there is a small deviation from the
latest price stored on‑chain. Blockchain applications can read the on‑chain price data and use it to execute automated functions,
such as ensuring a loan is fully collateralized or settling a derivatives contract.
(Source: https://chain.link/)
MakerDAO, a decentralized stablecoin protocol, uses oracles to fetch asset prices of supported collaterals. The reference price is
calculated via a medianizer – a smart contract that collates price data from several external independent price‑feed operators. These
operators monitor the asset prices from external sources. Third‑party relayers are then used to forward their prices on‑chain.
(Source: https://makerdao.com/)
FIGURE 10 A visualization of the decentralized oracle network supporting the Chainlink BTC/$ price feed
Chorus One
Chainlayer $9105.50 Easy 2 stake
$9099.54 $9109.4
Certus One
Fiews
$9098.869
$9102.82
B Harvest LinkForest
$9114.528 $9098.56
LinkPool
Anyblock $9102.82
$9105.501
Newroad
Alpha Vantage $9114.528
$9098.57 $9105.501
Omniscience
$9109.4
Ztake.org
$9105.501 P2P.org
$9138.476
Wetez
$9098.57 Prophet
$9138.476
Validation Capital
Secure Data Links
$9114.528
$9138.476
Staking Facilities
stake.fich Simply VC
$9098.632
$9104.4 $9102.82
Bridging the Governance Gap: Interoperability for blockchain and legacy systems 194.4 Demand oracles that support access to
authenticated data sources and credentials
management capabilities
Smart contracts use data to directly execute Data providers have two primary options for how
automated business processes without a human to make data available to blockchain networks:
intermediary. Thus, data quality is an extremely provide data to an existing oracle network and/or
critical component in determining the smart run their own oracle node to provide origin‑signed
contract’s security and reliability to avoid a “garbage data directly to smart contracts. Through an existing
in, garbage out” scenario. secure and reliable oracle network, data and API
providers do not have to change anything about
Data quality may take capital to create – often their existing business model. The oracle nodes can
the result of large network effects, unique pay the data providers in fiat currency using typical
business processes and/or an intelligent team of payment plans in use today.
data analysts. Incentives must exist to maintain
the production of high‑quality data, meaning Alternatively, data and API providers can run their
that there must be a framework for handling own oracles to both sign data with their private
credentials (login, passwords) to limit access to key and sell directly to blockchain‑based smart
paying users for high‑quality authenticated data contracts. This provides them with a direct method
APIs. Without having credential management of participating in the blockchain ecosystem by
capabilities built into an oracle system, data is earning additional revenue from selling directly into
at risk of being pirated, thus limiting the data new blockchain markets, as well as adding additional
provider’s ability to generate revenue and continue security to their data by cryptographically signing it
generating high‑quality data. Generally, free data when broadcasted to a blockchain. Governments
APIs lack incentives to keep data up to date and enterprises can benefit from such a method by
and machine‑readable, making them potentially running their own oracle nodes, providing users with
unreliable and insecure as inputs to trigger the strong guarantees that the data and services they
movement of large amounts of value. provide came directly from official channels.
Strategies to integrate high‑quality data sources
Develop/integrate highly secure credential Use decentralized oracle networks to Use oracle networks that keep on‑chain
management capabilities to access improve data quality by aggregating data data fresh and reflective of real‑time
high‑quality data providers and enterprise from multiple data sources to provide a conditions in an automated manner
systems such as web APIs, internet of single source of data that is likely to be while retaining key security properties.
things (IoT) networks, CRM/ERP systems accurate. Aggregation techniques can Oracle networks should also be flexible to
and various other legacy systems that be customizable (average, weighted etc.) support on‑demand requests for various
require authorized logins. and performed cost‑effectively off‑chain types of off‑chain data and API services.
with supporting cryptographic proofs that
are provable on‑chain.
Case study: Use of APIs
Making real‑world data available on‑chain requires oracle networks that are compatible with the existing API infrastructure already
widely used throughout global economic systems by businesses, enterprises and governments. To ensure an accelerated and
seamless transition, this means oracle networks need to support access to credentialed systems without any changes to the current
API systems, as well as a framework for API providers to run trusted oracles in order to sign their own data and sell it directly. A
robust blockchain oracle solution should enable data providers to both sell their APIs to oracle networks and publish signed data
directly on‑chain without any changes to their existing systems.
Bridging the Governance Gap: Interoperability for blockchain and legacy systems 20FIGURE 11 A visualization of predominant API ecosystem
Apis
Developer Developer
Free pays Indirect
gets paid
Pay as Unit Transaction Content Content Internal
Tiered Freemium Rev-share Affiliate SAAS
you go based fee acquisition syndication use
Sign-up Consumer- Internal-
CPA CPC Included Upsell
referral facing facing
One-time Recurring Public Mobile / Web /
SOA+
fees devices mobile
Case study: Ramp Network uses open banking APIs to execute crypto-to-fiat transactions
Ramp Network is one of the licensed players under the second Payment Services Directive (PSD2) for open banking (European
banks). In order to provide peer‑to‑peer, crypto‑to‑fiat transactions (using atomic swaps), it uses a payment oracle to verify a buyer’s
payment and send a proof‑of‑payment to the network. Here, the data source is single (just the user’s bank) and privacy requirements
are higher to ensure the user’s identity and transaction history aren’t made public.
(source: https://swaps.ramp.network/)
Bridging the Governance Gap: Interoperability for blockchain and legacy systems 21You can also read
