Scientifi c Computing Data Visualization Information Technology Quantum Computing - JULY 2020

Scientifi c Computing Data Visualization Information Technology Quantum Computing - JULY 2020
            Data Visualization

JULY 2020     
Scientifi c Computing Data Visualization Information Technology Quantum Computing - JULY 2020
Society Has
You Covered!
ahead of the curve by attending one of our
200+ globally recognized conferences.

DIGITAL LIBRARY — Easily access over 780k
articles covering world-class peer-reviewed
content in the IEEE Computer Society
Digital Library.

opportunities to write and present your
ground-breaking accomplishments.

EDUCATION — Strengthen your resume
with the IEEE Computer Society Course
Catalog and its range of offerings.

new positions posted in the IEEE Computer
Society Jobs Board.

NETWORK — Make connections that count
by participating in local Region, Section,
and Chapter activities.

Explore all of the member benefits
at today!
Scientifi c Computing Data Visualization Information Technology Quantum Computing - JULY 2020

   Editor                                                                           Publications Portfolio Managers
   Cathy Martin                                                                     Carrie Clark, Kimberly Sperka

   Publications Operations Project Specialist                                       Publisher
   Christine Anthony                                                                Robin Baldwin

   Production & Design Artist                                                       Senior Advertising Coordinator
   Carmen Flores-Garvey                                                             Debbie Sims

   Circulation: ComputingEdge (ISSN 2469-7087) is published monthly by the IEEE Computer Society. IEEE Headquarters, Three Park Avenue, 17th
   Floor, New York, NY 10016-5997; IEEE Computer Society Publications Office, 10662 Los Vaqueros Circle, Los Alamitos, CA 90720; voice +1 714 821 8380;
   fax +1 714 821 4010; IEEE Computer Society Headquarters, 2001 L Street NW, Suite 700, Washington, DC 20036.
   Postmaster: Send address changes to ComputingEdge-IEEE Membership Processing Dept., 445 Hoes Lane, Piscataway, NJ 08855. Periodicals Postage
   Paid at New York, New York, and at additional mailing offices. Printed in USA.
   Editorial: Unless otherwise stated, bylined articles, as well as product and service descriptions, reflect the author’s or firm’s opinion. Inclusion in
   ComputingEdge does not necessarily constitute endorsement by the IEEE or the Computer Society. All submissions are subject to editing for style,
   clarity, and space.
   Reuse Rights and Reprint Permissions: Educational or personal use of this material is permitted without fee, provided such use: 1) is not made for
   profit; 2) includes this notice and a full citation to the original work on the first page of the copy; and 3) does not imply IEEE endorsement of any third-
   party products or services. Authors and their companies are permitted to post the accepted version of IEEE-copyrighted material on their own Web
   servers without permission, provided that the IEEE copyright notice and a full citation to the original work appear on the first screen of the posted copy.
   An accepted manuscript is a version which has been revised by the author to incorporate review suggestions, but not the published version with copy-
   editing, proofreading, and formatting added by IEEE. For more information, please go to:
   /rights/paperversionpolicy.html. Permission to reprint/republish this material for commercial, advertising, or promotional purposes or for creating new
   collective works for resale or redistribution must be obtained from IEEE by writing to the IEEE Intellectual Property Rights Office, 445 Hoes Lane,
   Piscataway, NJ 08854-4141 or Copyright © 2020 IEEE. All rights reserved.
   Abstracting and Library Use: Abstracting is permitted with credit to the source. Libraries are permitted to photocopy for private use of patrons,
   provided the per-copy fee indicated in the code at the bottom of the first page is paid through the Copyright Clearance Center, 222 Rosewood Drive,
   Danvers, MA 01923.
   Unsubscribe: If you no longer wish to receive this ComputingEdge mailing, please email IEEE Computer Society Customer Service at help@ and type “unsubscribe ComputingEdge” in your subject line.
   IEEE prohibits discrimination, harassment, and bullying. For more information, visit

   IEEE Computer Society Magazine Editors in Chief

   Computer                                            IEEE Intelligent Systems                            IEEE Pervasive Computing
   Jeff Voas, NIST                                     V.S. Subrahmanian,                                  Marc Langheinrich, Università
                                                       Dartmouth College                                   della Svizzera italiana
   Computing in Science
   & Engineering                                       IEEE Internet Computing                             IEEE Security & Privacy
   Lorena A. Barba (Interim),                          George Pallis, University                           David Nicol, University
   George Washington University                        of Cyprus                                           of Illinois at
   IEEE Annals of the History                          IEEE Micro
   of Computing                                        Lizy Kurian John, University                        IEEE Software
   Gerardo Con Diaz, University                        of Texas at Austin                                  Ipek Ozkaya, Software
   of California, Davis                                                                                    Engineering Institute
                                                       IEEE MultiMedia
   IEEE Computer Graphics                              Shu-Ching Chen, Florida                             IT Professional
   and Applications                                    International University                            Irena Bojanova, NIST
   Torsten Möller,
   Universität Wien

2469-7087/20 © 2020 IEEE                              Published by the IEEE Computer Society                                         July 2020                   1
Scientifi c Computing Data Visualization Information Technology Quantum Computing - JULY 2020

                                Weather Report: A
        Scientific            Site-Specific Artwork       Bringing
       Workflows                      Interweaving          NASA
  into Computer              Human Experiences        Missions to
       Programs                and Scientific Data     the Public
Scientifi c Computing Data Visualization Information Technology Quantum Computing - JULY 2020
Scientific Computing
                          8      Metamorphic Testing: A Simple Yet Effective Approach
                                for Testing Scientific Software
                              UPULEE KANEWALA AND TSONG YUEH CHEN

                     16     SciPipe—Turning Scientific Workflows into Computer
                          OLA SPJUTH

                   Data Visualization
                 22   Weather Report: A Site-Specific Artwork Interweaving
                      Human Experiences and Scientific Data Physicalization

              30   OpenSpace: Bringing NASA Missions to the Public

            Information Technology
           38    A Manifesto for Energy-Aware Software

         42    Next Generation IoT: Toward Ubiquitous Autonomous
              Cost-Efficient IoT Devices

      Quantum Computing
      46    Powerball and Quantum Supremacy

  50       Beyond Quantum Supremacy

  4     Magazine Roundup
  7    Editor’s Note: Advancing Science with Software
56    Conference Calendar

                                         Subscribe to ComputingEdge for free at
Scientifi c Computing Data Visualization Information Technology Quantum Computing - JULY 2020
Magazine Roundup

    T     he IEEE Computer Society’s lineup of 12 peer-reviewed technical magazines covers cutting-edge topics rang-
          ing from software design and computer graphics to Internet computing and security, from scientific appli-
    cations and machine intelligence to visualization and microchip design. Here are highlights from recent issues.

                                             In this article from the March/April     were researched, implemented,
                                             2020 issue of Computing in Sci-          and launched into the market-
    Challenges and Opportunities             ence & Engineering, the authors’         place, where their intense compe-
    in the Detection of Safety-              experience shows that it is often        tition transformed the 500-year
    Critical Cyberphysical Attacks           straightforward to first define the      tradition of printing and publish-
                                             multiple iterations of tests for per-    ing—placing the electronic literacy
    Cyberphysical     systems     (CPSs)     forming continuous simulations,          on the screens of billions of digital
    are increasingly used in various         and then keep multiple and even          displays, computers, tablets, and
    application domains and face the         competing metamorphic relations          smart phones around the world.
    threat of cyberphysical attacks. In      open for investigating the test-         Read more in this article from the
    this article from the March 2020         ing-result patterns. The authors         January–March 2020 issue of IEEE
    issue of Computer, the authors dis-      call this new approach exploratory       Annals of the History of Computing.
    cuss challenges in detecting these       MT, and they report their experi-
    attacks. They use power grids and        ence of applying it to detect bugs,
    surgical robots to clarify their anal-   mismatches, and constraints in
    ysis, and they use this analysis to      automatically calibrating parame-        Illustrating Changes in Time-
    identify ongoing challenges and          ters for the United States Environ-      Series Data with Data Video
    future research directions.              mental Protection Agency’s Storm
                                             Water Management Model.                  Understanding the changes of
                                                                                      time series is a common task in
                                                                                      many application domains. Con-
    Exploratory Metamorphic                                                           verting time-series data into vid-
    Testing for Scientific Software          The Font Wars, Part 1                    eos helps an audience with little
                                                                                      or no background knowledge gain
    Scientific model developers are          The Font Wars were a decades-            insights and deep impressions. It
    able to verify and validate their        long competition in the computer         essentially integrates data visual-
    software via metamorphic test-           industry for dominance in font           izations and animations to present
    ing (MT), even when the expected         technology, viewed as a key suc-         the evolution of data expressively.
    output of a given test case is not       cess factor for personal computing       However, it remains challenging to
    readily available. The tenet is to       platforms. At the heart of the Font      create this kind of data video. First,
    check whether certain relations          Wars was a fundamental question:         it is difficult to efficiently detect
    hold among the expected outputs          What is the best way to turn tradi-      important changes and include
    of multiple related inputs. Contem-      tional printed letter forms into digi-   them in the video sequence. Exist-
    porary approaches require that the       tal fonts for computer screens and       ing methods require much man-
    relations be defined before tests.       printers? Answers to this question       ual effort to explore the data and

4              July 2020                        Published by the IEEE Computer Society                2469-7087/20 © 2020 IEEE
Scientifi c Computing Data Visualization Information Technology Quantum Computing - JULY 2020
find changes. Second, how these                                                  autonomous driving. It is based
  changes are emphasized in the                                                    on a new system-on-a-chip (SoC)
  videos is also worth studying. A         Container NATs and Session-             that integrates industry-standard
  video without emphasis will hinder       Oriented Standards: Friends             components such as CPUs, ISP,
  an audience from noticing those          or Foe?                                 and GPU, with custom neural net-
  important changes. This article                                                  work accelerators. The FSD com-
  from the March/April 2020 issue of       This article from the November/         puter is capable of processing up
  IEEE Computer Graphics and Appli-        December 2019 issue of IEEE Inter-      to 2,300 frames per second, which
  cations presents an approach that        net Computing highlights issues         is a 21× improvement over Tesla’s
  extracts and visualizes important        that arise when deploying network       previous hardware and at a lower
  changes of a time series.                address translation middle-boxes        cost. When fully utilized, it enables
                                           through containers. The authors         a new level of safety and autonomy
                                           focus on Docker as the container        on the road. Read more in this arti-
                                           technology of choice and pres-          cle from the March/April 2020 issue
  Research on Road Traffic                 ent a thorough analysis of its net-     of IEEE Micro.
  Situation Awareness System               working model, with special atten-
  Based on Image Big Data                  tion to the default bridge network
                                           driver that is used to implement
  Road traffic is an important com-        network address translation func-       Metric Learning-Based
  ponent of the national economy           tionality. They discuss some unex-      Multimodal Audio-Visual
  and social life. Promoting intelli-      pected shortcomings and elabo-          Emotion Recognition
  gent and Informa ionization con-         rate on the suitability of containers
  struction in the field of road traffic   for deploying services based on         People express their emotions
  is conducive to the construction         the Interactive Connectivity Estab-     through multiple channels, such
  of smart cities and the formula-         lishment standard protocol. To          as visual and audio ones. Conse-
  tion of macro-strategies and con-        support their findings, they present    quently, automatic emotion rec-
  struction plans for urban traffic        experiments that they conducted         ognition can be significantly ben-
  development. Aiming at the short-        in a real-world operational environ-    efited by multimodal learning.
  comings of the current road traffic      ment, namely a WebRTC service           Even though each modality exhib-
  system, this article from the Jan-       based on the Janus media server.        its unique characteristics, multi-
  uary/February 2020 issue of IEEE                                                 modal learning takes advantage of
  Intelligent Systems—on the basis                                                 the complementary information of
  of combining convolution neu-                                                    diverse modalities when measur-
  ral networks (CNNs), situational         Compute Solution for Tesla’s            ing the same instance, resulting in
  awareness, databases, and other          Full Self-Driving Computer              enhanced understanding of emo-
  technologies—takes the road traf-                                                tions. Yet, their dependencies and
  fic situational awareness system         Tesla’s full self-driving (FSD) com-    relations are not fully exploited in
  as its research object and analyzes      puter is the world’s first pur-         audio–video emotion recognition.
  the information collection, pro-         pose-built   computer      for   the    Furthermore, learning an effective
  cessing, and analysis process.           highly demanding workloads of           metric through multimodality is a                                                                                            5
Scientifi c Computing Data Visualization Information Technology Quantum Computing - JULY 2020

    crucial goal for many applications    PedibusSmart, SafePath, and Kids-         resourcing. In this article from
    in machine learning. Therefore,       GoGreen. It reports on four years         the March/April 2020 issue of IEEE
    in this article from the January–     of success with more than 1,800           Software, the authors define a
    March 2020 issue of IEEE Multi-       elementary-age       children,    their   systematical service design con-
    Media, the authors propose multi-     teachers, and their families. The         cept to enable all stakeholders to
    modal emotion recognition metric      authors further show how (i) dis-         achieve better outcomes in co-
    learning (MERML), learned jointly     appearing, pervasive technology           creation activities.
    to obtain a discriminative score      contributes to successful adop-
    and a robust representation in a      tion; (ii) properly balancing trust
    latent-space for both modalities.     and tracking leads to useful, nonin-
    The learned metric is efficiently     vasive technological support; and         Detecting Online Content
    used through the radial basis func-   (iii) in-classroom, gameful technol-      Deception
    tion (RBF)-based support vector       ogy engages and motivates partici-
    machine (SVM) kernel. The eval-       pation, with behavior changes per-        The surge of deceptive content
    uation of the framework shows a       sisting over time.                        (such as fake news) in the past few
    significant performance, improv-                                                years has made content decep-
    ing the state-of-the-art results on                                             tion an important area of research.
    the eNTERFACE and CREMA-D                                                       The authors of this article from the
    datasets.                             The Need for New                          March/April 2020 issue of IT Pro-
                                          Antiphishing Measures                     fessional identify two main types
                                          against Spear-Phishing                    of content deception based on
                                          Attacks                                   either fake content or misleading
                                                                                    content. They present a classifi-
    CLIMB: A Pervasive Gameful            In this article from the March/April      cation of deception attacks along
    Platform Promoting Child              2020 issue of IEEE Security & Pri-        with delivery methods. They also
    Independent Mobility                  vacy, the authors provide extensive       discuss defense measures that can
                                          analysis of the unique characteris-       detect deception attacks. Finally,
    Child independent mobility (CIM)      tics of phishing and spear-phishing       they highlight some outstanding
    refers to the freedom and capa-       attacks, argue that spear-phish-          challenges in the area of content
    bility of children to move about      ing attacks cannot be well cap-           deception.
    their local neighborhoods with-       tured by current countermeasures,
    out constant direct adult supervi-    identify ways forward, and analyze
    sion. Our CLIMB project combats       an advanced spear-phishing cam-
    an observed decline in CIM, offer-    paign targeting white-collar work-
    ing a pervasive gameful platform      ers in 32 countries.                         Join the IEEE
    for home–school mobility com-                                                      Computer
    posed of three primary compo-                                                      Society
    nents: the first two using technol-
    ogy to support different levels of    Three Phases of Transforming
    child independence and the third      a Project-Based IT Company
    providing an element of continu-      into a Lean and Design-Led
    ous motivation for positive behav-    Digital Service Provider
    ior change. This article from the
    January–March 2020 issue of IEEE      Digital transformation requires
    Pervasive Computing describes         a continuous review of value
    these three novel technologies:       creation,   value      capture,   and

6               ComputingEdge                                                                                    July 2020
Scientifi c Computing Data Visualization Information Technology Quantum Computing - JULY 2020
Editor’s Note

   Advancing Science
   with Software

  S      cientists increasingly use
         software to facilitate discov-
   eries through simulation and anal-
                                             Also from Computing in Science
                                             & Engineering, “SciPipe—Turning
                                             Scientific Workflows into Com-
                                                                                    people better understand NASA’s
                                                                                    missions and discoveries.
                                                                                        Whether scientific or com-
   ysis. Scientific software can enable      puter Programs” discusses a tool       mercial, software should be as
   research that might be imprac-            for automating complex scientific      energy-efficient as possible. The
   tical or impossible using experi-         computations involving multiple        authors of IEEE Software’s “A Man-
   mentation, observation, or theory.        programs.                              ifesto for Energy-Aware Software”
   Despite its importance, challenges            Scientific data can be chal-       argue that software developers
   remain in terms of testing, reli-         lenging to not only analyze but        must consider energy consump-
   ability, and reusability. This Com-       also convey to the public. This        tion when designing programs.
   putingEdge issue presents tools           ComputingEdge       issue   includes   IEEE Pervasive Computing’s “Next
   for helping scientists effectively        two examples of creative scien-        Generation IoT: Toward Ubiqui-
   utilize software in their research.       tific data visualizations from IEEE    tous Autonomous Cost-Efficient
      “Metamorphic Testing: A Sim-           Computer Graphics and Appli-           IoT Devices” also emphasizes the
   ple Yet Effective Approach for            cations. In “Weather Report: A         need for energy efficiency, specif-
   Testing Scientific Software,” from        Site-Specific Artwork Interweav-       ically in IoT devices.
   Computing in Science & Engi-              ing Human Experiences and Sci-             This issue of ComputingEdge
   neering, aims to help scientists          entific Data Physicalization,” the     concludes with two Computer
   who have limited training in soft-        authors describe a public art          articles on quantum supremacy.
   ware development employ sci-              installation that educates on cli-     “Powerball and Quantum Suprem-
   entific   software      successfully.     mate change. In “OpenSpace:            acy” teaches quantum concepts
   The authors explain that meta-            Bringing NASA Missions to the          using a lottery metaphor. “Beyond
   morphic testing is a good tech-           Public,” the authors present a visu-   Quantum Supremacy” identifies
   nique for testing exploratory soft-       alization program for use in inter-    advances that are needed to move
   ware with inherent uncertainties.         active museum displays that helps      quantum computing forward.

2469-7087/20 © 2020 IEEE                   Published by the IEEE Computer Society                    July 2020            7
Scientifi c Computing Data Visualization Information Technology Quantum Computing - JULY 2020
                                                                                                      This article originally
                                                                                                                appeared in

    Metamorphic Testing:                                                                                  vol. 21, no. 1, 2019

    A Simple Yet Effective Approach
    for Testing Scientific Software
    Upulee Kanewala, Montana State University
    Tsong Yueh Chen, Swinburne University of Technology

        Testing scientific software is a difficult task due to their inherent complexity and the
        lack of test oracles. In addition, these software systems are usually developed by end-
        user developers who are not normally trained as professional software developers nor
        testers. These factors often lead to inadequate testing. Metamorphic testing (MT) is a
        simple yet effective testing technique for testing such applications. Even though MT is a
        wellknown technique in the software testing community, it is not very well utilized by the
        scientific software developers. The objective of this paper is to present MT as an effective
        technique for testing scientific software. To this end, we discuss why MT is an appropriate
        testing technique for scientists and engineers who are not primarily trained as software
        developers. Specifically, how it can be used to conduct systematic and effective testing
        on programs that do not have test oracles without requiring additional testing tools.

     WHAT MAKES TESTING SCIENTIFIC                                  due to one-off errors,1 compromised performance in
     SOFTWARE DIFFICULT?                                            coordinate measuring machines (CMMs) due to soft-
    Scientific software is widely used for making criti-            ware faults,2 and geoscience software systems pro-
    cal decisions in various scientific and engineering             ducing seemingly correct yet different results that
    domains. For example, simulations are often used                are hard to categorize as incorrect. 3 Previous works
    in place of physical experiments due to the time                also report situations where software faults cause
    and cost constraints associated with conducting                 retractions of published work.4 Testing is the most
    physical experiments. Furthermore, decisions made               widely used approach for quality assurance of soft-
    by these software systems can affect day-to-day                 ware. But some inherent characteristics in scientific
    human life, such as predictions made by climate                 software make it difficult to conduct systematic test-
    models. Thus, it is important to make sure that these           ing in these programs. 5
    software systems are producing the correct results.
    Previous studies have reported many instances                      ›› Correct answers are often unknown. Typically,
    where scientific software systems were affected                       scientific software is exploratory in nature
    by faults such as seismic programs losing precision                   and due to this, the correct results are often
                                                                          unknown. If the result is known there would be
                                                                          no need to develop the software. In such situa-
    DOI No. 10.1109/MCSE.2018.2875368                                     tions, only bounds or ranges of solutions might
    Date of publication 18 October 2018; date of current version          be available. Typically, in testing, an expected
    6 March 2019.                                                         output is used to decide test case passing

8               July 2020                          Published by the IEEE Computer Society              2469-7087/20 © 2020 IEEE

  FIGURE 1. MT process. ts: source test case, tf : follow-up test case, os: output of the source test case and of: of : output of the
  follow-up test case.

        or failing. This would make it challenging to                   input list has two million real numbers. How can we
        conduct systematic testing in these programs.                   know the returned average is correct or not? Though
     ›› Practically difficult to validate the computed                  we are not able to validate the computed average in
        output. Scientific software often implements                    this case, we do know some relationships between the
        mathematical models that involve complex                        outputs of some related inputs. For example, consider
        calculations. Furthermore, they tend to produce                 a new list of real numbers, which is a permuted list of
        complex outputs. Both these characteristics                     the original list of real numbers, or which consists of
        make it hard to determine the correctness                       four million real numbers by duplicating the original list
        of the produced output of the software. This                    of real numbers. For either of the new lists of real num-
        makes it challenging to use automated test                      bers, the new average is expected to be the same as
        case generation approaches such as random                       the old average (subject to some round-off error toler-
        test generation since the output of such test                   ance). If the new and old averages are not the same,
        cases are difficult to validate.                                then we know that the program of computing average
     ›› Inherent uncertainties. Often scientific software               has bugs. This is the intuition of MT.
        is written to simulate models with inherent                         In software testing, passing, or failing of a test case
        uncertainties. For some of these scientific                     is decided using a test oracle and it is an essential com-
        programs, there may be more than one possible                   ponent for conducting systematic testing. MT uses
        output. This makes it challenging to conduct                    metamorphic relations (MRs) to determine whether
        testing on these programs.                                      a test case has passed or failed. An MR specifies how
     ›› Choosing suitable tolerances. Scientific                        the output of the program is expected to change when
        software systems often involve complex                          a specified change is made to the input. The following
        floating-point computations. Thus, specifying                   is the typical process for applying MT to a given pro-
        the acceptable tolerance for the expected                       gram and Figure 1 depicts this process:6
        output in test cases is difficult.
     ›› Incompatible testing tools. Programming                             1. MR identification: identifying MRs for the
        languages such as FORTRAN are widely used                              program under test can be done based on the
        in the scientific community for developing                             specification.
        scientific software. However, testing tools                         2. Source test case creation and execution: com-
        are usually developed for languages such as                            monly used test generation techniques such
        JAVA and C++ that are commonly used by                                 as random, structural coverage, or fault-based
        the software engineering community. Thus,                              test input generation can be used. Then the
        these testing tools are not effective for testing                      generated source test cases are executed on
        scientific programs.                                                   the program under test.
                                                                            3. Follow-up test case creation: use the MRs
   WHAT IS METAMORPHIC                                                         identified in Step 1 to transform the source test
   TESTING (MT)?                                                               case to obtain the follow-up test case.
  Consider a program that will accept a list of real num-                   4. Follow-up test case execution: Execute the
  bers and compute their average. Suppose that the                             follow-up test case and compare the outputs                                                                                                         9

     FIGURE 2. A JUnit test script that uses MT approach to test a matrix multiplication function in the JScience library.

                                                                                     1 2                 2 1
            of the source and follow-up test cases to verify                 A =         and B =             .
                                                                                     3 4                 3 4
            whether the corresponding MRs are satisfied.
            Violation of an MR indicates that the program                    This source test case is executed on the matrix
            under test is faulty.                                        multiplication function under test first. Next, based on
                                                                         the input relationship specified by the above MR, two
         Thus, MT checks whether relationships between                   follow-up test cases are created, namely, the test case
     inputs and outputs of multiple executions were pre-                 consisting of A and B1 and the test case consisting
     served during the program execution and can be used                 of A and B2. Here, B1 is randomly generated and B2 is
     without knowing the correctness of the output for                   defined as B − B1. Suppose that
     individual executions.
         Consider a program P that multiplies two matrices
                                                                                     1 6
     A and B. Assume that the result of multiplying A with B                 B1 =        .
                                                                                     3 5
     is C. Matrix multiplication has the following property:
     A × B = A × B1 + A × B2 where B = B1 + B2. We can use this          Then,
     property as an MR to conduct MT on P. For example,
                                                                                                  1     5
     Figure 2 shows a test script written using JUnit to con-                B2 = B      B1 =             .
                                                                                                  0     1
     duct automated testing on the matrix multiplication
     function in the JScience Matrix class (http://jscience                  As shown in Figure 3, these two follow-up test
     .org/api/org/jscience/mathematics/vector/Matrix                     cases are also executed on the matrix multiplication
     .html).                                                             function under test. Finally, the outputs of the source
         The source test case (which consists of A and B)                test and the follow-up test cases are validated against
     can be provided by the user or can be generated ran-                the above MR. As shown with this test script, this MR
     domly. For example, assume that the user provided the               based testing approach allows to generate follow-up
     following simple two matrices:                                      test cases automatically and verify relationships

10                 ComputingEdge                                                                                            July 2020

  FIGURE 3. MT of a matrix multiplication program.

  between multiple outputs without any manual inter-            them to test their programs using their domain
  vention. Readers who are interested to know more              knowledge. For example, consider a program
  about MT may consult the article by Chen et al.7              that computes the sine value of a given angle x.
                                                                We can derive the following two MRs for testing
   WHY USE MT FOR TESTING                                       the sine function based on its properties: MR1:
   SCIENTIFIC SOFTWARE?                                         sin(x’) = sin(x) where x’ = x + 360°. MR2: sin(x’)
     ›› Scientific software is often written by scientists      = −sin(x) where x’ = −x. Due to the constraints
        who have the domain knowledge required to               on the testing budget, suppose that we can
        develop them. However, scientists might lack the        conduct testing with only one MR. In such a
        knowledge to apply different forms of conven-           situation, an electrical engineer will most likely
        tional testing methods. But, as evident from the        choose MR1 to test her program due to the
        example in Figure 2, MT is simple in concept,           periodicity of current. But, on the other hand, a
        and hence could be easily learned and applied           land surveyor may choose MR2 since she usually
        without any prior knowledge of software testing         works with positive and negative angles to
        or without any software testing experience.8            represent clockwise and anticlockwise mea-
     ›› As shown in the example given in Figure 2, MT is        surements of angles.
        easy to implement: test scripts could be easily      ›› MT provides an effective way to conduct unit
        prepared by the scientific software developers to       testing for scientific software. One of the
        automate the testing process or to incorporate          reasons for the lack of unit testing in scientific
        MT into existing testing infrastructures such as        software is the difficulty in validating the
        JUnit. Thus, MT does not require the developers         expected output of the unit for a randomly
        to buy or maintain additional expensive testing         generated test input. In such situations, MT can
        tools.                                                  be used to conduct automated unit testing by
     ›› As we discussed in Section “WHAT IS META-               means of MRs.
        MORPHIC TESTING (MT)?”, many scientific              ›› Scientists often conduct testing using a limited
        and engineering applications face the test              number of test cases with known outputs that
        oracle problem. This makes it challenging to            they obtain from experiments or analytical
        conduct automated systematic testing on these           solutions. MT can be used to effectively extend
        programs. MT supports automated systematic              these limited number of test cases by deriving
        testing on such programs.                               MRs and creating follow-up test cases according
     ›› MT uses MRs to determine whether test cases             to the MRs. These follow-up test cases are most
        pass or fail. Often scientific software is devel-       likely to execute parts of the program that might
        oped by domain experts who know the properties          not have been executed with the original set of
        of these algorithms the best. Thus, it would be         test cases. Thus, MT provides a way to extend
        easy for them to derive MRs for testing these           existing test cases.
        programs.                                            ›› Many scientific software involves elements of
     ›› Scientific software developers will be able to          randomness which makes testing difficult. MT
        identify the most effective MRs and prioritize          can still be applicable in such situations.                                                                                      11

      SOME EXAMPLES                                             experts the outputs generated by versions of the pro-
                                                                gram injected with faults, they were unable to identify
     Testing Epidemiological Model                              that the outputs were produced by a faulty version of
     Implementations Using MT                                   the program.
     Pullum et al. used MT to verify and validate an epide-         Here we report the results of conducting auto-
     miological model implementation.9 Such implemen-           mated unit testing on the following functions that per-
     tations are used to model how diseases are spread          form several main calculations in the SAXS program:10
     in populations. Thus, it is important to verify and val-
     idate these models since they will be used to make            ›› calculateDistance (f1): computes the distance
     critical decisions during a disease spread. Epidemio-           between atoms;
     logical model implementations face the oracle prob-           ›› findGyrationRadius (f2): computes the gyration
     lem because these programs are written to find the              radius of groups of atoms;
     answer in the first place. Therefore, developing an ora-      ›› scatterSample (f3): main function responsible
     cle for testing these programs is practically difficult.        for scattering.
     One of the approaches used to test these models is to
     compare the output of the model with data obtained              We used the machine learning based MR predic-
     from real phenomena. Obviously, such data is limited.      tion approach proposed by Kanewala et al.12 to predict
     Other approaches used to test this type of programs        the likely MRs for these functions. The test inputs were
     include comparing the output with results obtained         generated randomly. There were no violations of these
     from mathematical models and comparing the results         predicted MRs when applied to these three functions.
     with other simulation models. These techniques are              To evaluate the effectiveness of MT for conduct-
     not sufficient for conducting systematic and compre-       ing unit testing, we created faulty versions, known as
     hensive testing on these programs.                         mutants, of these functions using the μJava (https:
         The authors tested an ordinary differential equa-      // mutation engine. This
     tion based epidemiological model and an agent-based        mutation engine creates mutants of the program by
     epidemiological model using MT. They used the data         making a syntactic change in the source code. With
     from the 1918 Influenza outbreak to calibrate the          MT, we say that a mutant is killed if an MR is violated
     models. The authors defined 11 MRs based on mak-           when the corresponding source and follow-up test
     ing changes to various model parameters and the            cases are executed on that mutant. Therefore, the
     expected effects that those changes would have on          fault detection effectiveness of MT can be measured
     the model output. These MRs were defined using             by the number of mutants killed during the MT pro-
     the authors' domain knowledge about these models.          cess. Obviously, the higher the percentage of mutants
     Through MT, authors identified an error in the output      killed, the more effective MT is in revealing bugs of a
     method of the agent-based epidemiological model.           program. We use this process to evaluate the fault
                                                                detection effectiveness of MT in the functions men-
     Using MT to Conduct Automated                              tioned above.
     Unit Testing on a Small Angle X-Ray                             Table 1 shows the percentage of mutants killed
     Scattering (SAXS) Program                                  through MT for individual functions. Overall, 90% of
     We used MT to conduct automated unit testing on an         the mutants could be killed using MT. The important
     open source program written to analyze small angle         thing to note here is that the entire unit testing pro-
     x-ray scattering data called SAXS.10, 11 This program      cess was fully automated starting with MR identifica-
     reconstructs macromolecular structures using scat-         tion, source test case generation, test execution and
     tering patterns obtained from experiments. This pro-       further, did not require the domain experts to evalu-
     gram was initially tested by running the program on        ate the correctness of the test outputs. Though no
     a selected set of inputs where the correctness of          violations of MRs were detected for SAXS, MT helps
     the produced outputs was determined by domain              to establish our confidence on the quality of the
     experts. However, when we showed the domain                SAXS program.

12               ComputingEdge                                                                                   July 2020

                                                             TABLE 1. Mutants detected by predicted MRs. f 1:
  Testing a Monte Carlo Simulation                           calculateDistance, f2: findGyrationRadius, and f3:
  Program With MT                                            scatterSample.
  Ding and Hu13 used MT for testing a Monte Carlo mod-
                                                                                           f1       f2        f3      Total
  eling program that simulates photon propagations in
                                                             No. of faulty
  biological tissues for the purpose of accurate genera-                                   19       54       139         212
                                                             versions used
  tion of reflectance images. The biggest challenge for
  testing this program is the lack of test oracles. One      No. of faulty versions
                                                                                           19       45       127         191
                                                             detected by MT
  solution is to compare the results of the Monte Carlo
  simulation program to experimental results. But, as        % of detected
                                                                                          100       83        91         90
                                                             faulty versions
  with many scientific software, building the necessary
  infrastructure to conduct the relevant physical exper-
  iments is time consuming and expensive. For exam-
  ple, in this specific case, conducting a physical exper-   and Monte Carlo simulations. We are strongly con-
  iment would require a laser beam that would produce        fident that MT is one of the most appropriate and
  a specific number of photons, an environment without       cost-effective testing techniques for scientists and
  interruptions from other light sources and good reac-      engineers.
  tive imaging cameras. Thus, the authors used MT to
  conduct testing on this program.                            REFERENCES
      The authors identified five MRs for the program         1. L. Hatton, “The T experiments: Errors in scientific
  based on domain knowledge and experimental results.            software,” IEEE Comput. Sci. Eng., vol. 4, no. 2, pp. 27–38,
  They generated tests that cover all the branches and           Apr.–Jun. 1997.
  functions in the program. Through the violation of one      2. A. J. Abackerli, P. H. Pereira, and N. Calônego, Jr., “A
  of the MRs used for testing, the authors discovered            case study on testing CMM uncertainty simulation
  faults in the program and corrected it.                        software (VCMM),” J. Brazilian Soc. Mech. Sci. Eng., vol. 32,
      They further evaluated the effectiveness of MT             pp. 8–14, Mar. 2010.
  using mutants. The authors created 150 mutants for          3. L. Hatton and A. Roberts, “How accurate is scientific
  the Monte Carlo simulation program and they were               software?,” IEEE Trans. Softw. Eng., vol. 20, no. 10,
  able to detect 90% (135) of these mutants using MT.            pp. 785–797, Oct. 1994.
                                                              4. G. Miller, “A scientist’s nightmare: Software problem
   SUMMARY                                                       leads to five retractions,” Science, vol. 314, no. 5807,
  Some characteristics in scientific software, such as           pp. 1856–1857, 2006. [Online]. Available: http://www
  not knowing the correct answers and inherent uncer-            .
  tainties in calculations, make testing them difficult.      5. U. Kanewala and J. M. Bieman, “Testing scientific
  MT can be an effective testing technique to test               software: A systematic literature review,” Inf. Softw.
  these programs. Instead of checking the correctness            Technol., vol. 56, no. 10, pp. 1219–1232, 2014.
  of individual test outputs, MT checks whether the           6. S. Segura, G. Fraser, A. B. Sánchez, and A. R. Cortés,
  changes in the test outputs are according to what is           “A survey on metamorphic testing,” IEEE Trans. Softw.
  expected by the program with respect to the changes            Eng., vol. 42, no. 9, pp. 805–824, 2016. [Online]. Avail-
  in the inputs. These relationships between inputs and          able:
  the expected changes in the outputs are referred to         7. T. Y. Chen et al., “Metamorphic testing: A review of
  as MRs. Scientists, who typically develop this scien-          challenges and opportunities,” ACM Comput. Surveys,
  tific software, would be in a great position to identify       2017, to be published.
  effective MRs because of their domain knowledge             8. T. Y. Chen, F.-C. Kuo, and Z. Q. Zhou, “An effective
  and, thus would be able to effectively test their soft-        testing method for end-user programmers,” in Proc.
  ware using MT. MT has been successfully applied for            First Workshop End-User Softw. Eng. (WEUSE 2005),
  testing various scientific software including epide-           2005, pp. 21–25. [Online]. Available: http://doi.acm
  miological model implementations, SAXS programs,               .org/10.1145/1082983.1083236                                                                                                  13

      9. L. L. Pullum and O. Ozmen, “ Early results from meta-              U.K., the DIC degree from the Imperial College, London, U.K.,
         morphic testing of epidemiological models,” in Proc.               and the Ph.D. degree from The University of Melbourne, Mel-
         ASE/IEEE Int. Conf. BioMed. Comput., Dec. 2012 ,                   bourne, VIC, Australia. Prior to joining Swinburne, he taught
         pp. 62– 67.                                                        at The University of Hong Kong and The University of Mel-
     10. U. Kanewala , A. Lundgren, and J. M. Bieman, “Auto-                bourne. He is the Inventor of metamorphic testing and adap-
         mated metamorphic testing of scientific software,”                 tive random testing. His contact address is: Department of
         J. C. Carver, N. P. Chue Hong, and G. K. Thiruvathukal,            Computer Science and Software Engineering, Swinburne
         Eds. Soft. Eng. Sci., Taylor & Francis, 2016 , doi:                University of Technology, VIC 3122, Australia. Contact him at                   
     11. [Online]. Available:
         /~mstrout/Projects/SAXS/software.php, 2011.
         Accessed on: Sep. 11, 2017.
     12. U. Kanewala , J. M. Bieman, and A. Ben-Hur, “ Predicting
         metamorphic relations for testing scientific software:
         A machine learning approach using graph kernels,”
         Softw. Testing, Verification Rel., vol. 26, no. 3, pp. 245 –269,
         2016, stvr.1594. [Online]. Available at:
     13. J. Ding and X. Hu, “Application of metamorphic
         testing monitored by test adequacy in a Monte Carlo
         simulation program,” Softw. Qual. J., vol. 25, no. 3,
         pp. 841 – 869, 2017. [Online]. Available at: https://doi

     UPULEE KANEWALA is an Assistant Professor with Mon-

                                                                                                                                                                        Cutting Edge
     tana State University, Bozeman, MT, USA. Her research
     interests include software testing, metamorphic testing,
     and quality assurance of scientific software. She received
     the Ph.D. degree in computer science from Colorado State
                                                                             on the
     University, Fort Collins, CO, USA, in 2015, the Master of Sci-
     ence degree in computer engineering from Purdue Univer-
     sity, West Lafayette, IN, USA, in 2010, and the Bachelor of
     Science degree in computer engineering from University of
     Peradeniya, Peradeniya, Sri Lanka, in 2007. She has authored
                                                                                                                                                                                                           J A N UA RY/ F E B R UA RY 2 016

                                                                                                                                                                                                                                                               IEEE Intelligent Systems provides peer-

                                                                                                                                                                                                  Also in this issue:
                                                                                                                                                                                                               AI’s 10 to Watch         56
                                                                                                                                                                                                Real-Time Taxi Dispatching              68
                                                                                JANUARY/FEBRUARY 2016

                                                                                                                                                                                        From Flu Trends to Cybersecurity 84

     or coauthored multiple peer-reviewed articles and book                                                                                P   U   T   T   I   N    G   A   I   I   N   T   O      P   R   A    C   T   I   C   E

                                                                                                                                                                                                                                                               reviewed, cutting-edge articles on the
     chapters on metamorphic testing including the first paper on
                                                                                                                                                                                                                                                               theory and applications of systems
     automatic detection of Metamorphic Relations. Her address
                                                                                ONLINE BEHAVIORAL ANALYSIS

                                                                                                                                                                                                                                                               that perceive, reason, learn, and
     is Gianforte School of Computing, 357 Barnard Hall, Montana
     State University, Bozeman, MT 59717. Contact her at upulee                                                                                                                                                                                                act intelligently.
                                                                                VOLUME 31
                                                                                NUMBER 1

                                                                                                             IS-31-01-C1   Cover-1                                                                                                  January 11, 2016 6:06 PM

     TSONG YUEH CHEN is a Professor with Swinburne University
     of Technology, Melbourne, VIC, Australia. His main research
     interest focuses on software testing. He received the B.Sc.
                                                                                                                       The #1 AI Magazine

     and M.Phil. degrees from The University of Hong Kong, Hong
     Kong, the M.Sc. degree from University of London, London,

14                 ComputingEdge                                                                                                                                                                                                                                                                    July 2020
PURPOSE: The IEEE Computer Society is the world’s largest            EXECUTIVE COMMITTEE
association of computing professionals and is the leading provider   President: Leila De Floriani
of technical information in the field.
                                                                     President-Elect: Forrest Shull
MEMBERSHIP: Members receive the monthly magazine                     Past President: Cecilia Metra
Computer, discounts, and opportunities to serve (all activities      First VP: Riccardo Mariani; Second VP: Sy‐Yen Kuo
are led by volunteer members). Membership is open to all IEEE        Secretary: Dimitrios Serpanos; Treasurer: David Lomet
members, affiliate society members, and others interested in the     VP, Membership & Geographic Activities: Yervant Zorian
computer field.                                                      VP, Professional & Educational Activities: Sy-Yen Kuo
                                                                     VP, Publications: Fabrizio Lombardi
                                                                     VP, Standards Activities: Riccardo Mariani
OMBUDSMAN: Direct unresolved complaints to                           VP, Technical & Conference Activities: William D. Gropp
                                                                     2019–2020 IEEE Division VIII Director: Elizabeth L. Burd
CHAPTERS: Regular and student chapters worldwide provide the         2020-2021 IEEE Division V Director: Thomas M. Conte
opportunity to interact with colleagues, hear technical experts,     2020 IEEE Division VIII Director-Elect: Christina M. Schober
and serve the local professional community.
AVAILABLE INFORMATION: To check membership status, report            BOARD OF GOVERNORS
an address change, or obtain more information on any of the          Term Expiring 2020: Andy T. Chen, John D. Johnson,
following, email Customer Service at or call
                                                                     Sy-Yen Kuo, David Lomet, Dimitrios Serpanos,
+1 714 821 8380 (international) or our toll-free number,             Hayato Yamana
+1 800 272 6657 (US):                                                Term Expiring 2021: M. Brian Blake, Fred Douglis,
  •   Membership applications                                        Carlos E. Jimenez-Gomez, Ramalatha Marimuthu,
  •   Publications catalog                                           Erik Jan Marinissen, Kunio Uchiyama
  •   Draft standards and order forms                                Term Expiring 2022: Nils Aschenbruck,
  •   Technical committee list                                       Ernesto Cuadros‐Vargas, David S. Ebert, William Gropp,
  •   Technical committee application                                Grace Lewis, Stefano Zanero
  •   Chapter start-up procedures
  •   Student scholarship information
  •   Volunteer leaders/staff directory                              EXECUTIVE STAFF
  •   IEEE senior member grade application (requires 10 years        Executive Director: Melissa A. Russell
      practice and significant performance in five of those 10)      Director, Governance & Associate Executive Director:
                                                                     Anne Marie Kelly
PUBLICATIONS AND ACTIVITIES                                          Director, Finance & Accounting: Sunny Hwang
                                                                     Director, Information Technology & Services: Sumit Kacker
Computer: The flagship publication of the IEEE Computer Society,
                                                                     Director, Marketing & Sales: Michelle Tubb
Computer publishes peer-reviewed technical content that covers
                                                                     Director, Membership Development: Eric Berkowitz
all aspects of computer science, computer engineering,
technology, and applications.
                                                                     COMPUTER SOCIETY OFFICES
Periodicals: The society publishes 12 magazines and 18 journals.     Washington, D.C.: 2001 L St., Ste. 700, Washington, D.C.
Refer to membership application or request information as noted      20036-4928; Phone: +1 202 371 0101; Fax: +1 202 728 9614;
above.                                                               Email:
Conference Proceedings & Books: Conference Publishing                Los Alamitos: 10662 Los Vaqueros Cir., Los Alamitos, CA 90720;
Services publishes more than 275 titles every year.                  Phone: +1 714 821 8380; Email:
Standards Working Groups: More than 150 groups produce IEEE
                                                                     MEMBERSHIP & PUBLICATION ORDERS
standards used throughout the world.
                                                                     Phone: +1 800 678 4333; Fax: +1 714 821 4641;
Technical Committees: TCs provide professional interaction in        Email:
more than 30 technical areas and directly influence computer
engineering conferences and publications.                            IEEE BOARD OF DIRECTORS
Conferences/Education: The society holds about 200 conferences       President: Toshio Fukuda
each year and sponsors many educational activities, including        President-Elect: Susan K. “Kathy” Land
computing science accreditation.                                     Past President: José M.F. Moura
Certifications: The society offers three software developer          Secretary: Kathleen A. Kramer
credentials. For more information, visit                             Treasurer: Joseph V. Lillie                                      Director & President, IEEE-USA: Jim Conrad
                                                                     Director & President, Standards Association: Robert S. Fish
BOARD OF GOVERNORS MEETING                                           Director & VP, Educational Activities: Stephen Phillips
                                                                     Director & VP, Membership & Geographic Activities:
24 – 25 September 2020 in McLean, Virginia, USA                      Kukjin Chun
                                                                     Director & VP, Publication Services & Products: Tapan Sarkar
                                                                     Director & VP, Technical Activities: Kazuhiro Kosuge

                                                revised 1 May 2020
                                                                                                      This article originally
                                                                                                                appeared in

     SciPipe—Turning Scientific                                                                          vol. 21, no. 3, 2019

     Workflows into Computer
     Samuel Lampa, Martin Dahlö, Jonathan Alvarsson, and Ola Spjuth, Uppsala University

      INTRODUCTION                                                  might not be obvious before trying to apply them to
     Scientific Workflows are becoming increasingly popu-           complex tasks.
     lar as a way to automate complex scientific computa-                At the Department of Pharmaceutical Biosciences,
     tions consisting of multiple programs.                         we have spent the last few years using workflow tools
         One of the main motivations behind this develop-           to automate machine learning pipelines for predictive
     ment is increased robustness and reproducibility of            toxicology among other things. We have reviewed the
     computational analyses. Chaining together multiple             top dozen workflow tools popular in our field of bioin-
     programs using plain scripts, as is often the first step       formatics. We even tried out Luigi, created by music
     in automating a pipeline, can easily become fragile            company Spotify, which is popular in industry and far
     and error prone due to the manual management of file           from a bioinformatics-aimed tool.
     paths and program invocations. Also, plain scripts are              A recurring theme has been how often tools
     not optimal if for some reason you have to cancel a run        contain various limits that make them hard to use
     and try to restart it from any partially finished steps. It    for complex use cases. Machine learning workflows
     can be hard to know which output files are properly fin-       in particular often lead to highly complex workflows
     ished and which are truncated from the cancelled run.          because of their common inclusion of cross validation
     Last but not least, plain scripts do not by default save       and parameter sweeps from hyperparameter optimi-
     an execution trace of what was run, such that the full         zation. Not only are these workflows complex, but they
     procedure used to create a specific output file can be         also show some characteristics not always common
     clearly presented. These are all aspects that scientific       in other domains: The need for dynamic scheduling.
     workflows are designed to help with.                           That is, they need to be able to parametrize and start
         Despite many hundreds of scientific workflow               tasks based on information obtained during the work-
     tools published over the years, there can still be sig-        flow run. Somewhat surprisingly, this is a problem in a
     nificant challenges when trying to use many of them.           majority of workflow tools because of how common
         One reason for this is that many workflow tools            it is that they have a strict separation between the
     have been designed with a very narrow use case in              scheduling and execution phases of the workflow run.
     mind, often building in assumptions unique to the              That is, after a workflow has progressed into its execu-
     specific problem domain aimed at, which might make             tion phase, it is commonly not possible to schedule and
     them less applicable for scientific pipeline needs in          start new tasks with parameter values obtained in the
     general.                                                       current run. At least not without initiating completely
         Even among the large numbers of general workflow           new workflow runs.
     tools, surprisingly many contain various constraints                Anyway, after a lot of evaluation, it seemed at the
     and assumptions limiting their generality. Often, this         time that Luigi1 was the most promising way forward
                                                                    for us. Later, we learned that Luigi's functional pro-
                                                                    gramming inspired API design was not quite fit for our
     DOI No. 10.1109/MCSE.2019.2907814                              needs of dynamic workflow rewiring, which is one of
     Date of current version 26 April 2019.                         the reasons why we later chose to develop the SciPipe

16               July 2020                         Published by the IEEE Computer Society              2469-7087/20 © 2020 IEEE

  library (more on that later), but there is an interesting        SciPipe7 is designed from the start as a pro-
  story to tell about our Luigi phase too.                    gramming library embedded in the implementation
                                                              language (Google's Go, or “Golang”), rather than
  Workflows as Computer Programs                              inventing new textual syntax or graphical tools. It
  It turned out that because Luigi was implemented            thus leverages the full power and flexibility of the Go
  as a programming library, it was flexible enough that       programming language for implementing workflow
  we could build an alternative API on top of it, which       logic. So far, we have not encountered a workflow use
  resulted in the SciLuigi helper library. 2 Specifically,    case that we have not been able to model with this
  SciLuigi enabled us to keep the dependency network          approach. Even complex machine learning workflows
  definition separate from task definitions—a core prin-      with nested branching has been solvable, as exem-
  ciple in flow-based programming, as we will explain         plified in a recent paper by the authors. 8 Another
  shortly—which makes it much easier to reconnect             nice side effect of the Go language in particular is
  workflows without changing internals of workflow            that it compiles to self-contained executable files,
  components in complex ways.                                 which makes deployment of most Go programs very
      This positive experience from a programming             straightforward. SciPipe is open source software
  API-based workflow tool helped push a realization that      (MIT licensed). A simple “Hello World” style workflow
  has grown over a number of years of discussions and         example is shown in Figure 2. For more information,
  experimenting: Workflows are in the end just a glori-       source code and documentation, see the work of
  fied version of computer programs.                          Lampa et al. 6 , 9
      It turns out that although many use cases can
  be modeled as simple linear sequences of program            Flow-Based Programming
  invocations depending on each other, not all cases          Focuses on Data Flow
  are that simple. There are many examples where the          Now, there are some differences between most work-
  need for logic to control the workflow structure is         flow programs and most normal programs. The main
  so complex that any attempt at modeling it with a           difference can be seen in the strong focus in work-
  declarative workflow language ends up implement-            flow programs on data flow. When defining how mul-
  ing what is already available in existing programming       tiple programs depend on each other we are in effect
  languages.                                                  defining how the data will flow through these pro-
      This tendency can also be seen in popular work-         grams. There is actually a looming risk for workflow
  flow engines building on domain specific languages          tool developers to miss this detail and model the work-
  (DSLs). Tools that become popular often either have a       flow dependency graph as just dependencies between
  really flexible DSL from the start, e.g., because the DSL   programs and not their inputs and outputs. This can
  is implemented in an existing scripting language (e.g.,     quickly lead to problems because one program typ-
  Nextflow, 3 building on the Groovy language), or their      ically do not depend on just one other program but
  DSLs are step by step becoming increasingly complex,        rather specific outputs of possibly multiple upstream
  and in the end approaching computer programming             programs. That is, data needs to be a first class citizen
  languages in their capability (e.g., Cuneiform,4 imple-     when defining workflows, or we risk missing important
  menting a powerful functional language 5).                  details that will otherwise be buried in less thought-out
                                                              ad hoc code.
  SciPipe                                                         One paradigm that takes note of this fact is
  The above lesson is something we were taking into           flow-based programming (FBP).10 Invented at IBM in
  account when we set out to design the SciPipe work-         the late 1960s and used on large mainframe computers
  flow tool from scratch after finding out about sev-         at banks and other large institutions, the flow-based
  eral limitations with the Luigi/SciLuigi setup we were      programming paradigm has seen some resurge in pop-
  already using (primarily the need for dynamic schedul-      ularity in recent years, possibly driven by the recent
  ing and lack of compile time warnings about errors in       trends toward multicore CPUs, distributed computing,
  workflow connectivity).                                     and message-oriented architectures.                                                                                           17

     FIGURE 1. Flow-based programs can be likened to a factory with processing stations connected with conveyor belts, upon which
     data items “flow” through the network of stations and conveyor belts.

         Flow-based programming ordains a number of                    implementations. It thus allows us to create libraries
     design principles. The most important one in the con-             of reusable components which can be plugged in at
     text of dependency definition though is that it models            any place in the program network, as long as its in- and
     dependencies between processes in an appropriate                  out-ports are compatible with the in- and out-ports
     level of detail; in terms of data inputs and outputs. The         they connect to.
     data itself are modeled through so-called “information                Note that while FBP is often associated with visual
     packets” and inputs and outputs as “ports”—a kind                 programming, that is far from a requirement. In our
     of pluggable component between which yet another                  experience, skipping the visual part and focusing on
     concept can be connected; “channels.” Channels                    a simple programming API has more than fulfilled
     have bounded buffers and act as a kind of conveyor                our needs, while letting us avoid depending on the
     belt between processes, letting processes work on                 complexity of a visual programming framework. The
     information packets from its in-ports asynchronously              fact that the Go language provides the most impor-
     and sending them on their out-ports (mostly) indepen-             tant pieces for this to work (independently running
     dently from the processing rate of other processes.               go-routines and channels with bounded buffers) cer-
     Figure 1 tries to depict this in an artistic way.                 tainly helps.
         The core idea of flow-based programming is how
     it draws all of these things together into a declara-             Reproducibility in Workflow Programs
     tive data flow definition separate from the process               We have presented some rationale for writing work-
     implementations. This allows easy rewiring of the                 flows as computer programs, but what about the
     data flow without changing a single line of process               other aspects important for workflows, such as

18                ComputingEdge                                                                                           July 2020
You can also read