DevOps Continuous Testing Best Practices - Een uitgave in de Continuous Everything reeks - CB Online
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
DevOps
Continuous Testing
Best Practices
Een uitgave in de Continuous Everything reeks
Bart de Best
Onder redactie van
Louis van Hemmen
Colofon Meer informatie over deze en andere uitgaven kunt u verkrijgen bij: Leonon Media (0)572 - 851 104 Algemene vragen : info@leonon.nl Sales vragen : verkoop@leonon.nl Manuscript / auteur : redactie@leonon.nl © 2021 Leonon Media Omslagontwerp : Eric Coenders, IanusWeb, Nijmegen Productie : Printforce B.V., Culemborg Titel : DevOps Continuous Testing Sub titel : Een uitgave in de Continuous Everything reeks Datum : 8 juni 2021 Auteur : Bart de Best Uitgever : Leonon Media ISBN13 : 978 94 92618 450 Druk : Eerste druk, 8 juni 2021 ©2021, Leonon Media Niets uit deze uitgave mag worden verveelvoudigd en/of openbaar gemaakt door middel van druk, fotokopie, microfilm, of welke andere wijze dan ook, zonder voorafgaande schriftelijke toestemming van de uitgever. TRADEMARK NOTICES Scaled Agile Framework and SAFe are registered trademarks of Scaled Agile, Inc.
"We build our computer (systems)
the way we build our cities:
over time, without a plan, on top of ruins."
by Ellen Ullma
X DevOps Continuous Testing Figuren FIGUUR 1-1, DEVOPS LEMNISCAAT. ........................................................................................................ 1 FIGUUR 4-1, VERANDERPARADIGMA. .................................................................................................... 11 FIGUUR 4-2, VERANDERPARADIGMA - BEELDVORMING. ............................................................................ 12 FIGUUR 4-3, VERANDERPARADIGMA- MACHTSVERHOUDING...................................................................... 14 FIGUUR 4-4, VERANDERPARADIGMA - ORGANISATIE................................................................................. 17 FIGUUR 4-5, VERANDERPARADIGMA - RESOURCES. .................................................................................. 19 FIGUUR 5-1, IDEAL TEST PYRAMID. ........................................................................................................ 21 FIGUUR 5-2, NON-IDEAL DESIGN PYRAMID. ............................................................................................ 22 FIGUUR 6-1, VALUE STREAM VOOR CONTINUOUS TESTING. ...................................................................... 27 FIGUUR 6-2, USE CASE DIAGRAM VOOR CONTINUOUS TESTING. ................................................................ 29 FIGUUR 7-1, VOORBEELD UNITTESTEN. .................................................................................................. 39 FIGUUR 8-1, DEVOPS CE-SPIDER MODEL. .............................................................................................. 45 FIGUUR 8-2, DEVOPS CT-SPIDER MODEL. .............................................................................................. 49 Tabellen TABEL 1-1, CONTINUOUS EVERYTHING ASPECTEN. ..................................................................................... 2 TABEL 1-2, BIJLAGEN. ........................................................................................................................... 3 TABEL 3-1, VEEL VOORKOMENDE PROBLEMEN BIJ HET TOEPASSEN VAN TESTEN............................................. 10 TABEL 5-1, TESTSOORT-MATRIX TEMPLATE. ............................................................................................ 23 TABEL 5-2, TESTSOORT-MATRIX VOORBEELD. .......................................................................................... 23 TABEL 5-3, TESTTECHNIEK-MATRIX TEMPLATE. ........................................................................................ 24 TABEL 5-4, TESTTECHNIEK-MATRIX VOORBEELD. ...................................................................................... 24 TABEL 5-5, TESTOBJECT-MATRIX. .......................................................................................................... 25 TABEL 5-6, TESTOBJECT-MATRIX VOORBEELD. ......................................................................................... 25 TABEL 5-7, TESTTOOL PATTERN. ........................................................................................................... 26 TABEL 5-8, TESTTOOL VOORBEELD. ....................................................................................................... 26 TABEL 6-1, USE CASE TEMPLATE. .......................................................................................................... 30 TABEL 6-2, USE CASE DIAGRAM VOOR CONTINUOUS TESTING. .................................................................. 33 TABEL 7-1, GHERKIN KEYWORDS. .......................................................................................................... 36 TABEL 7-2, GHERKIN FEATUREFILE VOORBEELD. ....................................................................................... 36 TABEL 7-3, PYTHON UNITTEST TEMPLATE. .............................................................................................. 37 TABEL 7-4, PYTHON UNITTEST VOORBEELD. ............................................................................................ 38 TABEL 7-5, TESTSTRATEGIE TEMPLATE. .................................................................................................. 40 TABEL 7-6, TESTSTRATEGIE VOORBEELD. ................................................................................................ 40 TABEL 8-1, DEVOPS CE-MODEL............................................................................................................ 43 TABEL 8-2, CONTINUOUS EVERYTHING. ................................................................................................. 44 TABEL 8-3, CMMI LEVELS VOOR CONTINUOUS EVERYTHING. .................................................................... 45 TABEL 8-4, PR-ORG-009. VOLWASSENHEIDSNIVEAUS............................................................................. 46 TABEL 8-5, CT MATURITY CHARACTERISTICS. ........................................................................................... 49 Bijlagen BIJLAGE A, LITERATUURLIJST................................................................................................................. 53 BIJLAGE B, BEGRIPPENLIJST .................................................................................................................. 55 BIJLAGE C, AFKORTINGEN .................................................................................................................... 61 BIJLAGE D, AGILE TOOLS ...................................................................................................................... 63 BIJLAGE E, WEBSITES .......................................................................................................................... 67 BIJLAGE F, INDEX ................................................................................................................................ 69
XII DevOps Continuous Testing
Voorwoord
Dit boek is samengesteld op basis van mijn Development & Operations (DevOps)
praktijkervaring. Voor DevOps is er geen framework of methode dan wel methodiek. Om toch
een structuur aan de content van dit boek te geven is daarom uitgegaan van het boek ‘DevOps
Assessments’, [Best 2019a]. Daarin is het concept van Continuous Everything gedefinieerd
waarvan Continuous Testing een onderdeel is. Tevens zijn de andere DevOps boeken die ik heb
gepubliceerd heb, gebruikt om invulling te geven aan dit boek.
Dit boek is in eerste instantie geschreven voor de training Continuous Testing. De structuur van
dit boek en de gelijknamige masterclass is gelijk gehouden. Tevens zijn de opdrachten
toegevoegd zodat dit boek ook gebruikt kan worden voor zelfstudie.
Hiermee is een boek tot stand gekomen dat niet een A-tot-Z boek is maar veel meer een
naslagwerk om te komen tot het invulling geven aan Continuous Testing best practices. Samen
met het boek DevOps Assessments is het daarmee een praktisch handvat om Continuous Testing
op de landkaart te zetten en verder te ontwikkelen.
Veel van mijn ervaringen heb ik ook al gedeeld in de artikelen op www.ITpedia.nl. Tevens heb
ik de kennis en kunde vertaald naar diverse trainingen die ik verzorg voor IT Management Group.
Deze zijn te vinden op www.dbmetrics.nl.
Hierbij dank ik de volgende personen van harte voor hun inspirerende bijdrage aan dit boek en
de fijne samenwerking!
• D. (Dennis) Boersen Argis IT Consultants
• Dr. L.J.G.T. (Louis) van Hemmen BitAll B.V.
• F. (Freek) de Cloe smartdocs.com
• J.W. (Jan-Willem) Hordijk CTO at Plint AB
• N (Niels) Talens www.nielstalens.nl
• W. (Willem) Kok Argis IT Consultants
Ik wens u veel plezier toe bij het lezen van dit boek en vooral veel succes bij het toepassen van
Continuous Testing binnen uw eigen organisatie.
Mocht u vragen of opmerkingen hebben, aarzel dan vooral niet om met mij contact op te nemen.
Er is veel tijd besteed om dit boek zo compleet en consistent mogelijk te maken. Mocht u toch
tekortkomingen aantreffen, dan zou ik het op prijs stellen als u mij daarvan in kennis stelt, dan
kunnen deze zaken in de volgende editie verwerkt worden.
Bijlagen 51 Bijlagen
Bijlage A, LiteratuurLijst 53
Bijlage A, Literatuurlijst
In Tabel A-1 is een overzicht gegeven van boeken die direct of indirect zijn gerelateerd aan
DevOps.
Referenties Publicaties
Agile Project K. Schwaber, “Agile Project Management with Scrum”, Microsoft
Management 2015 Press, ISBN13: 978 0 7356 1993 7.
Best 2011 B. de Best, “SLA best practices”, Leonon Media 2011, ISBN13: 978
90 71501 456.
Best 2012 B. de Best, “Quality Control & Assurance”, Leonon Media 2012,
ISBN13: 978 90 71501 531.
Best 2014a B. de Best, “Acceptatiecriteria”, Leonon Media, 2014, ISBN 13: 978
90 71501 784.
Best 2014b B. de Best, “Agile Service Management met Scrum”, Leonon Media,
2014, ISBN13: 978 90 71501 807.
Best 2014c B. de Best, “Cloud SLA, Leonon Media, 2014
ISBN13: 978 90 71501 739.
Best 2015a B. de Best, “Agile Service Management met Scrum in de Praktijk”,
Leonon Media, 2015, ISBN13: 978 90 71501 845.
Best 2015b B. de Best, “Ketenbeheer in de Praktijk”, Leonon Media 2015,
ISBN13: 978 90 71501 852.
Best 2017a B. de Best, “Beheren onder Architectuur”, Dutch language, Leonon
Media, 2017, ISBN13: 978 90 71501 913.
Best 2017b B. de Best, “DevOps best practices”, English language, Leonon
Media, 2017, ISBN13: 978 94 92618 078.
Best 2019a B. de Best, “DevOps Assessments”, Dutch language, Leonon Media,
2019, ISBN13: 978 90 71501 81 4.
Best 2019b B. de Best, “DevOps Architectuur”, Dutch language, Leonon Media,
2019, ISBN13: 978 94 92618 06 1.
Best 2020 B. de Best, “Agile Design”, Dutch language, Leonon Media, 2020,
ISBN13: 978 94 92618 375.
Caluwé 2011 L. de Caluwé en H. Vermaak, “Leren Veranderen”, Kluwer, 2011,
tweede druk, ISBN13: 978 90 13016 543.
Davis 2016 Jennifer Davis, Katherine Daniels, “Effective DevOps Building a
Culture of Collaboration, Affinity, and Tooling at Scale”, O'Reilly
Media; 1 edition, ISBN-13: 978 14 91926 307, 2016.
Deming 2000 W. Edwards Deming, “Out of the Crisis. MIT Center for Advanced
Engineering Study”, 2000, ISBN13: 978 02 6254 11 52.
Downey 2015 Allen. B. Downey, “Think Python”, O'Reilly Media, Inc, Usa; Druk 2,
ISBN-13: 978 14 91939 369, 2015
Galbraith 1992 Galbraith, J.R. “Het ontwerpen van complexe organisaties”, 1992,
Alphen aan de Rijn: Samson Bedrijfsinformatie.
Humble 2010 Jez Humble, David Farley “Continuous Delivery Reliable Software
Releases through Build, Test, and Deployment Automation”, Addison-
Wesley Professional; 1 edition, ISBN-13: 978 03 21601 919, 2010.
Kim 2014 Gene Kim, Kevin Behr, George Spafford “The Phoenix Project”, IT
Revolution Press, ISBN-13: 978 09 88262 508, 2014.
Kim 2016 Gene Kim, Jez Humble “The DevOps Handbook: How to Create
World-Class Agility, Reliability, and Security in Technology
Organizations, Patrick Debois, John Willis”, IT Revolution Press,
ISBN-13: 978 19 42788 003, 2016.54 DevOps Continuous Testing
Referenties Publicaties
Kotter 2012 John P. Kotter, “Leading Change”, Engels 1e druk, november 2012,
ISBN13: 978 14 22186 435.
Kaplan 2004 R. S. Kaplan en D. P. Norton, “Op kop met de Balanced Scorecard”,
2004, Harvard Business School Press, ISBN13: 978 90 25423 032.
Layton 2017 Mark C. Layton Rachele Maurer, “Agile Project Management for
Dummies”, tweede druk, John Wiley & Sons Inc, 2017,
ISBN13: 978 11 19405 696.
Looijen 2011 M. Looijen, L. van Hemmen, “Beheer van Informatiesystemen”,
zevende druk, Academic Service, 2011, ISBN13: 978 90 12582 377.
MAES R. Maes, “Visie op informatiemanagement”, www.rikmaes.nl
Scrum Ken Schwaber and Jeff Sutherland, “The Scrum GuideTM”, 2017,
www.scrumguides.org.
Toda 2016 (Luke) Toda, President Strategic Staff Services Corporation and
Director of TPS Certificate Institution Nobuyuki Mitsui, CTO of
Strategic Staff Services Corporation, “Success with Enterprise
DevOps Koichiro” “White Paper”, 2016.
Tabel A-1, Literatuurlijst.Bijlage B, Begrippenlijst 55
Bijlage B, Begrippenlijst
In Tabel B-1 is een begrippenlijst opgenomen. Deze is in het Engels opgesteld omdat zeer veel
termen uit de Engelse taal afkomstig zijn en de uitleg makkelijker leest als de hele uitleg in het
Engels is opgenomen.
Begrip Betekenis
Acceptance test For developers the acceptance testcases gives the answer “How do I
know when I am done?”. For the users the acceptance testcases
gives the answer “Did I get what I wanted?”. Examples of acceptance
testcases are Functional Acceptance Testcases (FAT), User
Acceptance Testcases (UAT) and Production Acceptance Testcases
(PAT). The FAT and UAT should be expressed in the language of the
business.
Agile Infrastructure Within DevOps both Development and Operations work in an Agile
way. This requires an Agile Infrastructure that can be changed with
the same pace as the application is changed through the deployment
pipeline. A good example of an Agile Infrastructure is the use of
Infrastructure as Code.
Alternate path See happy path.
Anti-pattern An anti-pattern is an example of the wrong interpretation of a
pattern.
The anti-pattern is often used to explain the value of the pattern.
Automated tests Testcases should be automated as much as possible to reduce waste
and to increase velocity and quality of the products that are to be
delivered.
Bad paths A ‘bad path’ is a situation where the application does not follow the
‘happy path’ or ‘the alternate’ path. In other words, something goes
wrong. This exception must be handled and should be monitorable.
Behavior driven The development of software requires that the users are asked to
development define the (non) functional requirements. Behavior driven
development is based on this concept. The difference however is that
the acceptance criteria of these requirements should be written in
the customer’s expectation of the behavior of the application. This
can be accomplished by formulating the acceptance criteria in the
Given – When – Then format.
Business value Applying DevOps best practices results in increasing the business
value. Research of Puppet Labs (State Of DevOps Report) proofs that
high-performing organizations using DevOps practices are
outperforming their non–high performing peers in many following
areas [Kim 2016].
Cloud configuration Cloud configuration files are used to initiate a cloud service before
files using it. In this way cloud service providers enable customers to
configure the cloud environment for their needs.
Commit code Committing code is the action in which the developer adds the
changed sourcecode to the repository, making these changes part of
the head revision of the repository [Wiki].
Commit stage This is the phase in the CI/CD secure pipeline where the sourcecode
is compiled to the object code. This includes the performance of the
unittestcases.
Configuration Configuration Management refers to the process by which all
management artefacts, and the relationships between them, are stored, retrieved,
uniquely identified and modified.56 DevOps Continuous Testing
Begrip Betekenis
Containers A container is an isolated structure that is used by developers to build
their application independently from the underlying operating system
or hardware. This is accomplished by interfaces in the container that
are used by developers. Instead of installing the application in an
environment, the complete container is deployed.
This saves a lot of dependencies and prevents configuration errors to
occur.
Cycle time (flow time) Cycle time measures more the completion rate or the work capability
of a system overall, and a shorter cycle time means that less time is
being wasted when a request has been made but no progress or work
is getting done.
Cycle time (lean) The average time between two successive units leaving the work or
manufacturing process.
Development Development is an activity that is performed by the DevOps role
‘developer’. A developer is responsible for the complete lifecycle of a
configuration item. Within DevOps there is no difference anymore
between designer, builder or tester.
Error path See happy path.
Fast feedback Fast feedback refers to the second way of the three ways of Gene
Kim. The second way is about having feedback on the functionality
and quality of the product that is created or modified as soon as
possible in order to maximize the business value.
Feature toggles A feature toggle is a mechanism that makes it possible to enable or
disable a part of the functionality of an application released in
production. Feature toggles enables testing the effect of changes on
users in production. Feature Toggles are also referred to as Feature
Flags, Feature Bits or Feature Flippers.
Feed forward Feed forward within the context of DevOps is the mechanism by
which experiences in the present value stream are used to improve
the future value stream. Feed forward is the opposite of feedback
since feedback is focused on the past and feed forward on the future.
Feedback Feedback within the context of DevOps is the mechanism by which
errors in the value stream are detected as soon as possible and is
used to improve the product and if necessary to improve the value
stream as well.
Given-When-Then The Given-When-Then format is used to define acceptance criteria in
a way that the stakeholders understand how the functionality
actually will work.
GIVEN – the fact that…
WHEN – I do this…
THEN – this happens
Happy path An application supports a business process by receiving, editing,
storing and providing information. The assumed steps in which the
information processing is performed is called the happy path. The
steps in alternate ways are called the alternate path. In that case,
the same result will be achieved via another navigation path. The
crawl of the application that causes an error is called an error path.
Infrastructure as Code Normally infrastructure components have to be configured in order
(IaC) to perform the requested functionality and quality for example a rule
set for a firewall or the allowed IP addresses for a network.Bijlage B, Begrippenlijst 57
Begrip Betekenis
These configurations normally are stored in configuration files which
enable the operators to manage the functionality and the quality of
the infrastructure components.
Infrastructure as code (IaC) makes it possible to programme these
infrastructure component settings and deploy these settings through
the CI/CD secure pipeline by the use of machine-readable definition
files, rather than physical hardware configuration or interactive
configuration tools.
Infrastructure Infrastructure management consists of the lifecycle management of
management all infrastructure products and services in order to support the correct
working of the applications that run on top of the infrastructure.
Latent defects Problems that are not visible yet. Latent defects can be made visible
by injecting faults into the system.
Lead Time (LT) Lead time is the time from when a request is made to when the final
result is delivered, or the customer’s point of view on how long
something takes to complete.
Lean tools • A3 thinking (problem solving)
• Continuous flow (eliminates waste)
• Kaizen
• Kanban
• KPI (Key Performance Indicator)
• Plan Do Check Act (PDCA)
• Root cause analysis
• Specific, Measurable, Accountable, Realistic, Timely (SMART)
• Value stream mapping (depict the flow)
• JKK (No defects are passed to next process)
Micro service Microservices are a variant of the service-oriented architecture (SOA)
architectural style that structures an application as a collection of
loosely coupled services.
In a microservices architecture, services should be fine-grained, and
the protocols should be lightweight [Wiki].
Micro service This architecture consists of a collection of services where each
architecture service provides a small amount of functionality and the total
functionality of the system is derived from composing multiple
versions of a service in production simultaneously and to roll back to
a prior version relatively easily.
Monitoring Framework A framework of components that together form a monitor facility that
is capable to monitor business logic, applications and operating
systems. Events, logs and measures are routed by the event router
to destinations [Kim 2016].
Non-Functional NFR are requirements that define the quality of a product like
Requirement (NFR) maintainability, manageability, scalability, reliability, testability,
deploy ability and security. NFR are also referred to as operational
requirements.
Non-Functional NFR testing is the testing aspect that focusses on the quality of the
Requirement (NFR) product.
testing
Operations Operations is the team often responsible for maintaining the
production environment and helping to ensure that required service
levels are met [Kim 2016].
Product owner The Product Owner is a DevOps role. The Product Owner is the
internal voice of the business.58 DevOps Continuous Testing
Begrip Betekenis
The Product Owner is the owner of the product backlog and
determines the priority of the product backlog items in order to
define the next set of functionalities in the service.
Programming paradigm A style of building the structure and elements of computer programs.
Pull request process This is a form of peer review that span Dev and Ops. It is the
mechanism that lets engineers tell others about changes they have
pushed to a repository.
QA Quality Assurance (QA) is the team responsible for ensuring that
feedback loops exist to ensure the service functions as desired [Kim
2016].
Release patterns There are two patterns of releases to be recognized [Kim 2016]:
• Environment-based release patterns: In this pattern there are two
or more environments that receive deployments, but only one
environment is receiving live customer traffic.
• Application-based release patterns: In this pattern the application
is modified in order to make selectively releases possible and to
expose specific application functionality by small configuration
changes.
Shared version control In order to be able to use trunk-based development developers need
repository to share their source code. The source code must be committed into
a single repository that also supports version control. Such a
repository is called a shared version control repository.
Single repository A single repository is used to facilitate trunk-based development.
System of Engagement SoE’s are decentralized Information Communication Technology
(SoE) (ICT) components that incorporate communication technologies such
as social media to encourage and enable peer interaction [What-is].
System of Information The term SOI includes are all the tools that are used to process and
(SoI) visualize information from SoR systems. Typically, examples are
Business Intelligence (BI) systems.
System of Records A SoR is an ISRS (information storage and retrieval system), that is
(SoR) the authoritative source for a particular data element in a system
containing multiple sources of the same element.
To ensure data integrity, there must be one -- and only one -- system
of record for a given piece of information [What-is].
The ideal testing The ideal testing automation pyramid is a way of testing that can be
automation pyramid characterized as follows:
• Most of the errors are found using unittests as early as possible.
• Run faster-running automated tests (e.g., unittests) before
slower-running automated tests (e.g., acceptance and integration
tests), which are both run before any manual testing.
• Any errors should be found with the fastest possible category of
testing.
The non-ideal testing The non-ideal testing automation pyramid is a way of testing that
automation inverted can be characterized as follows:
pyramid • Most of the investment is in manual and integration testing.
• Errors are found later in the testing.
• Slower running automated tests are performed first.Bijlage B, Begrippenlijst 59
Begrip Betekenis
Test driven Test driven development is the approach in which the source code is
development written after the completion of the testcase definition and execution.
The source code is written and adjusted until the testcase conditions
are met.
Test harness Software constructed to facilitate integration testing. Where test
stubs are typically components of the application under development
and are replaced by working components as the application is
developed (top-down integration testing), test harnesses are
external to the application being tested and simulate services or
functionality not available in a test environment.
The Agile Manifesto The Agile Manifesto (Manifesto for Agile Software Development) was
set up during an informal meeting of seventeen software developers.
This meeting took place from 11 to 13 February 2001 at "The Lodge"
in Snowbird, Utah.
The charter and the principles formed an elaboration of ideas that
had arisen in the mid-nineties, in response to methods traditionally
classed as waterfall development models. Those models were
experienced as bureaucratic, slow and narrow-minded and would
hinder the creativity and effectiveness of developers. The seventeen
people who have drawn up the Agile Manifesto together represented
the various Agile movements. After the publication of the charter,
several signatories set up the “Agile Alliance" to further convert the
principles into methods [Wiki].
The Lean movement An operating philosophy that stresses listening to the customer, tight
collaboration between management and production staff, eliminating
waste and boosting production flow. Lean is often heralded as
manufacturers’ best hope for cutting costs and regaining their
innovative edge.
Tool-assisted code This is a review technique where authors and reviewers use
review specialized tools designed for peer code review or facilities provided
by the source code repositories [Kim 0216].
Value stream The process required to convert a business hypothesis into a
technology-enabled service that delivers value to the customer [Kim
2016].
Value Stream Mapping Value stream mapping is a Lean tool that depicts the flow of
(VSM) information, materials and work across functional silos with an
emphasis on quantifying waste, including time and quality.
Virtualized environment An environment that is based on virtualization of hardware platforms,
storage devices and network resources. In order to create a
virtualized environment usually VMware is used.
Visualization In computing, virtualization refers to the act of creating a virtual
(rather than actual) version of something, including virtual computer
hardware platforms, storage devices, and computer network
resources. Virtualization began in the 1960s, as a method of logically
dividing the system resources provided by mainframe computers
between different applications. Since then, the meaning of the term
has broadened [Wiki].
Waste Waste comprises the activities that are performed in the
manufacturing process that are not adding value to the customer.
Examples in the context of DevOps are:
• Unnecessary software features.
• Communication delays.
• Slow application response times.60 DevOps Continuous Testing
Begrip Betekenis
• Overbearing bureaucratic processes.
Waste reduction Minimization of waste at its source is to minimize the quantity
required to be treated and disposed of, achieved usually through
better product design and/or process management. Also called waste
minimization [Businessdictionary].
Work In Progress (WIP Material that has entered the production process but is not yet a
finished product. Work in progress (WIP) therefore refers to all
materials and partly finished products that are at various stages of
the production process.
WIP limit This is a Key Performance Indicator (KPI) that is used in the Kanban
process to maximize the number of items that has been started but
that is not completed. Limiting the amount of WIP is an excellent way
to increase throughput in your software development pipeline.
Tabel B-1, Begrippenlijst.Bijlage C, Afkortingen 61
Bijlage C, Afkortingen
Afkorting Betekenis
%C/A Percent Complete / Accurate
AWS Amazon Web Services
BDD Behavior Driven Development
BI Business Intelligence
CA Continuous Auditing
CE Continuous Everything
CI Continuous Integration
CIA Confidentiality, Integrity & Accessibility (or Availability)
CD Continuous Delivery
CO Continuous dOcumentation
CIO Chief Information Officer
CL Continuous Learning
CM Continuous Monitoring
CMMI Capability Maturity Model Integration
CoP Communities of Practice (CoP)
CP Continuous Planning
CT Continuous Planning
CTO Chief Technical Officer
DevOps Development & Operations
DML Definitive Media Library
DevOps DevOps
DoD Definition of Done
DoR Definition of Ready
DTAP Development, Test, Acceptance and Production
E2E End-to-End
ESB Enterprise Service Buss
FAT Functionele AcceptatieTest
GAT Gebruiker AcceptatieTest
GDPR General Data Protection Regulation
GIT Global Information Tracker
GSA Generieke & Specifieke Acceptatiecriteria
GUI Graphical User Interface
GWT Given-When-Then
HRR Hand-off Readiness Review
IaC Infrastructure as Code
ICT Information Communication Technology
ID Identifier
INVEST Independent, Negotiable, Valuable, Estimatable, Small and
Testable
ISMS Information Security Management System
ISVS Information Security Value System
IT Information Technology
ITIL Information Technology Infrastructure Library62 DevOps Continuous Testing
Afkorting Betekenis
JVM Java Virtual Machine
KPI Key Performance
LRR Launch Readiness Review
LT Lead Time
MT Module Test
MVP Minimal Viable Product
NFR Non-Functional Requirement
PAT Productie AcceptatieTest
PBI Productie Backlog Item
PDCA Plan Do Check Act
PST Performance StressTest
PT Processing Time
QA Quality Assurance
QC Quality Control
RACI Responsibility, Accountable, Consulted and Informed
RASCI Responsibility, Accountable, Supporting, Consulted and Informed
REST API REpresentational State Transfer Application Programming
Interface
SAFe Scaled Agile Framework
SAT Security AcceptatieTest
SBB System Building Block
SBB-A System Building Block Application
SBB-I System Building Block Information
SBB-T System Building Block Technology
SIT Systeemintegratietest
SLA Service Level Agreement
SMART Specific, Measurable, Accountable, Realistic, Timely
SME Subject Matter Expert
SoE System of Engagement
SoI Systems of Information
SoR System of Records
SoX Sarbanes Oxley
SQL Structured Query Language
SRG Standards Rules & Guidelines
ST Systeemtest
TDD Test Driven Development
TFS Team Foundation Server
UAT User Acceptance Test
UT Unit Testing
VSM Value Stream Mapping
WIP Work In Progress
WoW Way of Working
Tabel C-1, Afkortingen.Bijlage D, Agile tools 63
Bijlage D, Agile tools
Op basis van een onderzoek dat bij tien organisaties is uitgevoerd [Best 2015a] is een lijst
samengesteld van de ingezette tools en de inzet van deze tools. De inzet van tools blijken zeer
veelvuldig te worden aangepast. Figuur D-1 is daarom een momentopname. Het model kan wel
goed gebruikt worden om de eigen DevOps tools op compleetheid te beoordelen. In Tabel D-1
is tevens per tool een URL opgenomen om meer informatie over de tools te geven.
Figuur D-1, Gebruik van tools door tien organisaties.
Short name Long name Website
#define issue-tracker (proprietary)
Bamboo Bamboo www.atlassian.com
CCCQ IBM Rational ClearCase ClearQuest
® ® ®
www.ibm.com
Cherwell Cherwell Software www.cherwell.com64 DevOps Continuous Testing
Short name Long name Website
ClearCase IBM Rational ClearCase
® ®
www.ibm.com
Clientele Mproof Clientele IT Service Management www.mproof.com
Enterprise Architect Sparx Systems Enterprise Architect www.sparxsystems.com
FitNesse FitNesse www.fitness.org
GIT Global Information Tracker www.git-scm.com
HP ALM HP Application Lifecycle Management www8.hp.com
HP QC HP Quality Center www8.hp.com
HP SC HP ServiceCenter Software H10076.www1.hp.com
HP Service Manager HP Service Manager www8.hp.com
IBM Tivoli Service IBM Tivoli Service Management Suite www-01.ibm.com
Management Suite
Ice-Scrum Ice-Scrum www.icescrum.org
Installshield Installshield www.installshield.com
IT Service IT Service Management (Mexon) www.mexontechnology.com
Management
Jenkins Jenkins www.jenkins-ci.org
Jira Jira www.atlassian.com
JMeter Apache JMeter TM
jmeter.apache.org
LeanKit LeanKitTM www.Leankit.com
Maven Apache Maven Project maven.apache.corg
Mavim Mavim www.mavim.com
McAfee McAfee ®
www.mcafee.com
MRM Microsoft Release Manager www.ibm.com/software
MS Excel MicroSoft Excel www.microsoft.com
MSI MicroSoft Installer msdn.microsoft.com
MTM Microsoft Test Manager msdn.microsoft.com
Nexus Nexus www.ichrome.com/solutions
/nexus
NPS Net Promoter Score www.netpromoter.com
Octopus Deploy Octopus Deploy www.octopusdeploy.com
Omnitracker Omnitracker www.omnitracker.com
OTRS Open-source Ticket Request System www.otrs.org
Powershell Automated Powershell script microsoft.com/powershell
ReportServer SQL Server Reporting Services (SSRS) msdn.microsoft.com
Robot Framework Robot Framework www.robotframework.org
ROSS RepliWeb Operations Suite for SharePoint www.repliweb.com
ScrumWise ScrumWise www.scrumwise.com
Selenium Selenium docs.seleniumhq.org
Serena Dimensions Serena Dimensions www.serena.com
SOAtest Parasoft SOAtest www.parasoft.com
SonarQube SonarQube TM
www.sonarSource.org
Subversion Apache SubVersioN (SNV) subversion.apache.org
TestComplete TestComplete SmartBear Software
TFS Team Foundation Server msdn.microsoft.com
TOPdesk TOPdesk www.topdesk.nlBijlage D, DevOps tools 65 Short name Long name Website Twist Twist www.thoughtworks.com WinMerge winMerge www.winmerge.com Tabel D-1, Tools.
Bijlage E, Websites 67
Bijlage E, Websites
Businessdictionary [Businessdictionary] http://www.businessdictionary.com
Collabnet [CollabNet] https://www.collab.net
CleanArchitecture [CleanArchitecture] https://www.freecodecamp.org/news/a-quick-
introduction-to-clean-architecture-
990c014448d2/
CleanCode [CleanCode] https://cvuorinen.net/2014/04/what-is-clean-
code-and-why-should-you-care/
dbmetrics [dbmetrics] http://www.dbmetrics.nl
DevOps [DevOps] • http://DevOps.com
DDD [DDD] https://www.slideshare.net/skillsmatter/ddd-in-
agile
doxygen [doxygen] http://www.doxygen.nl/manual/docblocks.html
doxygen voorbeeld [doxygen voorbeeld] http://www.doxygen.nl/manual/examples/qtsty
le/html/class_q_tstyle___test.html#a0525f798
cda415a94fedeceb806d2c49
EXIN [Exin] http://www.exin.nl
Gladwell [GLADWELL] • http://www.gladwill.nl
IIR [IIR] http://www.IIR.nl
Investopedia [Investopedia] https://www.investopedia.com
ITMG [ITMG] http://www.ITMG.nl
ITPedia [ITPEDIA] http://www.itpedia.nl
UnitTest [UnitTest] • https://docs.python.org/3/library/unittest.html
Wiki [Wiki] http://nl.wikipedia.org / wiki /
Cloud_computing
Wiki docgen [Wiki docgen] https://en.wikipedia.org/wiki/Comparison_of_d
ocumentation_generators
Tabel E-1, Websites.Epiloog 69
Bijlage F, Index
baseline · 43, 48
BDD · 3, 23, 35, 39, 44, 55, 61
% bedrijfsproces · 43
beeldvorming · 2
%C/A · 61 Behavior Driven Development · 47, Zie
BDD
beheerder · 12, 13
A beheervoorziening · 7
best practice · 2, 3, 11, 15, 35, 39, 46, 53,
A/B testing · 44, 47 55, 82, 84
acceptatiecriterium · 5, 9, 35, 55, 56 best-of-breed tool · 20
acceptatieomgeving · 7, 19, 21, 22, 28, 32, beveiliging · 44
41 BI · 61
acceptatietest · 2, 22, 32, 55 black box test · 6, 7, 30
actor · 30, 31 blameless post mortem · 44
ad hoc testing · 46 blue/green environment · 43
Agile · 1, 3, 15, 20, 35, 37, 39, 45, 53, 54, Boehm · 5
55, 59, 81, 82, 83, 84 bottom-up · 11, 17
- coach · 1 bouwen · 1, 20, 31
- design · 1 branch · 21
- infrastructure · 55 branching · 43
- proces · 45 broken build · 43
- werkwijze · 35 build · 43, 47, 48, 56
Agile Release Trains · 15 build-in failure mode · 44
Agile Scrum · 3, 14, 16, 18 business
- aanpak · 10 - case · 14
- framework · 14, 16 - perspectief · 21
- proces · 14, 15 - requirement · 21
- team · 15 - value · 55, 56
alternate path · 55 business DevOps · 44
Amazon Web Services · Zie AWS Business Intelligence · Zie BI
annotatie · 44
anti-pattern · 5, 9, 13, 16, 18, 20, 21, 37,
40, 41, 55 C
applicatie · 1, 6, 7, 12, 13, 14, 15, 24, 25,
26, 27, 30, 31, 32, 35, 37, 43 CA · 2, 61
applicatieobject · 18 canary releasing · 43
Applicatieobject-matrix · 18 capability · 56
applicatieobjecttype · 18 Capability Maturity Model Integration · Zie
architect · 1 CMMI
architectuur · 11, 16, 19, 83 CD · 2, 5, 15, 30, 31, 43, 46, 49, 55, 57,
architectuurmodel · 19 61
architectuurprincipes · 16, 19, 83 CE · 2, 46, 61
artefact · 2, 8, 28, 55 CE-model · 43
auditing · 44 check-in · 43
automated regression testing · 47 Chief Information Officer · Zie CIO
automated sign off · 47 Chief Technology Officer · Zie CTO
automated test · 55 CI · 2, 5, 15, 31, 43, 46, 49, 55, 57,
automatiseringstools · 35 CI/CD secure pipeline · 2, 5, 9, 12, 15, 20,
AWS · 61 21, 22, 31
CIA · 61
CIA-matrix · 7
B CIO · 61
CL · 2, 43, 44, 61
backlog item · 22, 23, 39, 58 cloud · 55
back-up & recovery · 7 cloud configuration file · 55
bad path · 48, 55 cloud service · 55
base use case · 30 CM · 2, 43, 44, 61
CMMI · 43, 45, 46, 6170 DevOps Continuous Testing
CO · 2, 43, 44, 61 Development & Operations · Zie DevOps,
codestandaard · 15 Zie DevOps
commit code · 55 development team · 15, 16, 17
commit stage · 55 Development, Test, Acceptance and
Communities of Practice · Zie CoP Production · Zie DTAP
compleetheid · 63 DevOps · 1, 2, 3, 9, 10, 11, 12, 14, 16, 17,
Completeness / Accurateness · Zie %C/A 18, 20, 24, 29, 31, 43, 45, 46, 47, 48,
compliance · 44 53, 54, 55, 56, 57, 59, 61, 67, 81, 82,
compliancy · 44, 48 83, 84
component · 56, 57, 58, 59 - architect · 16
concept · 1 - cyclus · 1
concurrent gebruikers · 7, 22 - team · 3, 24, 47, 48
Confidentiality, Integrity & Accessibility · - tool · 63
Zie CIA DevOps-organisatie · 20
configuratiebeheer · 43 DML · 61
configuration management · 55 documentatie · 37
container · 56 DoD · 10, 44, 61
Continuous DoR · 44, 61
- Auditing · Zie CA driver · 41
- Delivery · 46, 53, Zie CD DTAP · 48, 61
- Deployment · 2 DTAP environment · 48
- dOcumentation · Zie CO
- Documentation · 2
- Everything · 1, 2, 10, 43, 44, 45, 46, E
82, Zie CE
- Integration · 2, Zie CI E2E · 44, 48, 61
- Learning · 2, Zie CL effectief · 15
- Monitoring · 2, Zie CM efficiënt · 15
- Planning · 2, Zie CP eigenaarschap · 10, 11, 14, 15, 16, 17
- Testing · 1, 2, 3, Zie CT e-mail pass around · 44
- Testing roadmap · 10, 15 emerging design · 31
control · 46, 58 End-to-End · Zie E2E
CoP · 15, 16, 61 Enterprise Service Buss · Zie ESB
CP · 2, 61 epic · 15, 45
criterium · 1 error guessing · 23
CT · 2, 43, 44, 46, 48, 49, 61 error path · 56
CTO · 61 ESB · 61
Cucumber · 35 E-shaped · 43, 44
cycle time · 44, 56 esource · 11
event · 44, 57
D
F
data · 46
data driven testing · 48 fast feedback · 9, 10, 11, 13, 21
data masking · 48 FAT · 5, 7, 19, 22, 23, 24, 25, 26, 32, 39,
database · 6 55, 61
database management system · 47 feature · 36, 45, 47, 56, 59
defect · 9, 10, 12, 13, 18, 19, 21, 22, 28, feature toggle · 43, 56
31, 32, 44, 48, 57 feedback · 22, 44, 46, 47, 48, 56, 58
defect record · 48 feedforward · 56
defect reductie · 22 file system · 6, 38
Definition of Done · Zie DoD firmware · 9
Definition of Ready · Zie DoR five times why methode · 10
Definitive Media Library · Zie DML flow · 44, 45, 57, 59
dekkingsgraad · 13, 25 fout · 19, 21, 40
deployen · 2, 5, 8 framework · 57
deployment · 48, 55 functie · 6, 15, 16, 20, 31, 37, 38
deployment pipeline · 48, 55 functionaliteiteis · 6
Dev engineer · 1 functioneel beheerder · 19
developer · 8, 15, 17, 20, 25, 55, 56, 58, Functionele AcceptatieTest · Zie FAT
59 functionele requirement · 13
development · 55, 56, 59Epiloog 71
Information Security Value System · Zie
G ISVS
Information Technology · Zie IT
GAT · 7, 61 Information Technology Infrastructure
GDPR · 61 Library · Zie ITIL
gebruiker · 41 Infrastructure as Code · Zie IaC
Gebruiker AcceptatieTest · Zie GAT infrastructure component · 57
gedragsspecificatie · 35 infrastructure management · 57
General Data Protection Regulation · Zie integrated test tooling · 47
GDPR integrated VSM · 48
Generieke & Specifieke Acceptatiecriteria · INVEST · 61
Zie GSA IP address · 56
gereedschap · 43 I-shaped · 44
Gherkin · 5, 27, 35, 36 I-shaped people · 17, 20
GIT · 41, 61 ISMS · 61
Given When Then · 56, Zie GWT ISVS · 61
Global Information Tracker · Zie GIT IT · 61
governance · 16, 17 iteratief · 2, 13, 27
Grady Booch · 8 ITIL · 61
Graphical User Interface · Zie GUI
green build · 43
GSA · 61
GUI · 6, 7, 18, 25, 61
J
Guild · 16
GWT · 5, 21, 22, 35, 36, 56, 61 Java Virtual Machine · Zie JVM
JVM · 62
H
K
handmatige testcase · 22
Hand-off Readiness Review · 61 Kaizen · 48, 57
happy path · 23, 48, 55, 56 Kanban · 57, 60
hardware · 5, 9, 18, 56, 57, 59 Ken Schwaber · 14, 54
Hero gedrag · 16 kennis · 6, 9, 12, 16, 17, 18, 20, 41, 82
holistisch · 9, 12 kennisdomein · 20
holistische aanpak · 5, 9 kennisoverdracht · 16, 84
homoniem · 10, 12 keten · 15, 16, 27, 44
HRR · 43, 61 Key Performance Indicator · Zie KPI
hypothesis driven development · 43 kostprijs · 13, 20
KPI · 57, 60, 62
KPI trend measurement · 49
kwalitatieve requirement · 13
I kwaliteit · 1, 2, 5, 16, 19, 40, 41
kwaliteitscriteria · 44
IaC · 55, 56, 57, 61 kwaliteitseis · 6, 7, 18
ICT · 61
ID · 61
ideal test pyramid · 3, 18, 19, 21, 22, 44,
47, 58
L
IDentifier · Zie ID
inchecken · 40 late feedback · 3, 13, 19, 21
incident · 14 latent defect · 57
incrementeel · 2, 13, 27 Launch Readiness Review · Zie LRR
Independent, Negotiable, Valuable, Layton · 54
Estimatable, Small and Testable · Zie Lead Time · Zie LT
INVEST Lean · 9, 57, 59
informatiesysteem · 1, 15, 16, 18, 19, 35, Lean testaanpak · 9
36, 84 Lean tool · 57
informatievoorziening · 1 lemniscaat · 1, 2
Information Communication Technology · lifecycle · 5, 9, 20, 47, 56, 57, 84
Zie ICT lijnmanager · 1
Information Security Management System · log · 44, 57
Zie ISMS loosely coupled · 57
LRR · 43, 6272 DevOps Continuous Testing
LT · 57, 62 PAT · 7, 22, 23, 24, 25, 26, 39, 55, 62
pattern · 23, 37, 40, 41, 55, 58
PBI · 18, 21, 22, 23, 24, 25, 26, 62
M PDCA · 57, 62
peer review · 47
machtsverhouding · 3 PEN-test · 41
management · 46 performance · 22, 48, 55, 57
manual architecture · 44 Performance StressTest · Zie PST
manual testing · 46 performancecriterium · 29
manuele taak · 13 pipeline · 43, 44, 47, 48, 55, 57, 60
manufacturing process · 59 pipeline phase · 47
mastertestplan · 13, 27 Plan Do Check Act · Zie PDCA
MD · 6 planningsobject · 22, 23
merge-hell · 21 policy · 3
merging · 43 PPT · 2, 5, 9, 43
metadata · 14, 19, 48 PPT-aspect · 44
methodiek · 1 pre-condition · 36
methodology · 46 principe · 12
metrics · 44 probleem · 2, 9, 10
microservice · 57 problemen · 10
microservice architecture · 57 proceseigenaar · 1
Minimal Viable Metadata · 48 proceseigenaarschap · 15
Minimal Viable Product · Zie MVP procesmanager · 1
module · 6, 24, 25, 31 Processing Time · Zie PT
Module Test · Zie MT product backlog · 22, 23, 39, 58
monitoring · 43, 57 product owner · 1, 57
monitorvoorziening · 2 Productie AcceptatieTest · Zie PAT
MT · 6, 28, 62 Productie Backlog Item · Zie PBI
multiple expertise · 20 productieomgeving · 5, 13, 22, 28
MVP · 62 production data · 47, 48
programmeerstijl · 40
programmeur · 1, 6, 37, 38, 40, 41
programming paradigm · 58
N promoten · 5, 15
PST · 7, 22, 23, 24, 25, 26, 39, 62
navigator · 41 PT · 62
NFR · 57, 62 pull request process · 44, 58
Non Functional Requirement · Zie NFR pull-request · 41
non-ideal test pyramid · 21, 22 Python · 37
O Q
ontwerper · 31 QA · 44, 58, 62
ontwikkelaar · 12, 13, 16, 38 QC · 62
ontwikkelomgeving · 5, 6, 7, 9, 10, 12, 13, quality · 46
18, 19, 21, 22, 28, 30, 31, 38 Quality Assurance · Zie QA
ontwikkelproces · 37 Quality Control · Zie QC
oorzaak · 2
operations · 55, 57
operator · 31
Ops engineer · 1
R
Organisatievormgeving · 3
OTAP-straat · 8, 18 RACI · 16, 62
outcome · 20, 36 RASCI · 16, 62
over-the-shoulder · 44 redundantie · 13
refactoring · 28, 32, 37
referentie architectuur · 16
regressietest · 13, 21, 30, 32, 38
P regressietestbasis · 10
regression testing · 47
pair programming · 41, 44, 47 release · 2, 58, 64
paperless · 13 release pattern · 58
paperware · 9 repository · 37, 43, 45, 46, 55, 58Epiloog 73
REpresentational State Transfer Application Specific, Measurable, Accountable,
Programming Interface · Zie REST API Realistic, Timely · Zie SMART
requirement · 2, 5, 9, 12, 13, 14, 19, 20, Spotify · 15, 16
22, 27, 30, 31, 35, 43, 55, 57, 62, 84 Spotify organisatie · 15
Resources · 3 sprint backlog · 22, 23, 24, 26
Responsibility, Accountable, Consulted and sprint tijd · 16
Informed · Zie RACI SQL · 62, 64
Responsibility, Accountable, Supporting, squad · 47, 48, 49
Consulted and Informed · Zie RASCI SRG · 43, 44, 48, 62
REST-API · 6, 7, 18, 25, 62 ST · 6, 14, 28, 62
risico · 1, 10, 22, 23, 25, 26, 35, 37, 39, stakeholder · 56
40, 41, 44, 48 Standard Rules & Guidelines · Zie SRG
risicoanalyse · 7, 27 story · 48
roadmap · 15 strategy · 46
roadmap planning · 48 Structured Query Language · Zie SQL
rollback technique · 43 Subject Matter Expert · Zie SME
rootcause · 9 super use case · 30
rootcause analyse · 57 synoniem · 10, 12
systeemtest · 6, 18
System Building Block · Zie SBB
S System Building Block Application · Zie
SBB-A
sad path · 48 System Building Block Infrastructure · Zie
SAFe · 62 SBB-I
SAFe framework · 16 System Building Block Technology · Zie
Sarbanes Oxley · Zie SoX SBB-T
SAT · 7, 22, 23, 24, 25, 26, 39, 62 System Integration Test · Zie SIT
SBB · 30, 62 System of Engagement · Zie SoE
SBB-A · 62 System of Records · Zie SoR
SBB-I · 36, 62 System Test · Zie ST
SBB-T · 30, 62 Systems of Information · Zie SoI
Scaled Agile Framework · Zie SAFe
scheduler · 47
S-CI · 48 T
scrum master · 1, 14, 15, 16
secure code review · 48 TDD · 2, 3, 9, 21, 23, 30, 31, 35, 37, 39,
security · 46, 48, 57 41, 44, 46, 59, 62
Security AcceptatieTest · Zie SAT Team Foundation Server · Zie TFS
Service Level Agreement · Zie SLA technical debt · 46
shift left organisatie · 21 technical debt backlog · 11, 15, 17, 46
silo · 59 telemetry · 44
Simian army · 44 template · 23, 24, 25, 26, 29, 30, 37, 40,
SIT · 6, 22, 23, 24, 25, 26, 28, 39, 62 43
SLA · 44, 62 test · 7, 21
SMART · 57, 62 - aanpak · 5, 9
SME · 16, 62 - architectuur · 10
SoE · 15, 16, 58, 62 - automation · 19, 41, 47
software · 5, 8, 9, 13, 16, 24, 30, 31, 35, - basis · 5, 6, 7, 27, 30, 31
46, 47, 48, 55, 59, 60, 64, 81, 82 - case · 2, 5, 9, 10, 12, 14, 18, 19, 20,
Software · 53, 59, 63, 64 21, 22, 24, 28, 31, 32, 35, 37, 41, 46,
software lifecycle · 9 47, 48, 55
softwareontwikkelaar · 13 - cyclus · 31
softwareontwikkeling · 3, 37 - data · 47, 48
softwareontwikkelproces · 5, 9, 12, 46 - data generating tool · 47
SoI · 62 - Driven Development · Zie TDD
SoR · 15, 16, 27, 58, 62, Zie - generation · 47
source code · 43, 44, 46 - level · 47
sourcecode · 9, 12, 13, 24, 32, 35, 37, 40, - lifecycle · 47
41, 55, 58, 59 - management · 2, 5, 6, 9, 10, 27, 44,
sourcecode header · 13 46, 47, 48
sourcecodestandaard · 13 - object · 24, 25, 26, 31, 41, 47, 48
SoX · 62 - object-matrix · 21, 25, 26
- omgeving · 6, 774 DevOps Continuous Testing
- pattern · 18, 41, 46, 47
- script · 47
U
- situatie · 41
- skills · 10 UAT · 62
- soort · 3, 5, 6, 7, 9, 10, 14, 18, 21, 22, uitwijkvoorziening · 7
23, 24, 25, 26, 27, 28, 29, 31, 39, 41 uniforme
- soorten · 2, 3 - meta data · 48
- soort-matrix · 18, 22, 23, 39 - test terminology · 47
- strategie · 2, 3, 5, 18, 23, 25, 27, 28, - test tooling · 47
30, 35, 39, 40 - testproces · 47
- strategie pattern · 18 - WoW · 18
- strategy · 46, 47, 48 unit test case · 47
- taal · 10 Unit Testing · Zie UT
- techniek · 23, 24, 25, 26, 39 unittest · 3, 5, 6, 18, 21, 37, 38, 39, 67
- techniek-matrix · 23, 24, 25, 26, 39 unittestcase · 37, 41
- tool · 3, 11, 19, 25, 26 unittestcases · 12, 13, 18, 35
- type · 47 use case · 3, 5, 27, 28, 29, 30, 31
testen · 1, 2, 5, 6, 7, 9, 12, 13, 14, 15, 16, use case diagram · 3, 5, 18, 27, 28, 29,
18, 20, 21, 22, 23, 24, 25, 26, 27, 28, 30, 31
30, 31, 32, 35, 37, 38, 39, 41 User Acceptance Test · Zie UAT
tester · 20, 31, 56 User eXperience design · 43
testtechniek · 3, 5, 7, 10, 18, 21 user interface test · 41
- API-testing · 40 UT · 6, 28, 38, 48, 62
- branch testing · 40
- code-driven testing · 40
- error-handling testing · 40 V
- interface testing · 40
- loop testing · 40 value stream · 3, 5, 12, 19, 20, 27, 28, 29,
- negative testing · 40 44, 48, 56, 57, 59, 62, 84
- static testing · 40 Value Stream Mapping · Zie VSM
testtechniek-matrix · 21, 39 velocity · 16, 55
TFS · 62 veranderparadigma · 2, 11, 12, 14, 17, 19
The Agile Manifesto · 59 versiebeheer · 43
the ideal testing automation pyramid · 58 versioning · 46
The Lean movement · 59 virtualized environment · 59
the non-ideal testing automation inverted visibility · 44
pyramid · 58 visie · 12
theme · 45 visualization · 59
time-to-market · 13 volwassenheidsniveau · 45, 46
toetsen · 2, 5, 16, 18, 30 volwassenwording · 15
tool-assisted code review · 44, 59 VSM · 44, 62
tooling portfolio · 16
toolintegratie · 20
toolleverancier · 20 W
top-down · 59
traceability · 43 waste · 5, 9, 12, 13, 14, 16, 18, 22, 39,
traceerbaarheid · 43, 44 41, 55, 57, 59, 60
transactieverwerking · 15 waste reductie · 5, 9, 12, 19, 60
tribe · 47, 48 Way of Working · Zie WoW
trunk · 58 white box test · 6, 30, 38
T-shaped · 44 WIP · 62
T-shaped people · 20 Work In Progress · Zie WIP
WoW · 9, 11, 13, 14, 15, 17, 18, 19, 62Epiloog 75 Nawoord Mijn ervaring is dat de denkbeelden die ik vastleg in een artikel of een boek zich blijven evolueren. In geval u met een bepaald onderwerp uit dit boek aan de slag gaat in uw eigen DevOps organisatie, dan raad ik u aan om even met mij contact op te nemen. Wellicht zijn er aanvullende artikelen of ervaringen op dit gebied die ik met u kan delen. Dit geldt ook omgekeerd evenredig. Als u bepaalde ervaringen hebt die een aanvulling zijn op hetgeen in dit boek is beschreven, dan nodig ik u uit om dit met mij te delen. U kunt mij bereiken via mijn e-mail adres bartb@dbmetrics.nl. Over de auteur Drs. Ing. B. de Best RI is vanaf 1985 werkzaam in de ICT. Hij heeft voornamelijk bij de top 100 van het Nederlandse bedrijfsleven en de overheid gewerkt. Hierbij heeft hij gedurende 12 jaar functies vervuld in alle fasen van de systeemontwikkeling, inclusief exploitatie en beheer. Daarna heeft hij zich toegelegd op het service management vakgebied. Momenteel vervult hij als consultant alle aspecten van de kennislevenscyclus van service management, zoals het schrijven en geven van trainingen aan ICT-managers en service managers, het adviseren van beheerorganisaties bij het richting geven aan de beheerorganisatie, de beheerinrichting, het verbeteren van beheerprocessen, het uitbesteden van (delen van) de beheerorganisatie en het reviewen en auditen van beheerorganisaties. Hij is op zowel HTS-niveau als Universitair niveau afgestudeerd op het beheervakgebied.
You can also read
