Top 10 BI & Data Trends 2023 - Calibrate for Crisis - IBT
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Calibrate for Crisis
Top 10
BI & Data
Trends
2023
1 — TOP 10 BI & DATA TRENDS 2023
Power — As it plays out, weʼll see new
fractures in old structures and the
technology. (See sidebar.)
As data and analytics
and data — emergence of a state of multipolarity,
or the distribution of power among
professionals, we need to
adjust to more fragmentation,
with its disparate data centers,
Multipolarity
are shifting.
multiple entities. And while the
jostling will occur at the international disrupted supply chains,
level, weʼll feel the impacts locally, nonstop innovation, and
in both our businesses and personal hampered access to skilled
Both are becoming lives. Among other repercussions, labor. And in a world where Skills shortages exacerbated
weʼll be challenged with energy crisis has become a constant,
fragmented. shortages, currency fluctuations,
broken supply chains, and struggling
calibrating for it becomes a core
competency – so we can react in
VC funding dries out
markets. Multipolarity will also have the moment and anticipate Regulations get more complicated
At this moment in history, weʼre in a perfect a significant impact on information whatʼs coming next.
storm. Geopolitical, social, and economic Multi-cloud impacts architecture
concerns are churning. Weʼre seeing a rise
in conflict and isolationist tendencies; Possible splinternet
instead of a move toward cooperation, local
Data has left the building
regulations are amping up. In the economy,
confidence is low, recession fears are high, According to Gartner, by 2025, more
and rising interest rates – plus inflation – than 50% of enterprise-critical data
are impacting borrowing. will be created and processed outside
Distributed Data
the data center or cloud.
And hyperconnectivity, distributed
What do these factors have in common? More — Gartner 1
ledgers, and Web3 may push
than a few scholars are claiming that weʼre in fragmentation further.
the midst of a shift toward de-globalization.
2 — TOP 10 BI & DATA TRENDS 2023
It’s time to During the pandemic, organizations acquired new technology simply to keep
the lights on. In that sudden modernization, systems and processes became
calibrate
a chaotic tangle. Now itʼs time to play catch-up in areas like governance,
responsiveness, and cloud costs.
for crisis. In these challenging times, nearly 7 out of 10 global tech leaders are concerned
about the growing technology investment required to remain competitive.²
But few, if any, are looking to reduce their data efforts. Instead, surveys
And that requires indicate³ that data integration, analytics, automation, API management, and
two key AI are all top technologies CXOs rely on for crisis management. And nowʼs
the time to use them. In the coming year and beyond, we believe it will be
competencies. important to focus on two areas in particular:
Calibrate the decision Calibrate the integration
Hone your decision accuracy – at speed and scale – to Work to achieve connected governance – the ability to
better react to, adapt to, and even anticipate access, combine, and oversee distributed data sets – to
unexpected events. handle a fragmented world.
* What are the top 10 BI and data trends that will help you lead in an uncertain world? Find them in the pages ahead.
3 — TOP 10 BI & DATA TRENDS 2023
Top 10 BI & Data Trends 2023
Calibrate the decision Calibrate the integration
1 Supply chain disruption meets real-time data 6 Market consolidation opens new opportunities
2 Decision velocity – at scale 7 What’s old is new again – in the cloud
3 Optimizing across low-code and high-code 8 “X-fabric” holds connected governance together
4 The human/machine arms race 9 AI moves deeper into the pipeline
5 Data stories that compel action 10 The rise of derivative and synthetic data
4 — TOP 10 BI & DATA TRENDS 2023 QLIK.COM
Calibrate the decision
1 Supply chain disruption meets real-time data
Anyone who has attempted That means acting on contingency
to buy a new car (or plans and even, if possible, “pre-acting”
The Impact Analyst Prediction
– in other words, using forecasts and
computer, or construction scenarios to pivot before things begin
materials) in the last few to break down. The infrastructure to
The pandemic and conflict in Ukraine By 2027, 60% of spending
years knows how seriously handle real-time data has been in place
have created significant components on data capture and
shortages. This backdrop has become movement technology
supply chains have been for some time, but the critical use cases
the trigger for organizations to update
compromised. and ultimate potential havenʼt been fully will be on streaming data
their data-delivery pipelines, from batch-
explored. Now they should be. Weʼre pipelines, enabling a new
oriented to near real-time data. And as
Disruptions can happen faced with managing inventory when
more edge devices appear on the grid generation of real-time
raw materials are scarce and shipping is
anywhere in the world, – producing continuous, high-volume simulation, optimization,
disrupted; needing to pinpoint supply
and they require an streams of data – more opportunities and recommendation
chain bottlenecks to backfill and work
immediate response. to leverage real-time data will arise. capabilities.”
more effectively with partners; and
having to shift resources to tackle new
opportunities or address humanitarian — IDC⁴
needs when conflicts arise. And the pace
of these issues is only going to accelerate.
5 — TOP 10 BI & DATA TRENDS 2023
Calibrate the decision
2 Decision velocity — at scale
Once you have real-time Automation will help. According to Decision velocity at scale is also about
Gartner, 95% of decisions based shortening the data-to-action pipeline
data in place, the next step The Impact
on data can be at least partially for humans – decreasing the time
is to tune your operational automated,⁵ and in a more challenging it takes for people to find data and New roles will emerge with a focus on
decisions to the same pace. environment, automation will increasing the frequency of acting on it. decision innovation – such as Chief
accelerate. But even though analytics, In addition to technology, data literacy Decision Officer, Decision Designer, and
For example, during times of inflation, itʼs AI, and automation can make more and is a key enabler for that. And finally, Decision Engineer. These roles should
unsustainable for a retailer to push all their faster decisions than humans, make decision velocity leaves a big data trail, be tasked not only with automating
cost increases to customers. Instead, they sure to place humans at the beginning with patterns that can be analyzed. routine decisions but also with
should improve efficiencies – thousands of and the end of decision-automation That will create an opening for addressing the biggest,
them, occurring thousands of times a day. cycles for design and review. decision-mining. thorniest problems you face.
By 2026, 85% of enterprises will combine human expertise with AI, ML, NLP, and pattern recognition to
Analyst Prediction augment foresight across the organization, making workers 25% more productive and effective.” — IDC⁶
6 — TOP 10 BI & DATA TRENDS 2023
Calibrate the decision
3 Optimizing across low-code and high-code
In recent years, weʼve seen One prominent tool is GitHub Copilot (based on
GPT-3), which translates plain English into code.
the emergence of low- GitHub estimates that Copilot generates
The Impact Analyst
AnalystPrediction
Prediction
code tools for building roughly 30% of the application code created
applications, enabling non- on the site.⁷
These two camps will always By 2023, 60% of net-
exist, though many use cases new applications will be
technical workers to will gradually evolve from
compose their own apps. On the other hand, some organizations have developed with no-code/
high-code to low as repeatable
progr ammers and app developers who simply low-code platforms, up
These tools not only drive the creation of want prompts they can code in. This is workflows are identified and
markets mature. Still, the choice
from 30% today.”
apps, they also increase the consumption particularly the case in data engineering and
of data and insights. For example, data science, as those fields get reinvented for shouldnʼt be between low-code
and high-code. Instead, it should — IDC⁸
application automation enables workers to cloud. To cater to these needs, weʼve seen
create chains of events triggered by data. the emergence of high-code tools, which be code optimization, focusing
AutoML gives business analysts access provide templates for coders who want on the highest productivity and
to the most advanced algorithms. And maximum flexibility. best business outcomes given
data transformations within data-delivery the available skill sets.
pipelines can be largely automated, too.
7 — TOP 10 BI & DATA TRENDS 2023Calibrate the decision
4 The human/machine arms race
In the summer of 2022, a Google stated that his claims were Itʼs so capable that itʼs spawned a number of
unfounded ⁹– and the engineer was fired
Google engineer claimed services, from code optimization, to writing
The Impact
for violating company security policies – but marketing copy, to mimicking the voices of
that one of the companyʼs this incident shows how far machines have authors like Kafka and Hemingway.¹⁰ In the space of data and analytics,
chatbots (named LaMBDA) come in a short time.
natural language capabilities will have
had achieved consciousness, There are now 5 - 6 global developments huge implications for how we query
or a human level of Because natural language models have been even bigger than GPT-3,¹¹ models trained information and how itʼs interpreted and
self-awareness. trained on massive troves of data using deep- on even larger data sets. Where those will reported. Weʼll find not only the data
neural-network machine learning, theyʼve take us, we can only imagine. We may be weʼre looking for but also the data we
reached a paradigm shift. Perhaps the most about to cross the Rubicon where machines hadnʼt thought to ask about.
widely publicized is GPT-3. can finally pass the Turing test.
In the next five to 10 years or sooner, based on the groundbreaking innovation in AI,
Analyst Prediction TuringBots will be created by several tech vendors.” — Forrester Research¹²
8 — TOP 10 BI & DATA TRENDS 2023Calibrate the decision
5 Data stories that compel action
For decades, we in the data Fortunately, you donʼt have to get all the data to all the
people all the time. Having the right slices of small
industry have shared a mantra: The Impact
data at the right time is more useful. And not every
Provide the right information to the insight has to be arrived at through user exploration. To connect storytelling to action, you need to add three steps:
right user at the right time. Many can be more prescriptive and recommendation-
oriented, delivered straight from the data. 1. Predicting what will happen next and suggesting best
actions with AutoML
Thatʼs more important now than
Data storytelling has been touted as the way to get 2. Using alerting, reporting, and automation to bring stories
ever. But in a fragmented world, data to make sense to users; stories can reach people into workflows at the right time
where data is distributed and time emotionally – and compel them to act – when data 3. Embedding not just dashboards but micro-stories into
is scarce, itʼs tougher to do. alone does not. But data storytelling needs to be the systems where people work. That will move
much more than adding charts to infographics or data storytelling from insights you could act on to
PowerPoints. It needs to be connected with action. insights you do act on.
By 2025, data stories will be the most widespread way of consuming analytics, and 75% of
Analyst Prediction stories will be automatically generated using augmented analytics techniques.” — Gartner¹³
9 — TOP 10 BI & DATA TRENDS 2023Calibrate the integration
6 Market consolidation opens new opportunities.
In an increasingly Combining these functions opens
opportunities that werenʼt possible
fragmented world, before. It makes it easier for
The Impact Analyst Prediction
thereʼs also a market trend data producers and consumers
in the opposite direction: to collaborate, starting with the The move toward consolidation on the supply By 2023, the stand-alone
side is met by the demand side. In challenging data preparation market
convergence. product, outcomes, or decisions they
times, CFOs and CEOs get more involved
have in mind and working backward will disappear, and data
in the business, and they want to see ROI
to build agile data pipelines around
Weʼre seeing the articulated clearly. This will help drive pricing preparation capabilities
their business goals.
consolidation of previously models away from per-user toward the value will be embedded
generated. After all, you canʼt predetermine
siloed systems, including Common standards and APIs enable
who in your organization should use what
within modern data
data integration, interoperability. And when a vendor
tool when you donʼt know where the next
management, analytics,
management, analytics/AI, operates across more segments, challenge will come from. Instead, facilitate and data science tools.”
convergence is even easier. This
visualization, data science, isnʼt about going “all-in” on one
general access to tools and platforms, in a
governed way, and build from there. — Gartner¹⁴
and automation. data stack, which can lead to vendor
lock-in or compromise compliance.
Instead, choose platforms that can
work with multiple stacks, and
consolidate the data across them.
10 — TOP 10 BI & DATA TRENDS 2023Calibrate the integration
7 Whatʼs old is new again — in the cloud
During the pandemic, This has created a Wild West of
startups (often dubbing themselves
organizations quickly part of “the modern data stack”)
The Impact Analyst Prediction
modernized applications and fueled by venture capital, each going
From a cost perspective, itʼs not
moved data to the cloud. after one specialization. And while
sustainable for organizations to
To help alleviate the
winners will certainly emerge, the vast developer skills shortage,
As these changes mature, many of the work with a wide array of niche
majority will disappear as industries 55% of organizations will
same issues from the on-prem world are vendors. Fortunately, many of
mature and consolidate. And this
rearing their heads. For example, after you
the features will be recreated in use cloud marketplaces
trend will accelerate as VC funding
the larger integrated data and and tech startup
adopt a cloud warehouse or lake, you need goes from boom to bust. (In Q3 2022, analytics platforms. As cloud
to tackle data movement, transformation, VC funding declined 53%, an early acquisitions as their
markets mature, managers may
metadata catalogs, and so on. signal of what may come.¹⁵) In other most important
abandon architectures reliant on
words, expect a big wave of M&A as too many startups that struggle. approaches to software
These needs are driving investment in a small vendors look for the exit. It Instead, these startups may be sourcing by 2024.”
multitude of software segments around happened in the on-prem world, and used as a source for “acqui-hires.”
warehouses and lakes – including semantic itʼll happen again in the cloud.
layers and data integration, movement, — IDC¹⁶
sources, and observability.
11 — TOP 10 BI & DATA TRENDS 2023Calibrate the integration
8 “X fabric” holds connected governance together
The discussion in recent years In a world with millions of builders, we
need other fabrics, or “X fabrics.” These
has been about data fabric (as include application fabric, BI fabric, and
The Impact Analyst Prediction
well as hubs and mesh), an algorithm fabric – and right now, these
important methodology that methodologies are even less mature
For connected governance, you By 2023, 60% of G2000
need X fabrics. You also need enterprises will have
connects distributed data sets than data fabric.
to certify artifacts based on
through semantic models. But how trustworthy they are – for
a data control plane
Being able to reuse data and analytic architecture to enable
for connected governance, we assets is critical, spanning models,
example through watermarking
based on thresholds. Every DataOps, propel ML-
need more than that. scripts, and analytics content. And the
organization today is looking for
need for reuse also underscores the based data engineering,
better ways to access their data
importance of the catalog, as well as its and analytic artifacts. And in a
reduce data risks, and
evolving role. Common APIs will make distributed world, orchestration propel innovation among
it possible to have modularity and becomes even more important. Gen D workers.”
composability, and catalogs can provide
the oversight that spans artifacts.
— IDC¹⁷
12 — TOP 10 BI & DATA TRENDS 2023Calibrate the integration
9 AI moves deeper into the pipeline
As we mentioned in Trend 6, Using AI in data management would
shift the perennial 80/20 distribution
analytics, automation, and AI (between preparing the data and
The Impact Analyst Prediction
are converging, increasingly analyzing it) by automating more of
overlapping with each other. the rote tasks in data engineering. More AI in the data pipeline Through 2024, manual
doesnʼt mean that humans wonʼt data integration tasks will
In the process, theyʼre It could, for example, automate
be involved. After all, humans are
anomaly detection and reporting, be reduced by up to 50%
cross-pollinating, generating take advantage of self-healing,
exceptionally good at synthesizing
new insights that werenʼt complex problems with multiple through the adoption of
use just-in-time deployment, and
possible before. find risky attributes such as PII
component parts. But AI will data fabric design patterns
automate some of the more that support augmented
data. Algorithms would be able to
manual data preparation tasks, so
But what about moving those components “crawl” the data and surface insights
data engineers and scientists can
data integration.”
deeper into the data pipeline, before an outside your hypothesis. And finally,
focus on more impactful work.
application or dashboard has even been automated annotations and tagging
— Gartner¹⁸
built? There are several ways this could would drive better engagement with
benefit organizations. less skilled integrators.
13 — TOP 10 BI & DATA TRENDS 2023Calibrate the integration
10 The rise of derivative and synthetic data
Data is a liquid asset; it can look In other situations, useful data simply
doesnʼt exist. The lack of available user
different for different purposes. data, for example, can be problematic
The Impact Analyst Prediction
And today, itʼs easier than ever to alter data for small businesses, who wonʼt be able
Thanks to a number of factors By 2030, synthetic
for different use cases or transform it into to train their AI models with vast data
– including data re-use, testing,
formats for specific targets. Data that has sets. Or an enterprise may want to run
privacy laws, missing data, and
data will completely
been transformed, processed, aggregated, experiments and what-if analyses for overshadow real data in
the need for data to train AI
cases – simulations of financial crime
correlated, or operated on is called models – weʼll see more derivative AI models.”
“derivative” data. Derivative data has been and fraud, for example. and synthetic data.
especially useful for test data management –
In both of the scenarios above, — Gartner¹⁹
creating, managing, and delivering test data to
application teams. synthetic data can be an option.
Synthetic data is data that has not been
But now, with new privacy laws and integrity generated from real operations.
issues, itʼs becoming essential to obfuscate
data even further.
14 — TOP 10 BI & DATA TRENDS 2023The way Identify use cases where real-time
data and decision velocity can
Look for ways to converge
siloed technologies
forward.
address challenges
Use a fabric not just for your data
Leverage the right mix of but for other artifacts as well
code optimization for your
What do these trends business users and engineers Apply AI earlier in the
mean for you? data pipeline
In a fragmented world where See how data storytelling can be
more closely linked to action Leverage the VC crunch to Itʼs about more than
crisis has become a constant, itʼs
remediate urgent skills shortages
important to innovate and be just the technology.
prepared. Start by thinking Use innovations in natural language
through how these trends apply Data professionals of all kinds will
to bring data querying, insights, and Look at derivative and synthetic
play a key role in calibrating
to your organization. actions to more people approaches as ways to maximize
through crisis. In a deglobalizing
value in a distributed world world, localized sourcing of
those professionals will become
increasingly important. Key to this
is increasing the data literacy of
your existing workforce, using both
education and technology.
15 — TOP 10 BI & DATA TRENDS 2023Our goal
To give you the power to While multipolarity is an unpredictable state, data and
analytics can help reduce uncertainty. And fragmentation
anticipate, pivot, and does hold promise; it could move the world to a longer-term
vision of data democracy. In the meantime, addressing these
trends will drive critical efficiencies in the here and now. And
navigate through crisis. it could lay a foundation for a massive cycle of innovation and
prosperity, accelerating growth as we turn the corner.
We’re here to help
Get ready for
what’s coming.
Qlik® is designed to empower everyone in With end-to-end data integration and analytics
your organization, no matter their skill level, solutions, powerful boosts to data literacy from AI,
See How We’re Different
to combine data from a multitude of sources, and an independent open platform that enables
explore it freely in an intuitive way, and make you to embed analytics anywhere, Qlik helps you
associative discoveries that other solutions achieve Active Intelligence in your organization
wonʼt uncover. – continuous intelligence where technology and Prefer a
processes support the triggering of actions from conversation?
accurate, up-to-date data.
Get in Touch
16 — TOP 10 BI & DATA TRENDS 2023About Qlik Qlikʼs vision is a data-literate world, where everyone can use data and analytics to improve decision-making and solve their most challenging problems. Qlik offers real-time data integration and analytics solutions, powered by Qlik Cloud, to close the gaps between data, insights and action. By transforming data into Active Intelligence, businesses can drive better decisions, improve revenue and profitability, and optimize customer relationships. Qlik serves more than 38,000 active customers in over 100 countries. qlik.com © 2023 QlikTech International AB. All rights reserved. All company and/or product names may be trade names, trademarks and/or registered trademarks of the respective owners with which they are associated. 1 Goasduff, Laurence, “12 Data and Analytics Trends to Keep on Your Radar,” Gartner, April 5, 2022, https://www.gartner.com/en/articles/12-data-and-analytics-trends-to-keep-on-your-radar. 2 Future Enterprise Resiliency and Spending Survey, IDC, April 2022. 3 “The Foundation of Data and Analytics is Cloud!,” Gartner BI Summit, 2021 Gartner CIO Survey; and Qlik, QlikWorld, customer and former competitor interviews, BCG analysis. 4 IDC FutureScape: Worldwide Data and Content Technologies 2022 Predictions, https://www.idc.com/getdoc.jsp?containerId=US48082521. 5 “Striving to Become a Data-Driven Organization? Start with 5 Key D&A Initiatives,” Gartner, https://www.gartner.com/en/information-technology/insights/data-and-analytics-essential-guides. 6 IDC FutureScape: Worldwide Artificial Intelligence and Automation 2022 Predictions, https://www.idc.com/getdoc.jsp?containerId=US48298421. 7 Coberly, Cohen, “Almost 30 percent of new GitHub code is written with AI assistance,” TechSpot, October 28,2021, https://www.techspot.com/news/91984-almost-30-percent-new-github-code-written-ai.html. 8 IDC FutureScape Worldwide Cloud 2022 Predictions, https://www.idc.com/getdoc.jsp?containerId=US47241821. 9 Grant, Nico, “Google Fires Engineer Who Claims Its A.I. Is Conscious,” New York Times, July 23, 2022, https://www.nytimes.com/2022/07/23/technology/google-engineer-artificial-intelligence.html. 10 Branch, Jr., John E., “Machine writing is becoming more human – all too human, in some cases,” Fast Company, September 19, 2022, https://www.fastcompany.com/90784449/machine-writing-is-becoming-more-human-all-too-human-in-some-cases. 11 “Rapid Response: You Should Be Running Toward AI with Eric Schmidt,” Masters of Scale with Reid Hoffman podcast, https://podcasts.apple.com/us/podcast/rapid-response-you-should-be-running-toward-ai-w-eric/id1227971746?i=1000541474053. 12 Lo Guidice, Diego, et. al., “Prepare For AI That Learns To Code Your Enterprise Applications (Part 2),” Forrester Research, July 8, 2021, https://www.forrester.com/blogs/prepare-for-ai-that-learns-to-code-your-enterprise-applications-part2/. 13 Gartner: Data Storytelling: Analytics Beyond Data Visualizations and Slideshows, July 19, 2021. 14 Zaidi, Ehtisham, “Utilize Self-Service Data Preparation to Ease Rising Data Engineering Challenges,” Gartner Data & Analytics Summit 2022, slide 14. 15 Teare, Gené, “Global VC Pullback is Dramatic in Q3 2022,” Crunchbase News, October 6, 2022, https://news.crunchbase.com/venture/global-vc-funding-pullback-q3-2022-monthly-recap/. 16 IDC FutureScape: Top 10 Predictions for the Future of Innovation, November 12, 2021, https://www.idc.com/getdoc.jsp?containerId=prUS48381621. 17 IDC FutureScape: Worldwide Data and Content Technologies 2022 Predictions, https://www.idc.com/getdoc.jsp?containerId=US48082521. 18 Gartner, Magic Quadrant for Data Integration Tools, August 17, 2022, https://www.qlik.com/us/gartner-magic-quadrant-for-data-integration-tools. 19 Linden, Alexander, “Is Synthetic Data the Future of AI?” Gartner, June 22, 2022, https://www.gartner.com/en/newsroom/press-releases/2022-06-22-is-synthetic-data-the-future-of-ai. 17 — TOP 10 BI & DATA TRENDS 2023
You can also read
