Hitachi Vantara Overview Pentaho 8.0 and 8.1 Roadmap - Pedro Alves - it-novum
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Hitachi Vantara Overview Pentaho 8.0 and 8.1 Roadmap Pedro Alves
Safe Harbor Statement The forward-looking statements contained in this document represent an outline of our current intended product direction. It is provided for information purposes only and is not a commitment to deliver any new or enhanced product or functionality, or that we will pursue the product direction described. Facts and circumstances may occur which may impact current plans, resulting in changes to the information in this presentation. This information is current only as of the date it is made and should not be relied upon in making purchasing decisions. The development, release (if at all), and timing of any features or functionality described for the Pentaho products remains at the sole discretion of Pentaho.
Hitachi Vantara
Advanced X as a Data
Analytics Service Scientists
OT/IT
Software
Expertise
Hitachi Data
Systems
Hitachi VantaraLumada
Pentaho
Solutions
Infrastructure
Partner Vertical
and Services Machine
Ecosystem Expertise Learning
Hitachi Vantara:
A New Company With 100+ Years of Heritage
HITACHI RECOGNIZED
HITACHI, LTD.
VANTARA CULTURE
Founded in 1910
10,000 customers US$81.8B FY16
7,000 employees 950+ subsidiaries
2,000 partners 304,000 employees
100 countries #1 worldwide for
“I’d posit that the reason these
patent applications
Trusted by 85% of in big data analysis
three values have lasted for
FORTUNE® Global more than 100 years at Hitachi – foundation technology
100 and will likely last for 100 more –
is because of the emphasis they #71 in the 2017
place on human relationships.” FORTUNE® Global
– Inc. Magazine
500
Hitachi Vantara Is Trusted by 85% of the
FORTUNE Global 100 and 10,000 Customers
FINANCE I . T. S E R V I C E S H E A LT H C A R E R E TA I L
First People’s
Hospital of Yibin
M AN U FAC T U R I N G SERVICES/OTHER MEDIA TELECOM
Xinhua
News Agency
Hitachi Vantara Is Trusted by 85% of the
FORTUNE Global 100 and 10,000+ Customers
H I TA C H I CORE INFRASTRUCTURE H I TA C H I
A N A LY T I C S CONTENT
PORTFOLIO/ PORTFOLIO
P E N TA H O
First People’s
Hospital of Yibin
Xinhua
News Agency
Xinhua
News Agency
Rely on Real, Ongoing Innovation
4,900 R&D employees, Over $5.4B revenue
$2.8 billion annually annually from IoT
committed to R&D
Computerworld:
2,500+ patents awarded
Hitachi Is #1 Most
in technologies for
Powerful IoT Company
big data analysis
Thomson Reuters:
119,000+ patents worldwide “Top 100
Global Innovators”
Hitachi’s Unique Value
CITY CLOUD
INFRASTRUCTURE
Only Hitachi Vantara combines
INDUSTRIAL
deep IT, OT OT IoT IT
108 andISdomain
59 expertise
ANALYTICS
IN
BUSINESS
YEARS OUR YEARS
APPLICATIONS
DNA
CONSUMER BIG DATA
Every Company Must Now
Be a Technology Company
We all need to find the most value in our data.
That’s with big data,
That means fundamental change to business and operations.
but Many
what have
abouta REALLY
long waybig data?
to go:
Only 40% of industries are truly
There will be nearly 21 billion connected
digitized (McKinsey)
“things” by 2020 (Gartner)Swimming in Data But Searching for Insight
The full potential of data is elusive.
A precious few companies have discovered
how to find the full value of their data.
Hitachi Vantara is a partner with
data-driven solutions and services
for people who see data as an opportunity.We Help the Two Basic Business Directives:
Make Money and Save Money
Top-Line Benefit INTERNET OF
THINGS
BUSINESS
Asset Avatars
INSIGHT Edge to Outcomes
Data Orchestration Co-Creation
Embedded
DIGITAL
Analytics
WORKPLACE Data Ingest and
INFRASTRUCTURE Enterprise Mobility Blending
MODERNIZATION Content Intelligence
Object
Private, Public,
Hybrid
Flash
Software-Defined
Bottom-Line BenefitOur Core Beliefs
Data is at the center of all Your data will outlive the
you do and is the key to application that created it and
your transformation. the underlying infrastructure.
We want you to own your data. We enable you to have your
Not us. data and insight when, where
and how you need it.
Metadata is the new data.
© Hitachi Vantara Corporation 2018. All Rights ReservedOur Commitment: Deliver Outcomes That Matter for Business And Society
Pentaho 8
Future Vision: A Single Consistent Experience
Data Engineering Data Prep Analytics
Ingestion Processing Blending Data Delivery Data Analysis
Discovery & Dashboards
Administration Security Lifecycle Data Dynamic Data Monitoring Automation
Management Provenance PipelinePentaho 8.0
Pentaho 8.0 • Connect to Kafka streams
Challenge #1
Broadens connectivity
Data volumes and velocity • Stream processing with Spark
to streaming data
are growing exponentially • Big data security with Knox
sources
Challenge #2 Pentaho 8.0 • Enhanced Adaptive Execution (AEL)
Processing and storage Optimizes processing • Native Avro and Parquet handling
resources are constrained resources • Worker nodes for “Scale-out”
Pentaho 8.0 • Data explorer filters
Challenge #3
Boosts team
Shortage of Big Data talent • Improved repository UX
productivity across
and lack of productivity • Extended operations mart
the pipelinePlatform and Scalability
Worker Nodes
New themeWorker Nodes for Scaling Out
Scale work items across multiple nodes
(containers) Worker Node (a)
Distribute and Scale Worker Node (b)
Easily add and remove resources as
Worker Node (c…)
required
Monitor and balance changing workloads
NEW in Pentaho 8.0
Deploy on premise, cloud and hybrid
Container framework
Orchestration framework
Node monitoring
Enhanced HA implementationWorker Nodes Architecture
WORKER NODES
Orchestration Framework
Orchestration
(Scheduler, monitoring, security, etc.) Powered by …
Controller (HA)
Pentaho Clients Master
(Working) Master Master
(Standby) (Standby)
Pentaho Server Container Framework
WN 1 WN 2 WN …n
Pentaho Repository
e.g. KJB e.g. KTR “Executor”New Theme - Ruby
Data Integration
Streaming support!
Run configurations for Jobs
Filters in Data Explorer
New Open / Save experienceStreaming for Time Sensitive Insight
Enable use cases that require real-
time processing, monitoring and
aggregation
Real-time device monitoring
Log-file aggregation
Notifications
And more… NEW in Pentaho 8.0
Kafka Producer Step
Kafka Consumer Step
Get records from stream Step
Spark streaming via AELRun configurations for Jobs Run Configuration Execute on server Leverage worker nodes
Pentaho 7.0 – Data Explorer
Access visualizations during data prep for inspection and prototypingData Explorer Filters
Enhanced data inspection in PDI
Identify data to be cleaned or
removed
Deliver data to the business more
quickly
New in Pentaho 8.0
Numeric filters
String filters
Include/Exclude data pointsNew PDI Repository Dialogs
Enhanced user experience for
content repository
‒ Consolidated open and save dialogs
‒ Enhanced search
‒ “Recently used” files option
‒ “Sticky” directories
Business Benefits
Increased productivityBig Data
Improvements on AEL
Big Data File Formats - Avro and Parquet
Big Data Security - Support for Knox
VFS improvements for Hadoop ClustersPentaho 7.1 – Adaptive Execution for Spark
PDI
No Coding
Pentaho
Kettle
Build Once
Execute on Any*
Engine
*Currently Available EnginesEnhanced Adaptive Execution
Simplified setup HADOOP CLUSTER
Eliminated “Zookeeper” component
Spark/Hadoop Processing Nodes
Reduced number of setup steps PDI AEL-Spark
Client Daemon on
Edge Nodes Spark
Hardened deployment Executors
Fail-over at the edge
Hadoop/Spark Compatible
Storage Cluster
Kerberos impersonation for client
HDFS Azure Amazon
AEL-Spark Storage S3
Engine Etc…
More flexible (Spark Driver)
Support multiple run configurations
Customize cluster settings per job typeNative Avro and Parquet Handling Visual handling of data files with common big data file formats Reading and writing files with specific steps Natively executes in Spark via AEL NEW in Pentaho 8.0 Avro Input and Output Steps Parquet Input and Output Steps
Knox Gateway for More Secure Clusters Pentaho can interact with data services in Knox gateway protected Hadoop clusters* Benefits • Highly secured Knox-protected Hadoop clusters can now be integrated in ETL pipeline *Knox is only available for Hortonworks HDP distro.
VFS improvements for Hadoop Clusters
In order to simplify the overall lifecycle of
jobs and transformations we made the
hadoop clusters available through VFS, on
the format
hc://hadoop_cluster/.Others
Ops Mart for Oracle, MySQL, SQL Server
Platform password security improvements
PDI mavenization
Documentation changes on help.pentaho.com
Feature Removals:
Analyzer on MongoDB
Mobile Plug-in (Deprecated in 7.1)Product Documentation Updates
MindTouch Documentation Improvements (continued)
Streamlining of site content pages:
‒ Simpler site structure and click paths to accelerate user content access
‒ Eliminates pages with little content value
‒ Fun fact: 7.1 contained 1200+ pages; 8.0 contains 900+ pages with no elimination of valuable content
Ongoing migration of big data steps from the wiki pages to the MindTouch Doc site
Business Benefits
Improved access to on-line documentationPentaho 8.0 - Detailed
Data Integration Big Data
• Drag and drop numeric and non-numeric filters in Data Explorer • Simplified Spark AEL setup
• Chart actions in Data Explorer to include or exclude data • Spark AEL fail-over and load balancing
• Cleaner and easier design/UX for PDI Repository dialogs • Support for Spark AEL on Hortonworks
• Consolidated PDI Repository dialogs, replacing multiple pop-ups • Enhanced security for Spark AEL including
• PDI Repository Dialogs remember last opened directory Kerberos impersonation and secure connectivity
• Jobs can leverage Run Configurations for seamless user • Customization of Spark AEL processing (i.e.
experience memory and other settings)
• Run Configuration option to run jobs on Pentaho Server • Parquet input and output steps
• Avro input and output steps
Streaming
• Big Data Security with HDP Knox Gateway
• Kafka Producer step for outputting streaming data
• VFS Improvements for named Hadoop clusters
• Kafka Consumer step for streaming data input
• Get Records from Stream step for stream processing
• Ability to run AEL on Spark Streaming for transformations that use Additional Items
Kafka streaming ingest • Big Data Sandbox VM updates
• Cumulative service pack framework
Enterprise Platform • Documentation improvements on
• Worker Nodes Scale-Out solution to drive superior agility help.pentaho.com
• Ops Mart for Oracle, MySQL, SQL Server
• Ruby Theme – new platform branding for browser and client tools
• Platform password security improvements
• PDI Mavenization for infra alignmentPentaho 8.1
Pentaho 8.1 – Investment themes
Focus on Cloud
‒ Entry to Google Cloud Platform
“Must-have” response to
‒ Make AWS Easy
market demand
‒ Validate WN for cloud, hybrid deploy
Harden Core Capabilities
‒ Continue establishing recent innovations
Targeted improvement for
‒ Update overlooked platform areas unfinished/neglected areas
‒ 20% “Continuous Improvement”
Begin the Next Phase
‒ “All-in” on Foundry
Get started now.
‒ Web-based design and data exploration
Release vehicles TBD.
‒ Vantara “solution” integrationsQ&A
You can also read
