The enhancements include new Lumada data services, an intelligent data lake offering, a new version of Pentaho 9.0, and an emphasis on expanding Lumada beyond the industrial IoT use cases that have been its main base.

LAS VEGAS – Hitachi Vantara’s NEXT event here this year emphasized the focus of their integrated solution set from the core to the edge to the cloud, with each component receiving a major product refresh. For the edge, the news was what the company was calling Lumada 3.0, consisting of a broad expansion of Lumada services designed to expand it beyond its base in the industrial IoT space.

“Last year, we were told we were very light on the product announcements – and we have changed that this year,” said Lothar Schubert, Senior Director, Technical Marketing, Lumada and IoT, at Hitachi Vantara.

This is the 3.0 version of Lumada, but the product’s origin is rather older than that would seem to indicate.

“What became Lumada started ten years ago in the smart manufacturing space,” Schubert said. “We productized it five years ago, and we commercialized it four years ago. The focus originally was on IoT, especially in manufacturing. The Go-to-Market was very much industrial, around things like manufacturing optimization, predictive maintenance, and operations planning.”

With Lumada 3.0, the Go-to-Market has been explicitly broadened to give prominence to other verticals, and take advantage of what Hitachi Vantara – and most other people – perceive as a blossoming market opportunity.

“We see a similar expansion of the architecture out at the edge,” said Brad Surak, Hitachi Vantara’s chief product and strategy officer. “IoT shouldn’t be a separate silo for your data and applications. Your platform needs to be able to handle all of your data. In our minds, Lumada is for all of your business.

“We need a collaborative approach for AI management in the AI era,” he emphasized. “Every data management doesn’t have to be a one-off project that takes days to months to complete. We see big opportunities if you can do this in AI and analytics, governance and compliance and operational agility. So we are announcing Lumada Data Services, a suite of services that brings data management into the AI era.”

Lumada Data Services are accessible through a common cloud console, and can be deployed in the cloud, in the data center or at the edge work. They will be available starting this month and into the next year.

“They are targeted at both business users and data analysts, and utilize a metadata layer that goes from core to edge to cloud,” Schubert stated

One of these is a Data Flow Service.

“Data pipelines are the lifeblood of organizations,” said Sanjay Chikarmane, SVP Products, Hitachi Vantara. “This service provides a tailored experience to each one of the personas involved, like engineers and scientists, so that they can collaborate. We have converged both structured and unstructured data Most analytics has been structured but AI illuminates unstructured data. You need a single tool for both. This operates with Pentaho for structured data and Hitachi Content Intelligence for unstructured data.”

Three core data products around Lumada are the Lumada Data Lake, Pentaho 9.0, and Lumada Data Optimizer – complemented by Lumada Edge Intelligence and additional Lumada solutions.

“The time is right for a new approach to go beyond single data lakes,” Chikarmane said. “The Lumada Data Lake is a next-generation data lake.”

This essentially means two things.

“It can intelligently place data in the optimal location, such as intelligently placing data in a secure data centre for privacy instead of in S3, or as a built- in catalogue to tell where data sets are, and how to retrieve them,” Chikarmane noted. “In addition, it is self-governing, so you can see data zones, and you can set the policies for raw, curated or published data. Self-governing goes a long way to mitigating this problem of data swamps.”

The Lumada Data Optimizer is designed to improve Hadoop resource utilization with an intelligent data tiering solution that integrates tightly with Hadoop Distributed File Systems.

“Data Optimizer services bring the economics of the cloud into on-prem,” Chikarmane stated. “It moves less-used data into inexpensive object storage in HCP. It’s all transparent to your Hadoop-based applications – a very non-disruptive way to bring immediate cost relief to any Hadoop data lake.”

Lumada Edge Intelligence is another new offering to manage data and analytics at the network edge.

“Lumada Edge intelligence will provide you with preconfigured hardware and software configurations for things like data movement and AI at the edge,” Surak said. “It’s part of our growing portfolio under Lumada.”

Almost buried amid the net-new announcements was Pentaho 9.0. A key here is new data flow services for data pipeline sharing, management and monitoring.

“This Pentaho release is really about data services, including the data flow management,” Schubert said. “Business analysts will now have the power to execute pipelines by themselves – self service data flows. That’s huge.”

Also extremely significant, Schubert added, is extensive new telemetry in this release.

“We have also added more integrations,” he said. “This includes a mainframe integration that financial and other customers have asked us for, as well as one into cloud warehouses like Snowflake.”

A new Web-based design wasn’t quite ready for this Pentaho release, Schubert said to look for it in an imminent minor one.

Lumada has been more of a direct play than the storage components within the Hitachi Ventara portfolio. Fewer than 30 per cent of partners handle it, and the rollout of 3.0 is more gradual to partners than say, the rollout of the VSP 5000 high-end array. Schubert said that the broader focus of Lumada going forward, together with the company’s overall focus on and end-to-end strategy which integrates core storage with the edge and the cloud, will inevitably make Lumada more important to more partners.

“This transition broadens the importance of partners learning it,” Schubert said. “Ultimately, reaching the highest level of DataOps requires orchestration across data pipelines. I also think that partners will increasingly look for more higher value services to differentiate themselves.”

Lumada Data Optimizer for Hadoop will be available in November 2019. Lumada Edge Intelligence will be available in January 2020. Pentaho 9.0 will be available in February 2020. Lumada Data Lake will be available in March 2020.