SwiftStack upgrades platform for Big Data and advanced machine learning with SwiftStack 7

Both the enhancements to the core platform and services around it have a strong emphasis on increasing performance and AI at the edge.

Joe Arnold, SwiftStack’s Co-Founder and Chief Product Officer

Multi-cloud data storage and management vendor Swiftstack has announced SwiftStack 7, the new version of their platform. The idea here is not just to add some new services or capabilities but to evolve the platform fundamentally so that it can handle the most demanding workloads occasioned by today’s data and analytics demands.

“We don’t do a lot of big product launches, but this one is exciting,” said Erik Pounds, SwiftStack’s Vice President of Marketing. “It shows how we are focusing our clients to be more data driven.”

The proliferation of data-intensive applications required a redo of the platform.

“You now have tons of sensors collecting tons of data – like autonomous vehicles, medical, media, gaming, and connected cities,” said Joe Arnold, SwiftStack’s Co-Founder and Chief Product Officer. “All of these have big data challenges. That’s what we have built this platform for. Today’s demands require a new storage and data management architecture, with data control separate from the core infrastructure, a true scale-out storage, and a single global namespace.”

The new platform will support data sets that scale to hundreds of petabytes and even exabytes, at speeds of over 100GB per second.

“We are now multiple times faster than competitors, with a linear ability to scale,” Arnold said. “We are evolving SwiftStack to be the platform for intelligent data to enable deep learning at ultra-scale.”

SwiftStack 7 also adds some new features which both enable and leverage this new to enable this ultra-scale performance. ProxyFS Edge extends SwiftStack’s ProxyFS file service to the edge.

“We previously released ProxyFS for file system data,” Arnold stated. “Now we have built something that runs in a Docker container or in a Linux container to run at the edge. This is the first parallel clustered file system for SwiftStack and can handle the large footprint of data around those applications at the edge.”

ProxyFS Edge is optimized for large data sets and around machine learning and training workloads.

“It is a whole new stack we have built out at the edge,” Arnold said.

Another enhancement of an existing component is 1space File Connector, which extends SwiftStack’s 1space solution that creates a single cloud namespace for multi-cloud management.

“1space File Connector brings enterprise file data into the cloud namespace, and lets cloud-native applications easily access data through S3 or Swift object APIs,” Arnold stated.

“The File Connector works by connecting into an existing file system, and allowing it to be accessible as an Object  via S3,” Pounds said. “Data can be moved from that file system to the object storage system.” This lets businesses and organizations modernize their workflows incrementally and apply 1space data placement and protection policies to file data when needed.

A new service implemented as part of SwiftStack 7 is SwiftStack PRO [Professional Remote Operations]. Developed from best practices of SwiftStack customers, it is an optional offering for the 24×7 remote management and monitoring of SwiftStack production clusters.

“SwiftStack PRO combines software and professional services to remotely manage and monitor customer storage environments. Pounds said. “It is always checking cluster health, managing the alerting system, keeping the system tuned, and doing frequent operational reviews. It also provides data immutability – like with points of time snapshots with neural networks. This lets you test against what data looked like a year ago.”

For now, this isn’t something that partners can sell.

“Right now, it’s something that we are offering direct,” Pounds indicated. “We will evaluate that as our services relationships move forward. We have always had a professional services team, but this is the first time, we have had a service to monitor customer management over time.”

“We started piloting this offering with two customers over the last 12 months and are now launching it more broadly,” Arnold added.

With SwiftStack 7 developing AI to a higher degree than previously, the plan is to deliver a White Paper on the new performance and scale requirements of AI with SwiftStack.

“Weve been working with clients in AI Deep learning for a year and a half, but have yet to put out an AI architecture for what this looks like,” Pounds said. “We will do a white paper on the SwiftStack AI architecture that shows how a system like SwiftStack can perform at ultra-scale.”

The white paper “The SwiftStack AI Architecture” will be available at swiftstack.com/ai.