Datadobi sees the big market need for this not for moving S3 data from the cloud back to on-prem, but to provide more flexibility in choosing S3 clouds, in the same way that Datadobi has enabled NAS migrations between different vendors.
Carl D’Halluin, Datadobi’s Chief Technology Officer
Today, Datadobi, which has made unstructured data migration software for large complex NAS migrations, is expanding their portfolio. The new DobiMigrate 5.9 release adds new support for S3-to-S3 object migrations.
“DobiMigrate 5.9 migrates both objects and their metadata,” said Carl D’Halluin, Datadobi’s Chief Technology Officer. “We now can do the same for S3 – both AWS and its competitors – that we have done for NAS storage, as well as for companies that do on-prem S3 object storage.”
While the initial surge in interest from customers experiencing sticker shock at AWS S3 costs wanting to move back from the cloud has subsided somewhat, D’Halluin said that many customers still want more flexibility.
“There was initial concern about both latency and S3 fees,” he noted. “Latency can be addressed, but the egress fees – typically 9 cents per GB and 5 cents for over 150 TB a month, cannot. Not all S3 clouds are as expensive as AWS, so some customers still want the flexibility to move some or all of their data.” He noted that customers typically will ask for an ROI assessment from them about moving compared to staying in AWS S3. He also indicated that the increasing range of options in the industry – like Pure Storage adding object on their all-flash – has led to more queries from customers asking where their object belongs and if they should move it.
“A year and a half ago, there was a big flood of interest in repatriation of data, and we have certainly seen that cool down,” said Michael Jack, Datadobi’s co-founder and Vice President of Global Sales. “We don’t, however, see a great deal of distance between the decision-making process here and in NAS takeout, where customers move from one vendor to another vendor. We believe that the ability to move their S3 data in the same way that they move their NAS data will give them the freedom to move when they want to move, rather than be locked in.”
D’Halluin said that the technology is essentially an extension of what they have done with NAS, using parallel scanning and copying. They scan the source S3 bucket with a patent-pending process, listing objects, as well as for the destination bucket. They then calculate the delta between them, comparing the source and the destination. All this is done on-prem, behind the data centre firewall. It’s not a cloud service.
“What is critical for enterprise migration is keeping impact on the organization minimal, and Datadobi does all the work before the short maintenance window,” D’Halluin said. “We thought this would be simple because it’s a well-defined protocol, new and modern, but we learned that storage systems do not sort their data in the same order. We had to learn a lot about storage order, and about error codes, and how to get the data through effectively.”
Some things still remain on the roadmap
“We don’t have AWS Glacier yet, or object locks and object tags, or non-current object versions, although the latter is not a very big ask at the moment,” D’Halluin said.
DobiMigrate 5.9 is scheduled to be released to general availability some time in the middle of the summer.
