AWS adds file access to S3, taking on NetApp and Qumulo

AWS has added S3 Files to its S3 object storage service, creating new competition to existing file services on AWS, and suppliers that support object and file storage.

S3 is Amazon's incredibly popular object storage service. AWS is adding file services to S3 using its Elastic File System (EFS). This provides NFS support for S3 and is a managed file storage service with no need for provisioning capacity or performance management. File storage suppliers such as NetApp and Qumulo already provide combined file and object storage services in AWS. They now face more competition. 

AWS says any NFS-based application, agent, or tool can now access and work with S3 data as a file system with no change. Specifically,  S3 Files supports the NFS v4.2 and 4.1 protocols. It provides file-system-access semantics, such as read-after-write data consistency, file locking, and POSIX permissions. AWS says S3 Files provides the performance and capabilities of a file system with the scalability, durability, and cost-effectiveness of S3. 

Up until now, AWS customers wanting application and user file-based access to data kept in an AWS (S3) data lake needed to copy the object data into a file-based pool, either an AWS one or a third-party supplier's resource. Now they can stay entirely inside the S3 environment.

Azure Data Lake Storage (ADLS) provides file system semantics, file-level security, and scale built on Blob storage. ADLS supports NFS v3.0, SMB, and the Apache Hadoop Distributed File System (HDFS) as a data access layer. Admins can't natively combine file and object storage on the same Azure share, but the two live in the same storage account as Blob/Object storage.

Google Cloud does not offer combined file and object storage services like AWS, nor does Oracle Cloud Infrastructure (OCI).

With S3 Files, data is accessible through the file system and directly through S3 APIs at the same time. AWS says thousands of compute resources can connect to the same S3 file system simultaneously, enabling shared access across clusters without duplicating data. 

S3 Files works with all the existing data in S3 buckets, with no migration required. It caches actively used data for low-latency access and provides up to multiple terabytes per second of aggregate read throughput. 

Some third-party suppliers offer similar capabilities. Amazon FSx for NetApp ONTAP is a cloud-native ONTAP service running on AWS and is S3-accessible. Customers can connect S3-based AWS services and ISV applications to FSX for ONTAP file systems as if they were S3 buckets by using S3 Access Points. Dell PowerScale for AWS supports object storage through the S3 protocol as well.

Qumulo's Core software is available on AWS as Cloud Native Qumulo (CNQ), offering both file and object protocols. It has achieved more than a terabyte per second throughput and more than 1 million IOPS using standard NFS clients. CNQ also runs on Azure and Google Cloud, as does ONTAP.

The VAST AI OS, supporting file and object access, is also available on AWS, Azure, and Google Cloud.

S3 Files is available in most AWS regions. Visit the AWS Capabilities tool in Builder Center for the full list. To learn more, visit the product page, S3 pricing page, documentation, and AWS News Blog.