S3 Storage: What it is, how it works, use cases

11 November 2025 Antony Adshead Block, file, object, etc, Cloud storage, Learning, Storage arrays, Storage media 0

Discover how AWS S3 object storage works, its classes, use cases, and S3-compatible on-prem options that bring cloud-scale storage to hybrid environments.

Amazon’s Simple Storage Service (S3) is the cornerstone of cloud object storage and one of the earliest building blocks of AWS’s global infrastructure. Since its launch in 2006, S3 has grown into the de facto standard for object storage in the public cloud — and by extension, the reference point for many on-premise systems that mimic its API.

S3 stores hundreds of trillions of objects — from documents and images to videos and backups — representing many exabytes of data. Its scalability, flexible pricing and metadata-driven architecture make it indispensable for cloud-native workloads, analytics and AI.

How does S3 object storage work?

Unlike file or block storage, which rely on hierarchical directories or fixed volumes, S3 stores data as objects — self-contained packages comprising the data itself, unique identifiers and descriptive metadata.

Each object is addressable by a unique key rather than a traditional file path. The underlying infrastructure can physically place objects anywhere in the AWS global system, while the metadata and identifier ensure instant retrieval. Metadata can include size, date stamps, content type, encryption settings, versioning, and even user-defined fields such as classification or department tags.

Objects can be up to 5 TB in size, uploaded in multi-part segments of up to 10,000 parts. This makes it possible to handle extremely large datasets efficiently via the console, command line or API.

What is the structure of S3 buckets?

All S3 objects live inside buckets, the top-level containers that define storage boundaries, access policies and regions. Each bucket resides in a specific AWS Region, which determines cost, latency and data sovereignty characteristics.

Buckets can contain lifecycle policies, replication rules, logging, event notifications and encryption settings. Access is managed through AWS Identity and Access Management (IAM), bucket policies and Access Control Lists (ACLs).

Though the S3 console allows users to create “folders”, these are only visual groupings — not actual directory structures. In practice, S3 is completely flat, with object keys providing logical organisation.

What commands or APIs does S3 use?

S3 operations are built on familiar HTTP methods — such as GET, PUT, and DELETE — which can be executed via the AWS web console, command-line tools or programmatically through the S3 REST API.

Developers can script these actions or embed them into application code to:

  • Create, list or delete buckets.
  • Upload, copy or move objects.
  • Manage access permissions and metadata.
  • Receive notifications of object events.

This simple API design is one reason why S3 has become the universal language of object storage, adopted by many third-party systems beyond AWS itself.

What are the S3 storage classes?

AWS S3 offers multiple storage classes, allowing customers to optimise cost and performance according to data access frequency and retention needs:

  • S3 Standard – for frequently accessed data; high durability and low latency.
  • S3 Express One Zone – single-zone millisecond latency for high-performance workloads.
  • S3 Standard-IA (Infrequent Access) and One Zone-IA – cheaper for data accessed less often, with small retrieval fees.
  • S3 Glacier Instant Retrieval, Glacier Flexible Retrieval, and Glacier Deep Archive – for archival data, with retrieval times ranging from milliseconds to 48 hours.
  • S3 Intelligent-Tiering – automatically moves data between tiers based on access patterns, optimising cost without user intervention.

This tiered model ensures users pay only for the access speed and availability they need.

What are the main use cases for S3?

Because S3 is scalable, durable and metadata-rich, it excels at unstructured and bulk storage scenarios. Typical use cases include:

  • Data lakes and analytics repositories.
  • Backup and disaster recovery storage.
  • Content delivery and media archives.
  • Long-term compliance or regulatory storage.
  • AI and machine learning training datasets.

However, object storage is not ideal for transactional workloads or low-latency databases, where block storage offers more consistent performance.

What on-premises or private S3 options exist?

Although S3’s natural home is the AWS cloud, many organisations need local or hybrid options for data residency, latency or compliance reasons. AWS offers S3 on Outposts, bringing native S3 APIs and buckets to on-premise infrastructure.

In addition, several vendors provide S3-compatible systems that replicate AWS’s API and behaviour, including Cloudian (HyperStore), Dell ECS, MinIO, NetApp StorageGRID, Pure Storage, QNAP, Red Hat, Scality (via HPE) and StoneFly. These allow customers to build hybrid architectures or private clouds using the familiar S3 interface while maintaining on-site control.

The takeaway

S3 has become the blueprint for modern object storage. Its simplicity, scalability and universal API have made it the go-to platform for unstructured data, analytics, and backup. As hybrid and multi-cloud models grow, S3’s reach extends far beyond AWS — shaping a whole ecosystem of compatible storage products that bring cloud-like capabilities to private infrastructure.

Read more about storage systems

File, block and object storage: How do they work in the cloud era? File, block, and object storage remain core to data management in the cloud era. Learn how they differ, how they evolved, and where each model fits best

Vector databases: The hidden engine of AI. Vector databases power modern AI by storing and searching high-dimensional data. Learn how vector embeddings work, their effect on data storage, plus a survey of the key providers.