Storage Backends

vcpkg-harbor supports multiple storage backends through a plugin architecture.

Available Backends

MinIO (Default)

MinIO is an S3-compatible object storage server. It’s the default backend and works great for on-premises deployments.

VCPKG_STORAGE_TYPE=minio
VCPKG_MINIO_ENDPOINT=localhost:9000
VCPKG_MINIO_ACCESS_KEY=minioadmin
VCPKG_MINIO_SECRET_KEY=minioadmin
VCPKG_MINIO_BUCKET=vcpkg-harbor

Filesystem

Store packages directly on the local filesystem. Best for development and testing.

VCPKG_STORAGE_TYPE=filesystem
VCPKG_STORAGE_PATH=/var/lib/vcpkg-harbor/cache

Warning: Network Storage When using filesystem storage with NFS or other network filesystems, ensure proper locking is configured to avoid corruption.

AWS S3

Native AWS S3 support for scalable cloud storage.

VCPKG_STORAGE_TYPE=s3
VCPKG_S3_BUCKET=my-vcpkg-cache
VCPKG_S3_REGION=us-west-2
VCPKG_S3_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
VCPKG_S3_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

For IAM role-based authentication (recommended in AWS):

VCPKG_STORAGE_TYPE=s3
VCPKG_S3_BUCKET=my-vcpkg-cache
VCPKG_S3_REGION=us-west-2
# Leave credentials empty to use IAM role

Azure Blob Storage

Azure Blob storage for Microsoft Azure deployments.

VCPKG_STORAGE_TYPE=azure
VCPKG_AZURE_CONNECTION_STRING="DefaultEndpointsProtocol=https;AccountName=..."
VCPKG_AZURE_CONTAINER=vcpkg-harbor

Or with account key:

VCPKG_STORAGE_TYPE=azure
VCPKG_AZURE_ACCOUNT_NAME=myaccount
VCPKG_AZURE_ACCOUNT_KEY=yourkey
VCPKG_AZURE_CONTAINER=vcpkg-harbor

Google Cloud Storage

GCS for Google Cloud Platform deployments.

VCPKG_STORAGE_TYPE=gcs
VCPKG_GCS_BUCKET=my-vcpkg-cache
VCPKG_GCS_PROJECT=my-gcp-project
VCPKG_GCS_CREDENTIALS_FILE=/path/to/service-account.json

For workload identity (recommended in GKE):

VCPKG_STORAGE_TYPE=gcs
VCPKG_GCS_BUCKET=my-vcpkg-cache
VCPKG_GCS_PROJECT=my-gcp-project
# Leave credentials file empty to use workload identity

Backend Selection Guide

Use Case	Recommended Backend
Local development	Filesystem
On-premises	MinIO
AWS deployment	S3
Azure deployment	Azure Blob
GCP deployment	GCS
Hybrid/multi-cloud	MinIO (self-hosted)

Performance Considerations

Caching

All backends stream data directly without buffering entire packages in memory. This allows handling large packages efficiently.

Concurrency

The async architecture supports many concurrent connections. Tune VCPKG_SERVER_WORKERS based on your workload.

Network

For cloud storage backends, deploy vcpkg-harbor in the same region as your storage to minimize latency.

Creating Custom Backends

vcpkg-harbor uses entry points for backend discovery. To create a custom backend:

Implement the StorageBackend protocol
Register via pyproject.toml entry points

See Architecture for details.