Skip to main content

MinIO/S3 Configuration

This document outlines the environment variables required to configure the application's object storage client for services like DSAR exports.

When EXPORT_STORAGE=minio, the following variables are used to connect to an S3-compatible object store:

  • EXPORT_ENDPOINT: The endpoint of the S3 service.
    • Example: localhost:9000
  • EXPORT_BUCKET: The name of the bucket to use for uploads.
    • Example: compliance-exports
  • EXPORT_ACCESS_KEY: The access key for the S3 service.
    • Example: minioadmin
  • EXPORT_SECRET_KEY: The secret key for the S3 service.
    • Example: minioadmin
  • EXPORT_PREFIX: An optional prefix to prepend to all object keys.
    • Example: dev/
  • EXPORT_MINIO_USE_SSL: Set to true or 1 to use SSL (HTTPS). Defaults to false.
    • Example: false

When EXPORT_STORAGE is not set to minio, the application defaults to using the local filesystem. The output directory can be configured with:

  • EXPORT_DIR: The local directory to store exported files.
    • Default: ./tmp/exports

Local testing with MinIO (DSAR export)​

Use the built-in DSAR → MinIO smoke to verify object uploads:

  1. Start MinIO locally (console on http://localhost:9001, API on http://localhost:9000) and ensure the bucket exists (e.g. s3-test).
  2. Start the backend with the MinIO envs set before launching the server: 
    export EXPORT_STORAGE=minio
    export EXPORT_ENDPOINT=http://localhost:9000   # scheme allowed; path is stripped
    export EXPORT_BUCKET=s3-test
    export EXPORT_ACCESS_KEY=minioadmin
    export EXPORT_SECRET_KEY=minioadmin
    export EXPORT_MINIO_USE_SSL=false
    export EXPORT_PREFIX=dsar/
    go run ./cmd/server
  3. Configure the local MinIO client (optional, for listing): 
    mc alias set minio http://localhost:9000 minioadmin minioadmin --api S3v4
  4. Run the end-to-end smoke: 
    cd backend/tests
    ./dsar_minio_e2e.sh
    The script uses auth_login_helper.sh to mint an owner token automatically—no manual token export is required. It enqueues a dsar_export job, polls completion, and lists objects in the bucket.

Expected result:

  • New objects in s3-test/dsar/<job>.zip and …-receipt.json.
  • Ledger entries at /api/v1/compliance/ledger with artifact_ref/receipt_ref pointing at the MinIO URLs.