Storage API

Storage API gives each application an isolated S3-compatible object namespace. File bytes move directly between the client and object storage through short-lived presigned URLs. StarEcho handles permissions, capability policy, upload sessions, metadata, quotas, and auditing without proxying large payloads.

Default capabilities

Capability	Default
Single PUT limit	1 GiB, effective per application
Multipart object limit	5 GiB, effective per application
Default part size	16 MiB, configurable from 8–64 MiB
Maximum parts	10,000
Upload URL lifetime	1 hour, configurable from 60 seconds to 7 days
Multipart session	24 hours, automatically aborted after expiry
Capacity quota	2 GiB
Range and HEAD	Supported

Do not hard-code these values. Query capabilities before choosing an upload strategy. An app owner may lower the single PUT limit immediately; increasing it requires administrator approval. Lowering a limit does not delete existing objects above that limit.

Authentication and replay protection

The API base path is /storage/api/. It accepts:

• Authorization: Basic base64(client_id:client_secret) for trusted servers and internal tools only.
• Authorization: Bearer <access_token> for a user OIDC identity.

Never ship client_secret in a browser, game build, or desktop client. Basic and internal Cloud Function calls operate as the application. Direct Bearer calls additionally require storage.read for reads, storage.upload for upload/confirm/multipart operations, or storage.delete for deletion. A user-facing upload endpoint should perform its business permission check in a Cloud Function, then use $storage to issue short-lived credentials.

Storage API requests also require X-Timestamp and a unique 16–64 character X-Nonce. These replay headers are not added to presigned URLs. Treat a presigned URL as an opaque method-bound credential: do not decode or reorder its query, change its Host, or substitute a custom domain.

Capabilities and usage

GET /storage/api/capabilities
GET /storage/api/usage

Example capabilities:

{
  "provider": "cloudflare-r2",
  "maxSingleObjectBytes": 1073741824,
  "multipartUploadSupported": true,
  "minPartBytes": 8388608,
  "maxPartBytes": 67108864,
  "defaultPartBytes": 16777216,
  "maxParts": 10000,
  "maxObjectBytes": 5368709120,
  "checksumAlgorithms": ["md5", "sha256"],
  "rangeDownloadSupported": true,
  "headSupported": true,
  "uploadUrlExpiresSeconds": 3600
}

Usage includes quotaBytes, usedBytes, reservedBytes, remainingBytes, activeMultipartUploads, and maxConcurrentMultipartUploads. Valid unfinished uploads reserve capacity so concurrent URL issuance cannot over-allocate the app quota. See “Audit, alerts, and retention” for the monthly download-URL issuance diagnostics and the explicit unavailable marker for actual transfer bytes.

Single upload (compatible API)

Existing small-file integrations do not need to migrate:

1. POST /storage/api/upload returns a PUT URL and one-time confirmation token.
2. The client uploads the final binary body directly.
3. POST /storage/api/confirm reads authoritative object metadata and confirms it.

Issue a URL

POST /storage/api/upload
Content-Type: application/json

{
  "path": "releases/app.apk",
  "size": 253023298,
  "contentType": "application/vnd.android.package-archive",
  "sha256": "4ed50bf91e8c3c9e5e6a30ea4fcc4827aa91dd26c3d1e9e1b98ca524960b9d17",
  "metadata": { "versionName": "2026.07.29" },
  "allowOverwrite": false,
  "expiresIn": 3600,
  "bindContentLength": true
}

path is a 1–512 character relative object path without a leading slash, backslashes, or . / .. segments. size is the final HTTP body byte count, not a character count.

For compatibility, an old call that omits allowOverwrite retains the historical overwrite behavior. New integrations should explicitly use false, changing it to true only for an intentional replacement.

The response includes url, token, expiresIn, requestedSize, contentType, contentLengthBound, requiredHeaders, and checksum trust information. New clients should pass bindContentLength: true; the signature then binds the exact HTTP body byte count and MIME type, and the client must send the returned required-header values. A browser derives Content-Length from its Blob/File body, while a Node client should set it explicitly. For existing Unreal Engine and S3 HTTP integrations, omitting the additive option retains the legacy unsigned-header behavior. At confirmation, StarEcho always reads the authoritative byte length and type through HeadObject: when the caller explicitly passes size/contentType in the confirm body, a mismatch is rejected; cloud-function and legacy callers that omit them adopt the actual object-store values for metadata and quota.

A single upload first targets a non-enumerable staging key. After validation, confirm performs an internal object-store copy to a version key that has no PUT credential, atomically switches the compatible storage_file.path mapping, and reclaims the staging object. Replaying a still-unexpired upload URL can therefore modify only unpublished staging data, never the confirmed download object. Startup and hourly cleanup delete expired single-upload staging objects and tokens, record upload_expire, and retain failed provider deletions for retry. Pre-migration in-flight tokens, existing deterministic physical keys, and logical paths remain compatible.

Confirm

POST /storage/api/confirm
Content-Type: application/json

{
  "path": "releases/app.apk",
  "token": "opaque-one-time-token",
  "size": 253023298,
  "contentType": "application/vnd.android.package-archive"
}

The result includes path, actual size, contentType, etag, sha256, sha256Verification, sha256VerifyStatus, business metadata, and timestamps. Confirmation uses HeadObject to read the authoritative size and type (validated when explicitly supplied in the confirm body; otherwise the actual values are adopted) and records the ETag.

sha256Verification: "declared" means a trusted client or developer tool supplied the digest, but the platform has not yet read the entire object. Default-R2 objects are durably queued for a throttled background job whose sha256VerifyStatus progresses through pending, processing, verified, or failed. Only after StarEcho streams every byte and the digest matches is sha256Verification atomically promoted to platform_verified. Custom OSS objects are not read automatically because doing so could create third-party transfer charges.

Owners can requeue a failed job:

POST /storage/manage/{clientId}/files/verify-sha256
Authorization: Bearer <owner_access_token>
Content-Type: application/json

{ "path": "releases/app.apk" }

R2 multipart SHA-256 is a composite checksum, not a FULL_OBJECT SHA-256, so neither it nor a multipart ETag is reported as platform verification. The worker uses durable database state and a global lock across PM2 instances, verifies one object at a time, and automatically retries transient failures up to three times.

Multipart and resume

Multipart is recommended above 100 MiB. The public uploadId is a StarEcho opaque session identifier; the provider upload ID is never exposed.

1. Create a session

POST /storage/api/multipart/create

{
  "path": "website/releases/android/2026.07.29/xutry.apk",
  "fileSize": 253023298,
  "contentType": "application/vnd.android.package-archive",
  "sha256": "4ed50bf91e8c3c9e5e6a30ea4fcc4827aa91dd26c3d1e9e1b98ca524960b9d17",
  "metadata": { "versionName": "2026.07.29" },
  "partSize": 16777216,
  "allowOverwrite": false
}

The result includes uploadId, partSize, partCount, expiresAt, and expiresIn.

2. Batch issue part URLs

POST /storage/api/multipart/urls

{
  "path": "website/releases/android/2026.07.29/xutry.apk",
  "uploadId": "opaque-starecho-upload-id",
  "partNumbers": [1, 2, 3, 4]
}

A request may contain up to 100 unique part numbers. Every item contains its partNumber, required size, method: "PUT", requiredHeaders: { "Content-Length": size }, and url. Multipart signatures always bind exact part length. A browser must upload the exact Blob.slice() and a Node client must send the returned Content-Length. Save the response ETag after every successful PUT. Browser CORS must expose ETag.

3. List uploaded parts

GET /storage/api/multipart/parts?path={path}&uploadId={uploadId}

The provider-backed result contains each partNumber, etag, and size, plus uploadedBytes. After an interruption, request URLs only for missing part numbers.

4. Complete idempotently

POST /storage/api/multipart/complete

{
  "path": "website/releases/android/2026.07.29/xutry.apk",
  "uploadId": "opaque-starecho-upload-id",
  "parts": [
    { "partNumber": 1, "etag": "..." },
    { "partNumber": 2, "etag": "..." }
  ],
  "sha256": "4ed50bf91e8c3c9e5e6a30ea4fcc4827aa91dd26c3d1e9e1b98ca524960b9d17"
}

StarEcho verifies continuous numbering, every provider ETag, equal non-final part sizes, the exact final part size, and the final object byte count. Retrying completion returns the same object result without creating a duplicate. If the provider committed the object but the response timed out, a retry recovers metadata through HEAD.

5. Abort and list sessions

POST /storage/api/multipart/abort
GET  /storage/api/multipart/uploads?includeTerminal=false

The abort body is { "path": "...", "uploadId": "..." }. Explicit abort and hourly expiry cleanup release provider parts. Aborting an already completed session returns the idempotent completed result.

Object metadata and downloads

GET /storage/api/head?path={path}
GET /storage/api/meta?path={path}

head reads the current authoritative object-store size, MIME type, ETag, and last-modified time. meta returns the confirmed platform record.

The compatible download route accepts optional response overrides:

GET /storage/api/download?path={path}&expiresIn=3600&downloadName=xuTry.apk&contentType=application%2Fvnd.android.package-archive&cacheControl=public%2C%20max-age%3D3600

The response has:

• url: temporary GET credential; normal Range requests receive provider 206 Partial Content behavior.
• headUrl: a separate HEAD credential. SigV4 binds the HTTP method, so do not change a GET URL into a HEAD request.
• size, contentType, etag, sha256, sha256Verification, and sha256VerifyStatus.

Download bytes never pass through the Cloud Function. A private R2 presigned URL must retain its R2 S3 endpoint and cannot be rewritten to a custom domain. For CDN delivery, configure a public R2 Custom Domain and caching policy for immutable release paths separately.

List and delete

GET    /storage/api/files?prefix=releases/&limit=50&page=1
DELETE /storage/api/file

The delete body is { "path": "releases/old.apk" }. Listings contain confirmed objects only. Objects associated with valid single or multipart sessions are protected from orphan cleanup.

Browser CORS

The management UI stores separate production and development origins per app, for example:

production: https://app.example.com
development: http://localhost:3000

Enter origins only, without paths. The scheme, host, and port together form the Origin, so http://localhost:3000 allows only that port rather than every localhost port. The platform baseline contains only the site origins declared by the current StarEcho deployment: a local deployment can configure its own localhost through STARECHO_CORS_ORIGIN, while production does not allow an application's localhost or arbitrary ports by default. Application owners can register the exact development origins they need here; saving Storage settings makes StarEcho synchronize R2 automatically without a platform administrator editing bucket rules.

The shared default bucket merges explicit origins from approved apps, allows GET, PUT, and HEAD, permits Content-Type, Content-Length, checksum, and signing request headers, and exposes ETag, Content-Length, Content-Range, Accept-Ranges, and checksum response headers. Write origins never use *. CORS is not authorization; a short-lived path- and method-bound presigned URL remains required.

The database migration only adds the CORS settings shape; it cannot choose trusted origins for an application. After deploying the matching server, the application owner must save the production and development origins in Storage settings. Saving re-merges and applies the default-bucket CORS policy without requiring a platform administrator to sign in to Cloudflare. A settings response with corsSync: "applied" means this request reached the provider. pending means the settings were persisted but provider synchronization failed or another instance currently owns the lock; the console keeps a warning visible and the hourly reconciler retries automatically. not_requested means the request did not change CORS or the storage type. Invalid origins, origins containing a path, and origins without an exact required port return STORAGE_INVALID_CORS_ORIGIN instead of being silently discarded.

StarEcho recalculates immediately when Storage is approved, an application is deleted, or an application switches between default and custom storage, and performs an hourly MySQL-named-lock reconciliation to prevent multi-instance lost updates and database/R2 drift. Before the real APK acceptance run, verify that provider preflight responses include both origins.

Management and operation audit

• PUT /storage/manage/{clientId}/settings configures URL lifetime, default part size, the 1–32 concurrent multipart-session limit, and production/development CORS origins.
• POST /storage/manage/{clientId}/object-limit/request adjusts either the single-PUT threshold or the true multipart object limit. A decrease is immediate; an increase requires administrator review.
• GET /storage/manage/{clientId}/object-limit/requests reads the latest request state for both limit types.
• GET /storage/manage/{clientId}/operations lists initialize, complete, abort, expire, delete, and download-credential events.

Limit adjustment uses one extensible request shape:

{
  "limitType": "multipart_object",
  "requestedBytes": 10737418240,
  "reason": "Distribute an immutable 10 GiB release artifact"
}

limitType is single_put or multipart_object. Omitting it retains the deployed-client meaning of single_put. New applications default to a 1 GiB single-PUT limit and a 5 GiB multipart object limit. The management response exposes the currently valid ranges in adjustableLimits. The single-PUT limit cannot exceed the multipart object limit. With the current maximum 64 MiB part and 10,000-part policy, the effective platform hard limit is about 625 GiB. A decrease only affects new upload sessions and never deletes existing objects. Administrator approval atomically updates both the request and the application Storage settings. The per-object limit and total application capacity quota are independent, so both must be sufficient.

retentionRules accepts up to 32 { id, prefix, expireAfterDays } rules with a 1–3650 day age. The default empty array never deletes automatically. A daily job selects expired logical paths by prefix and last object update, then uses the normal Storage deletion path so database metadata, quota, audit, and unreferenced physical data stay consistent. StarEcho does not push a shared-bucket lifecycle rule directly because that would bypass logical metadata and leave broken records. An empty prefix matches every object in the application and must be enabled carefully.

storagePrefix affects only the physical prefix of future objects. Changing or clearing it does not move, delete, or hide confirmed objects; existing logical paths continue to use their recorded complete physical keys. An empty string restores the default clientId prefix.

The management console calculates capacity pressure from usedBytes + reservedBytes, shows a warning at 80% and a critical alert at 95%, and displays the upload reservation separately. The server-side authoritative quota check still determines whether a new session is accepted.

Data-plane audit records app identity, caller mode, caller UID, logical path, size, session, and time. It never stores complete presigned URLs, provider upload IDs, or permanent credentials. Capacity, reservation, and concurrent-session values returned by getUsage() are platform-authoritative. It also returns current UTC-month issuance diagnostics as monthlyDownloadUrlIssueCount, monthlyDownloadUrlIssuedObjectBytes, and monthlyWindowStart, while explicitly returning monthlyTransferBytes: null and monthlyTransferMetering: "not_available". R2 Operations Analytics can attribute request counts and statuses by objectName, but currently exposes no authoritative response-byte total for each GET/Range request. Exact per-application monthly transfer therefore cannot be derived inside a shared bucket, and StarEcho does not misrepresent URL-issued size as transferred bytes. Enforcing exact transfer quotas requires a Worker/custom-domain logging meter in the download data path.

Cloud Function $storage

Existing calls remain valid. Capabilities, multipart, HEAD, and download options are additive:

const caps = await $storage.getCapabilities();
const usage = await $storage.getUsage();

const single = await $storage.getUploadUrl(path, size, contentType, {
  sha256,
  metadata,
  allowOverwrite: false,
  expiresIn: 3600,
});
await $storage.confirmUpload(path, single.token);

const upload = await $storage.createMultipartUpload(path, {
  fileSize: size,
  contentType,
  sha256,
  partSize: 16 * 1024 * 1024,
  allowOverwrite: false,
});
const urls = await $storage.getMultipartUploadUrls(
  path,
  upload.uploadId,
  [1, 2, 3],
);
const current = await $storage.listMultipartUploadParts(path, upload.uploadId);
const file = await $storage.completeMultipartUpload(path, upload.uploadId, {
  parts,
  sha256,
});
await $storage.abortMultipartUpload(path, upload.uploadId);

const authoritative = await $storage.head(path);
const download = await $storage.getDownloadUrl(path, {
  expiresIn: 3600,
  downloadName: "release.apk",
  cacheControl: "public, max-age=3600",
});

withBasic() and asUser() return the same method set. Structured bridge errors preserve code, retryable, and optional retryAfterSeconds fields.

Stable errors

Errors retain the compatible error and error_description fields while adding code, message, and retryable.

code	Meaning	Retry
STORAGE_FILE_TOO_LARGE	Above the app object limit	No
STORAGE_QUOTA_EXCEEDED	Quota or reservation is insufficient	After quota change
STORAGE_UPLOAD_EXPIRED	URL or session expired	Reissue/create
STORAGE_PART_MISMATCH	Number, size, or ETag mismatch	After correction
STORAGE_CHECKSUM_MISMATCH	Conflicting declared digest	Re-upload
STORAGE_FORBIDDEN	Bearer user lacks the required Storage role permission	No
STORAGE_OBJECT_NOT_FOUND	Object or session does not exist	No
STORAGE_RATE_LIMITED	Concurrent upload limit reached	Yes
STORAGE_INVALID_CORS_ORIGIN	Origin is incomplete, has a path, or lacks an exact port	After correction
STORAGE_INVALID_RETENTION_RULE	Invalid retention rule id, prefix, or age	No
STORAGE_TEMPORARILY_UNAVAILABLE	Completing or provider temporarily unavailable	Yes

VS Code and Codex tools

The extension exposes capability/usage queries, workspace-scoped file upload, object read/list/delete, temporary download credential creation, and multipart list/abort tools. starecho_upload_storage_file requires an absolute path inside the selected workspace, calculates or checks SHA-256, chooses single or multipart automatically, and keeps resume state under .starecho/storage-uploads/. starecho_create_storage_download_url accepts method: "GET" | "HEAD" to select which method-bound credential is copied and remains backward-compatible by defaulting to GET. Tool results do not reveal permanent credentials or complete presigned URLs. Platform failures preserve code, HTTP status, retryable, and retryAfterSeconds in VS Code AI and MCP errors, so callers can decide whether to stop, retry, or reissue without parsing prose.

Provider references

First run node P:\Project_Web\StarEcho\scripts_test\storage-platform-local-e2e.cjs for a fully isolated platform acceptance check. It uses temporary MySQL 8.4 and MinIO containers, the current compiled server, and the exact requirement APK. It covers migration, Basic/Bearer authorization, ReplayGuard, browser CORS request shape, legacy single upload, interrupted multipart resume, concurrent-session limits, monthly download issuance diagnostics, wrong path/length/hash rejection, idempotent completion, HEAD, Range, full SHA-256 verification, prefix transitions, expired staging, audit, and cleanup without contacting production. Random container names and a dedicated label are verified before deletion. MinIO is not the R2 bucket-CORS implementation, so the real Origin allowlist still requires post-deployment acceptance.

After deployment, run the repository-only scripts_test/storage-production-e2e.cjs from a trusted operator terminal to validate the real APK. It requires explicit API, application credential, APK path, and STARECHO_E2E_ALLOW_WRITE=I_UNDERSTAND environment variables. It never prints presigned URLs, uses an isolated logical path, and cleans up by default after success or failure. When both STARECHO_E2E_ANDROID_SERIAL and STARECHO_E2E_ALLOW_ANDROID_INSTALL=I_UNDERSTAND are set, the script installs the freshly downloaded, full-SHA-256-verified APK on that one authorized ADB device and verifies com.starecho.xutry; it never modifies an Android device without both values. The script is not registered in package.json and never enters production artifacts.

• Cloudflare R2 Limits
• R2 S3 API compatibility
• R2 presigned URLs
• R2 metrics and analytics
• R2 object lifecycles