Lifecycle and deduplication

Lifetime (TTL)Copy Anchor Link

Each image can have a lifetime after which it is deleted automatically. The lifetime is resolved across three levels; the first one that is defined wins:

• An explicit 0 or never on an upload overrides both the user and the global default, so that image never expires.
• S3 uploads cannot set a lifetime per request and use the user default, then the global default.
• The expiry time is computed and stored at upload. Changing the user or global default later does not affect existing images.
• To make an image temporary, give it a short lifetime; there is no separate temporary folder or temporary upload type.
• Per-upload lifetimes, per-user defaults and the global default are all capped at 876000 hours (100 years); larger values are rejected. Every other runtime setting that converts to a duration (sweep intervals, the delete grace period, retention periods, timeouts and so on) is capped at 100 years in its own unit as well.

The lifetime of an existing image is changed in its details (by the image's owner or an administrator):

• Change expiry → enter a number of days and click Extend: the expiry becomes N days from now, and that duration becomes the image's new lifetime (renewals use it from then on).
• Make permanent: removes the expiry.

Renewal on accessCopy Anchor Link

Renew on access (ttl_renew_on_access) is on out of the box. With it on, every public access to an image that has a lifetime pushes its expiry back to "the time the access was recorded + its lifetime". In effect the lifetime turns into "expires after this long without a single access": images in use stay, images nobody looks at get cleaned up. With it off, the expiry time stays fixed.

• Accesses are collected in memory and written to the database in batches every activity_flush_seconds (60 seconds out of the box); renewal happens at that write. An image that has already expired at the moment of the write is not renewed: expiry is final.
• Only public accesses through /i/ and /o/ count; console thumbnails and previews and S3 GetObject do not. See Browsing in the console is not counted.
• Images without a lifetime are unaffected.

Expiry sweepsCopy Anchor Link

The cleanup sweep runs once when the process starts and then every cleanup_sweep_seconds (60 seconds out of the box). Each round, in order:

1. deletes expired images (at most 500 per round; any backlog continues next round);
2. physically deletes files whose grace period has passed (see the next section);
3. tags cold images for cleanup (see Cold images);
4. reclaims upload idempotency keys past their retention period;
5. reclaims old audit log entries if a retention period is set (audit_retention_days; 0, the default, keeps them forever).

As soon as an image expires, its public links return 404 and it can no longer be renewed by access; the cleanup sweep reclaims the record and the file afterwards. Until then it shows as Expired on the Images page and can still be deleted by hand. Images that expire while the process is down are deleted in the sweeps after it restarts.

The delete flowCopy Anchor Link

Every deletion takes the same path: expiry, deleting in the console (one image, or up to 100 at a time), deleting from the cleanup review, and S3 DeleteObject.

Reference counts: one physical file can be shared by several image records (the result of deduplication merges). Deleting one record does not affect the others; the physical file enters the delete flow only once the last reference is gone. References in the image details shows the current count.

The delete confirmations in the console tell you roughly when the physical file will go, based on the current grace period. A user can only be deleted after every image they own has been deleted.

Cold imagesCopy Anchor Link

Images nobody has accessed for a long time are tagged for cleanup and wait on the Cleanup page for an administrator's decision. They are never deleted automatically. Deletion cannot be undone, so the machine only finds candidates and a person decides; merging near-duplicates follows the same principle.

• The threshold is Cold image threshold (cold_cleanup_days), 90 days out of the box; 0 stops tagging.
• Tagging: on each cleanup sweep, images whose last public access (or upload time, if never accessed) is older than the threshold and which are not exempt get tagged.
• The tag is removed when:
  • the image is publicly accessed (at the next batch write of access counts);
  • an administrator decides to Keep it;
  • an administrator removes its cleanup exemption (the next sweep judges it again);
  • the image is deleted.
• Raising the threshold or setting it to 0 does not remove existing tags.
• Cold tagging and lifetimes are independent: images with a lifetime can be tagged too, and exempt images still expire.

ReviewingCopy Anchor Link

The Cleanup page (administrators only) lists images from the longest-unaccessed down, and works on single images or selections:

• Keep: sets the cleanup exemption on the image, so it is never tagged as cold again; its lifetime and renewal are unaffected. The exemption can be removed in the image details.
• Delete: runs the delete flow above.

Only administrators can set cleanup exemptions, as part of the same authority as cold-image decisions, so an image's owner cannot exempt their own images to dodge cleanup. The Cleanup exempt switch in the image details can be turned on before an image ever goes cold; regular users see the state but cannot change it. The Exempt only filter on the Images page lists every exempt image.

DeduplicationCopy Anchor Link

Deduplication never runs on the upload path, so it never slows uploads down. The background loop runs once at startup and then every dedup_scan_seconds (300 seconds out of the box); each round merges exact duplicates first, then scans for near-duplicates.

Exact duplicates: merged automaticallyCopy Anchor Link

Physical files whose stored bytes are identical (same SHA-256) can be merged with no risk at all, so this happens automatically:

• the oldest copy is kept, the image records of the others are pointed at it, and their reference counts are added to it;
• the extra files are marked for deletion and removed after the delete grace period;
• files of different users are merged as well. Every image record keeps its own short ID, owner, lifetime and access count, and every link keeps working;
• quotas count every reference in full, so merging does not reduce anyone's usage.

Near-duplicates: reviewed by handCopy Anchor Link

Every physical file gets a 64-bit perceptual hash (pHash) at upload. When two files' hashes differ in no more bits (Hamming distance) than Similarity threshold (dedup_phash_max_distance, 10 out of the box), the pair becomes a candidate on the Duplicates page.

• A higher threshold is looser and produces more false positives; 0 turns similarity scanning off. The console notes that distances of 0–2 are near identical.
• Even a distance of 0 only makes a candidate and is never merged automatically: equal perceptual hashes do not mean equal bytes.
• Scanning is incremental: each round compares only new files against the existing ones, computing distances one by one in memory; at self-hosted scale no index structure is needed. After a restart everything is rescanned once, and dismissed pairs do not come back.

Deciding, and which image to keepCopy Anchor Link

The Duplicates page (administrators only) shows each candidate pair side by side as Left and Right, with format, dimensions, size, reference count and distance, closest pairs first:

• Keep this one → confirm Merge: every reference of the other image now points to the kept one, the merged file is removed after the grace period, and any other candidates involving it are dropped. This cannot be undone.
• Dismiss: the pair is never shown again; several pairs can be dismissed at once.

After a merge, every link of the merged image (including other users' references) serves the kept image's content, which may differ in size, format and even picture. So keep the higher-resolution or better-format image, and choose Dismiss when the two are actually different pictures. Browsers and CDNs that already cached the old content keep showing it until their cache expires.

Merges and dismissals are recorded in the audit log (actions "Merge duplicates" and "Dismiss duplicates").

Related settingsCopy Anchor Link

All of them are changed under Settings → Runtime settings and take effect on save; see Configuration.