Offline Repository Mirrors

Offline repository mirrors are pointing to APT repositories, for example from Proxmox VE, Proxmox Backup Server or Debian. After the initial setup, you can mirror all the available packages locally. They are organized by creating point-in-time snapshots of the repositories. Those snapshots can then be exported to a configured medium.

Setting Up a Mirror

First, either run the setup wizard (proxmox-offline-mirror setup), or the config mirror add command.

Tip

The quickest way to set up all relevant repositories for a Proxmox solution is to use the setup wizard. Choose the product when adding a mirror and confirm the question regarding auto-adding the Debian base repos.

Note

To be able to access and mirror a product's enterprise repository, proxmox-offline-mirror requires both an active product subscription key and a Proxmox Offline Mirror subscription (see Setup Offline Mirror Key).

For example, to manually add a mirror entry for the Debian Bookworm security repository, the following command can be used:

proxmox-offline-mirror config mirror add \
 --id debian-bookworm-security \
 --architectures amd64 \
 --architectures all \
 --repository 'deb http://deb.debian.org/debian-security bookworm-security main contrib non-free' \
 --key-path /etc/apt/trusted.gpg.d/debian-archive-bookworm-security-automatic.gpg \
 --sync true \
 --verify-mode size \
 --base-dir /path/to/mirror/base-dir

Note

The base-dir directory can be shared by mirrors for repositories that have common contents to avoid storing files more than once. For example, having a single base directory for all mirrors referencing Proxmox repositories is recommended.

Note

The all architecture is meant for architecture independent packages, not for all possible architectures. It is almost always sensible to add it in addition to the host-specific architecture.

Syncing a Mirror

To create the first (and subsequent) snapshots, the following command can be used:

proxmox-offline-mirror mirror snapshot create debian-bookworm-security

Note

Depending on the parameters used and the size of the original repository, creating a snapshot can take both time and require significant disk space. This is especially true for the initial snapshot, as subsequent ones will re-use unchanged package files and indices.

The --max-downloads option sets the number of concurrent binary package downloads. The default is 1, with a maximum of 4 to limit load on the source repository. Increasing it can speed up snapshot creation, especially the first run. Source packages are always downloaded sequentially. For example, to allow four concurrent binary package downloads:

proxmox-offline-mirror mirror snapshot create debian-bookworm-security --max-downloads 4

Only one snapshot creation per mirror can run at a time. A second creation waits up to five seconds for the first to finish. If the first is still running, the second fails. Snapshots of different mirrors can be created in parallel, even when they share a base directory.

The same lock protects other operations that change a mirror's snapshots or symlinks: removing, naming, unnaming, and pruning snapshots; removing the mirror; and changing its base directory or latest-symlink settings. These operations also wait up to five seconds for the lock, then fail if the mirror is still busy. Since snapshot creation can take hours, schedule operations for the same mirror so they do not overlap.

Verify Mode

The --verify-mode option controls how snapshot creation checks files reused from the pool. Newly downloaded files are always checked against the checksums provided by the repository. Three modes are available:

none

Only check that the file exists. This is the fastest mode, but it does not detect damaged files.

size

Also check that the file size matches the repository index. This detects truncated files at negligible cost and is the recommended mode.

full

Read every reused file and verify its checksums. This is the most thorough mode, but it can read tens of gigabytes when creating a snapshot of a large repository, even if nothing has changed.

A file that fails verification is downloaded again to repair its existing pool entry. Because snapshots use hard links to pool entries, this also repairs the file in older snapshots that reference the same entry.

The previous boolean option, verify, is still accepted as an alias: true means full and false means none, so existing configurations keep their behavior. On the command line, the alias requires an explicit value. Use --verify true or --verify-mode full instead of a bare --verify. To switch an existing mirror to size mode:

proxmox-offline-mirror config mirror update --id debian-bookworm-security \
  --verify-mode size

For on-demand full integrity checks without changing the configured mode, use the mirror verify command (see below).

Request Timeouts

By default, each repository request has a total time limit of 15 minutes, plus the transfer time at 64 KiB/s when the expected file size is known. This bounds stalled requests while allowing large packages to download over slow connections. The request-timeout option overrides this with a fixed limit in seconds, up to 604800 (one week). Set it to 0 to disable the limit:

proxmox-offline-mirror config mirror update --id debian-bookworm-security --request-timeout 0

To return to the default limit:

proxmox-offline-mirror config mirror update --id debian-bookworm-security \
  --delete request-timeout

Named Snapshots

Snapshots can be given human-readable names, which are symlinks next to the timestamp-based directories. Unlike the last symlink, a name continues to point to the same snapshot. Commands that take a snapshot argument still require its timestamp, not its assigned name.

To name a snapshot at creation time:

proxmox-offline-mirror mirror snapshot create debian-bookworm-security --name stable

To name an existing snapshot:

proxmox-offline-mirror mirror snapshot name debian-bookworm-security 2025-03-05T14:30:45Z stable

To remove a name without removing the underlying snapshot:

proxmox-offline-mirror mirror snapshot unname debian-bookworm-security stable

The mirror snapshot list command shows names alongside the timestamps they point to.

Removing a snapshot also removes its names.

Like the last symlink, names exist only on the mirroring host: syncing to an offline medium transfers the snapshots themselves.

Reducing Mirror Scope

There are different mechanisms for reducing a mirror's scope (and correspondingly, the amount of traffic and disk space required to keep it synced):

  • architecture filters

  • components (as part of the repository specification)

  • package name and section filters

By default, only packages for the architectures all (see note above) and amd64 are mirrored.

Optionally, it's possible to set up filters for downloaded binary or source packages via the --skip-packages and --skip-sections options. The package filters support globbing, for example linux-image-* will skip all packages with a name starting with linux-image-. The section filters match the full value, or the value prefixed with the package's component (for example, games will match both the section games, as well as non-free/games in a packages index of the non-free component).

Some examples for packages and section filters:

  • --skip-packages 'linux-image-*' - filter Debian linux kernel image packages

  • --skip-sections 'games' - filter sections containing game packages

  • --skip-sections 'debug' - filter sections containing debug information

If you want to skip all debugging related packages (which might not always be in the debug section mentioned in the example above), the following set of filters should work:

--skip-packages '*-dbgsym' \
--skip-packages '*-dbg' \
--skip-packages '*-debug' \
--skip-sections debug

Please refer to https://packages.debian.org/bookworm/ for a list of Debian archive sections and their contents.

Note

Proxmox releases based on Debian Bookworm and Trixie ship their debug symbol (-dbgsym) packages in a dedicated repository, so the filters above do not apply to them. To have the debug symbols offline, mirror that repository like any other Proxmox repository: component <product>-dbgsym at https://dbgsym.proxmox.com/debian/<product>, for example pve-dbgsym at https://dbgsym.proxmox.com/debian/pve.

Pruning Snapshots

Over time, many snapshots accumulate. The prune command removes old snapshots based on configurable retention rules, similar to Proxmox Backup Server:

proxmox-offline-mirror mirror snapshot prune debian-bookworm-security \
  --keep-last 3 --keep-daily 7 --keep-weekly 4 --keep-monthly 6

The available retention options are:

--keep-last N

Keep the N most recent snapshots regardless of age.

--keep-hourly N

Keep snapshots for the last N different hours. If there is more than one snapshot for a single hour, only the latest is kept.

--keep-daily N

Keep snapshots for the last N different days. If there is more than one snapshot for a single day, only the latest is kept.

--keep-weekly N

Keep snapshots for the last N different ISO weeks. If there is more than one snapshot for a single week, only the latest is kept.

--keep-monthly N

Keep snapshots for the last N different months. If there is more than one snapshot for a single month, only the latest is kept.

--keep-yearly N

Keep snapshots for the last N different years. If there is more than one snapshot for a single year, only the latest is kept.

At least one --keep-* option must be set. Rules are applied in the order listed above. Each time-based rule skips periods in which an earlier rule already kept a snapshot. For example, --keep-last 1 --keep-daily 1 keeps the newest snapshot and the latest snapshot from the most recent earlier day that contains one.

Only periods containing snapshots count toward the limits. If the only snapshot is three years old, --keep-hourly 1 still keeps it. Period boundaries use UTC, like the snapshot timestamps, and may therefore differ from local calendar boundaries.

Named snapshots are protected by default and do not count toward the retention limits. Use --ignore-names to apply the retention rules to them as well. The latest-snapshot symlink's target is always protected, even if newer snapshots were created with --latest-symlink false. These protections and the combination of rules can retain more than one snapshot from a period.

Use --dry-run to preview what would be kept or removed:

proxmox-offline-mirror mirror snapshot prune debian-bookworm-security \
  --keep-daily 7 --dry-run

Use --gc to automatically run garbage collection after pruning:

proxmox-offline-mirror mirror snapshot prune debian-bookworm-security \
  --keep-last 5 --gc

To prune all configured mirrors at once:

proxmox-offline-mirror mirror snapshot prune-all --keep-daily 7 --keep-weekly 4

Pruning uses the per-mirror lock described above and fails if it cannot acquire the lock within five seconds. prune-all skips a busy mirror, continues with the others, and exits with an error. A dry run does not acquire the lock.

Comparing Snapshots

The mirror snapshot diff command shows what changed between two snapshots. By default it lists the stored files that were added, removed, or changed:

proxmox-offline-mirror mirror snapshot diff debian-bookworm-security <older> <newer>

Replace <older> and <newer> with the snapshot timestamps shown by mirror snapshot list. Pass --packages to compare the package indices instead. This lists binary and source packages that were added or removed, or whose versions changed:

proxmox-offline-mirror mirror snapshot diff debian-bookworm-security <older> <newer> --packages

Binary packages are reported as <name>:<architecture>; a package that appears with a different version in the newer snapshot is reported as changed, showing both versions.

Both diff variants support --output-format json for use in scripts. The same option is available for mirror snapshot list, remove, prune, prune-all, mirror gc, and mirror verify. See JSON Command Output for the output formats.

Space Management

Removing snapshots, with mirror snapshot remove or by pruning without --gc, does not free disk space by itself. Run mirror gc to delete the pool files that no snapshot uses anymore.

A failed or interrupted snapshot creation can leave a temporary directory named <snapshot>.tmp. Its hard links protect files already added to the pool from garbage collection, allowing a retry to reuse them. The next successful snapshot creation for that mirror removes these temporary directories. Medium synchronization does not copy them.

Package downloads are staged in a hidden .downloads directory next to the snapshots before being added to the pool. Unlike files already stored in the pool, partial downloads cannot be reused. The next snapshot creation for the mirror removes partial downloads left by a crashed run.

Snapshot removal first renames the directory to .<snapshot>.removing before deleting its files. This prevents a partially deleted snapshot from appearing in listings or being copied to a medium. If deletion fails, for example because of a permission error, the command reports the cleanup error and exits with a nonzero status. Garbage collection retries deletion of the remaining files.

Verifying Pool Integrity

The mirror verify command checks pool files for damage independently of the verify-mode used during snapshot creation. It reports damage without repairing files and exits with a nonzero status if verification fails. Creating a snapshot in full verify mode repairs damaged files that are still needed by that snapshot and available from the repository.

By default, the command reads each pool file and verifies its checksums. This can read tens of gigabytes for a large mirror, so consider scheduling it. For a faster check, use --mode size. This detects files truncated to zero bytes, but not other size changes, because the pool does not store expected sizes. Files whose checksums identify empty content are accepted in either mode. For example:

proxmox-offline-mirror mirror verify debian-bookworm-security
proxmox-offline-mirror mirror verify debian-bookworm-security --mode size

Without an id argument, all configured mirrors are verified (pools shared by multiple mirrors are only verified once):

proxmox-offline-mirror mirror verify

Note

mirror verify and mirror gc act on all mirrors if no id is given. mirror snapshot create and mirror snapshot prune require an id instead, and have separate create-all and prune-all variants.

Global Options

--config <path>

Path to the mirroring config file. Can be placed anywhere on the command line (before or after the subcommand). If omitted, the PROXMOX_OFFLINE_MIRROR_CONFIG environment variable is used, falling back to /etc/proxmox-offline-mirror.cfg.

Environment Variables

PROXMOX_OFFLINE_MIRROR_CONFIG

Path to the mirroring config file, if --config is not given. Defaults to /etc/proxmox-offline-mirror.cfg.

ALL_PROXY

When set, proxmox-offline-mirror uses the specified HTTP proxy for all HTTP(s) connections. Currently only HTTP proxies are supported. Valid proxy configurations have the following format: [http://][user:password@]<host>[:port]. Default port is 1080, if not otherwise specified.

Note

The proxy server must allow HTTP CONNECT for all ports that are used to connect to mirrors (e.g. port 80 for HTTP mirrors). For Squid, the appropriate configuration parameter is http_access allow CONNECT <acl> (http://www.squid-cache.org/Doc/config/http_access/). By default, Squid only allows HTTP CONNECT for port 443.