- Rust 95.7%
- HTML 1.9%
- CSS 1.4%
- Shell 0.9%
Till Wegmueller
110ee513bc
rebuild_catalog_batched accumulated three Vec<(Fmri, Option<Vec<String>>,
String)> for base/dependency/summary entries, then materialised three
CatalogPart trees and serialised each in one go. On a 200k-package
publisher these accumulators plus the trees overran available RAM and
OOMed midway through the rebuild, after the per-manifest pest fix.
Add StreamingCatalogPartWriter, which emits `{"<publisher>":{<stem>:[
versions...]...}` directly to a temp file as the caller produces stems,
mirroring every byte into a SHA-1 hasher so the part signature is
computed without buffering. The trailing `,"_SIGNATURE":...}` is
appended after finalisation to keep the on-disk byte sequence
identical to write_catalog_part's output (verified by a new round-trip
test).
Refactor rebuild_catalog_batched to walk the FMRI-sorted package list,
buffer one stem's PackageVersionEntry instances at a time, and flush
them to the three streaming writers when the stem changes. Memory now
scales with versions-per-stem, not packages-per-publisher.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|---|---|---|
| .github | ||
| .idea | ||
| .junie | ||
| .solstice | ||
| .vscode | ||
| doc | ||
| docker | ||
| docs/ai | ||
| libips | ||
| pkg6 | ||
| pkg6depotd | ||
| pkg6recv | ||
| pkg6repo | ||
| pkgtree | ||
| ports | ||
| specfile | ||
| testing/manual | ||
| userland | ||
| xtask | ||
| .dockerignore | ||
| .gitignore | ||
| Cargo.lock | ||
| Cargo.toml | ||
| CLAUDE.md | ||
| docker-compose.yml | ||
| Dockerfile | ||
| IMPLEMENTATION_STATUS.md | ||
| LICENSE | ||
| Makefile | ||
| pkg6depotd.kdl | ||
| ports.spec | ||
| README.md | ||
| rebuild_test_repo.sh | ||
| run_local_import_test.sh | ||
| run_openindiana_image_import.sh | ||
| run_sample_install.sh | ||
| test_repo_operations.sh | ||
| Vagrantfile | ||
Image Packaging System for illumos
libips
The libips crate contains all modules and functions one needs to implement an Image Packaging System based utility.
Be that a server, client or other utilities.
This project is intended to gradually replace the current python based implementation of IPS. Most things are documented in the docs directory but some things have been added over the years which has not been properly documented. Help is welcome but be advised, this is mainly intended for use within the illumos community and it's distributions. Big changes which are not in the current IPS will need to be carefully coordinated to not break the current IPS.
Development and Release
Releasing
This project uses cargo-release for versioning and tagging.
- Ensure you have
cargo-releaseinstalled:cargo install cargo-release - Prepare the release (dry-run):
cargo release [level] --dry-run(e.g.,cargo release patch --dry-run) - Execute the release:
cargo release [level] --execute - Push the changes and tags:
git push --follow-tags
Pushing a tag starting with v (e.g., v0.5.1) will trigger the GitHub Actions release pipeline, which builds artifacts for Illumos (OpenIndiana) and Linux, and creates a GitHub Release.
