Brioche Blog

Brioche v0.1.8: Preparing for the future

Kyle Lacy — Sat, 30 May 2026 07:19:57 GMT

Brioche v0.1.8 is a pretty small release overall, with a few quality-of-life improvements, plus some enhancements to prepare for some future changes.

To get right into it, you can run brioche self-update to upgrade, or check the installation docs if you don't have Brioche installed yet. You can also take a look at the release notes, if that's more your speed.

Oh, and also a PSA about upcoming brioche-packages stuff that will impact older versions of Brioche!!

(Minor breakage) TypeScript v6 and linting changes

This release jumps us from TypeScript v5.9.3-- introduced in the last release-- up to TypeScript v6.0. I don't think most projects will see much impact one way or the other from this upgrade... but this is laying the groundwork to be able to use TypeScript v7 in the future!

Additionally, Brioche now bundles a newer version of ESLint and typescript-eslint for linting, and enables some new lints by default. These linting changes could cause some breakages, especially in CI-- e.g. if you're using either the brioche check subcommand or the --check flag. I think most projects will get by without any changes required.

The new lint rules we're enabling now are:

For more context, check out @jaudiger's PRs that introduced these changes: #465 and #467

Show source paths during builds

When jobs run building a package-- say, when a process recipe or a download is running-- you'll now see the source path in the TUI view, which gives you a better sense of which step in your build is currently running.

Small heads up: some of the source paths that get shown aren't always the most helpful. We thread through metadata to hide utility types wherever possible, but we'll probably need to make some updates to the std package to make this more useful!

This improvement was introduced by @jaudiger in #466

Linux sandbox improvements

This release includes a few more minor improvements to the Linux sandbox environment:

The sandbox user is now part of a group (via a new /etc/groups file in the sandbox).
Sandboxed programs now see a fixed hostname of localhost, rather than the host system's hostname. To accommodate this, the sandbox now runs within an isolated UTS namespace.
The sandbox now includes the files /etc/services and /etc/protocols. This helps with glibc compatibility (e.g. the functions getservbyname and getprotobyname).

These improvements were added in #459 by @jaudiger.

Reader support for new cache archive tag

Since the cache overhaul in Brioche v0.1.5, we've used a custom archive format to store artifacts in the cache. This format has gone unchanged since it was first introduced... until now that is!

Brioche v0.1.8 now supports reading archives with a new R tag. This tag references another entry in the artifact by path, allowing for efficiently deduplicating complex directory trees.

@jaudiger added this change in #460, in order to unblock the (longstanding!) abseil_cpp PR in the brioche-packages repo.

Note that Brioche v0.1.8 doesn't support writing this new archive tag yet! This is part of a 3-step plan:

Release support for reading the new archive tag in a stable release (you are here!)
Add support for writing the new archive tag in a nightly release. This will then be used by brioche-packages-- unblocking packages like abseil_cpp.
Release support for writing the new archive tag in a future stable release.

PSA: Heads up that, once support for writing the new archive format lands, using the package cache for builds from brioche-packages will require Brioche v0.1.8 or later! Users of Brioche v0.1.7 should upgrade to v0.1.8 soon! Otherwise pulling new builds from the Brioche cache will start to fail!

And... those are the highlights! There's some more goodies to be found in the release notes too.

I'm glad to be getting into a better flow of "smaller" releases, with a few nice quality-of-life improvements and bugfixes. That said, there's some bigger stuff cooking too: I've been tinkering with a larger-scale refactor, which should hopefully start to open more doors towards the larger ideas I have for Brioche! ...but, that may still be quite a ways off before that work is ready to see the light of day, so stay tuned for more news on that front over time!

Brioche v0.1.7: Simplified CLI

Kyle Lacy — Sun, 29 Mar 2026 22:25:48 GMT

It's finally spring, and it's finally time for the first Brioche release of 2026!

When you're ready to jump in, run brioche self-update to upgrade, or check the installation docs if you're getting set up to the first time. If you want the nitty-gritty details, also feel free to check the release notes.

Simplified CLI syntax

In Brioche v0.1.6 and prior, the CLI used the flags -p, -r, and -e for specifying local projects, registry projects, and exports, respectively. With Brioche v0.1.7, we have a new, sleeker syntax:

brioche check ./project # Previously: brioche check -p project
brioche build ./packages/llvm # Previously: brioche build -p package/llvm
brioche run curl -- https://example.com # Previously: brioche run -r curl -- https://example.com

Building a local project is now specified by a relative path or absolute path, while remote registry projects are just specified with their name directly.

Note that there's a trade-off here: relative paths always need to start with ./ or ../. That is, brioche build ./nodejs builds the local nodejs directory, where brioche build nodejs builds the nodejs project from the registry!

As for exports, these are specified with ^:

brioche build ./packages/llvm^test # Previously: brioche build -p packages/llvm -e test
brioche build ^frontend # Previously: brioche build -e frontend

The thing before the ^ is the project (either a local path or registry project), and the thing after is the export name. Defaults to the current dir if no project is specified. You can even specify multiple exports separated by commas, e.g. brioche build ^frontend,backend

This change applies across the board to the CLI subcommands that take projects and/or exports:

brioche build
brioche run
brioche install
brioche check
brioche publish
brioche live-update
brioche fmt
- bonus: brioche fmt can also now take .bri files by path, instead of reformatting a whole project!

The new version a lot clearer in my opinion. It also means a shell command like brioche build ./projects/* will work fine now, which is really convenient for complex CI pipelines!

Big props to @jaudiger for implementing this across a series of PRs, and thanks all around for all the discussion about this change in brioche-dev/brioche#320!

Oh, and the existing -p, -r, and -e are considered deprecated, but will continue to work for the forseeable future. They might go away in Brioche v0.2.0 or something, idk ¯\_(ツ)_/¯

Configurable split read/write cache

Do you ever find yourself in the situation where you use an S3-compatible object store for your content-addressed storage system, but you want to minimize reads to reduce your monthly bill, so you deploy your homegrown caching server for it, but you still want writes to go directly to S3?

...no? Well, that's what happened to me! The CI pipeline for brioche-packages uses GitHub Actions with ephemeral runners, which means each change needs to pull a fair amount of stuff from the Brioche cache. Nearly every S3 provider bills for egress bandwidth or per API call or both, which means each CI run costs some money (though not much!) from the cache usage.

That led me down a little detour to write a little caching server called server3-- which is now what powers https://cache.brioche.dev. For a number of reasons, the cache is read-only: it can pull objects that already exist in S3 but can't write new objects. So ideally, we want the CI pipeline to read from server3 but to write to S3 directly.

Enter: the new cache.write_url config option! This can be used to specify a cache URL just for writing. In the case of our CI pipeline, cache.url is set to our server3 cache, while cache.write_url is set to our upstream S3 provider.

We've seen pretty high cache-hit rates (~90% - 95%) for brioche-packages builds, even with a pretty modest cache size (10 GB) and even across lots of package updates:

Anyway, check out the "Cache configuration" docs for details, or see #422 where this option was added.

Sandbox: Resolve `localhost`

When building recipes in the Linux sandbox, localhost now resolves to the loopback IP addresses 127.0.0.1 (IPv4) and ::1 (IPv6), as it should. Yay!

This was added by @jaudiger in #445, which came up in response to an update for the filebrowser package. For the update, File Browser switched from Vite 7 to Vite 8, which seems to do a DNS lookup for localhost during the build. This seems totally reasonable to do during a build, even when networking is disabled, so it works now.

...honestly, I hadn't though about localhost not resolving in the sandbox, and I'm more surprised we've gone this long without noticing issues before!

Experimental: Skipping cached artifacts

@jaudiger added a nifty debugging tool in #401: $BRIOCHE_SKIP_CACHE_ARTIFACTS. This env var is set to a comma-separated list of artifact hashes, and each is skipped from the upstream cache-- allowing you to build an artifact locally instead. This allows for investigating potential build issues where the cache is "poisoned" with bad artifacts (e.g. due to a bug in Brioche, memory corruption/bit flips, or any other reason). Or maybe you just want to benchmark how long it takes to build LLVM yourself.

This addresses a long-standing need in #69 (nice), although with a pretty rough interface. Longer term, I still want to see something that's more user-accessible that ideally doesn't require working out which specific hash(es) to ignore. But, the need for this came up, and perfect is the enemy of good, as they say. I'm all for adding weird/fun debugging tools as necessary, and so I thought it made sense to get this in as-is to buy us time for a "proper" solution for #69 (nice).

TypeScript v5.9.3 + ESLint upgrade

This release jumps us up from TypeScript v5.3.3-- introduced in the initial Brioche release-- all the way to v5.9.3! We actually haven't had any type-checking regressions or breakages from this (that we've seen anyway). So overall, this upgrade was pretty pedestrian (by @jaudiger in #388), which is good!

More importantly, this gets us closer to TypeScript v6.0, which in turn will be a stepping stone for us to use TypeScript's Go rewrite in v7! I've been keeping a watchful eye on TypeScript v7's progress, although at the time of writing, it seems like the compiler API is still not far enough along for us to integrate with: we're still waiting with bated breath for news on that front.

As for TypeScript v6, it was just released last week, and I didn't feel like it was worth holding up the Brioche release to get the next upgrade in. But anticipate TypeScript v6 in Brioche in the near future!

...oh, and we had a bunch of other package upgrades too, including ESLint. We're still using a pretty small list of lint rules, but keeping up on ESLint updates keeps the door open for newer/better/improved Brioche lints over time.

Anyway, that's the release! This is our second release with our new release process... which if you think about it, really makes this the first release with the new release process. (Okay, but really: this is the first release where updates utilize the new release process!)

I've also been pretty quiet on the blog lately. Partially that's because 2026 has been... weird for me so far, and I've had other Life Things come up that have taken away my time from Brioche in general. Partially it's because, for 2025, I made it a goal of mine to get out a Project Update post every month before the end of the month-- which I did and I'm glad about! But on reflection, I think I'd enjoy doing less frequent, more focused blog posts in general. I'm still really proud of the "Portable, dynamically linked packages on Linux" article, and I've got a small queue of other articles to work on writing soon(-ish).

I think there will be more Project Updates, but probably less often and "when it makes sense", instead of on a specific schedule.

Brioche Project Update - December 2025

Kyle Lacy — Wed, 31 Dec 2025 10:45:08 GMT

import PackageList from "./project-update-2025-12/PackageList.astro";

It's the final project update of the year! With the holidays and... life stuff, this month was a little slower for me for Brioche work. But there's still a lot of exciting stuff to share, and we just crossed another major package milestone!!! 🎉

Highlights from the year

With the year coming to a close, I wanted to start with a quick look back at some of the major milestones we reached this year:

Packages: from 50 packages at the start of January to over 500 packages 🎉
Brioche: 3 new releases this year:
- Brioche v0.1.4 (January): prettier output, packed Brioche builds
- Brioche v0.1.5 (April): new cache, debug shell for failed builds
- Brioche v0.1.6 (November): aarch64 support, new release process
Made possible with the help of 10 contributors in the past year!
- Members of the brioche-dev org: @jaudiger, @asheliahut, and @kylewlacy (me!)
- @chad-russell, @jokeyrhyme, @paricbat, @kaathewisegit, @running-grass, @nz366, and @easrng
- ...plus everyone who participated with issues, discussions, and ideas (and any other contributors I missed!)

Status report

TypeScript 5.9.3 upgrade

@jaudiger opened #388 to upgrade the TypeScript version embedded in Brioche from v5.3.3 to v5.9.3! We didn't spot any breakages across the brioche-packages repo during this upgrade, so this should be a pretty safe update that should give us a few new type-level tools and checks for free.

This feels like the right time to upgrade, since this is the last TypeScript version before v6.0, which is then the final release before the native TypeScript port in v7.0! There was a blog post earlier this month about TypeScript's progress towards the native port in v7.0. I'm hoping that we'll be able to integrate TypeScript 7 into Brioche not long after it's ready to go, so we can take advantage of the major speed boost in type-checking!

`brioche fmt` improvements

The brioche fmt got some nice quality-of-life improvements from @jaudiger in #386.

The -p flag is no longer necessary (since you'd only ever want to format a local project, never one remotely!), and additionally you can now format individual .bri files instead of a full project directory!

These commands are now all (roughly) equivalent:

brioche fmt packages/std
brioche fmt -p packages/std
brioche fmt packages/std/**/*.bri

More major groundwork packages

@jaudiger has continued a streak of gettings lots of new exciting packages done, and that includes a ton more work towards Linux GUI applications! I'll share his summary of the changes from just the past week:

I did a lot of things during the last week, I was mainly focused on:

packaging the core libraries to later build bigger ones such as Mesa, FFMPEG, Wayland, etc. We do need a lot of core things right now, but I'm moving in the right direction

adding more common CLI tools to reach a broader audience

There's a lot of X11/Wayland packages in the modern Linux desktop stack, but we're starting to get a lot of packages from that ecosystem wired up, so it's really exciting progress!

(and personally, ffmpeg is one I'm really excited about-- although that one is kind of its own little black hole of package management challenges!)

Sad personal news

Midway through the month, we put our 15-year-old cat to sleep. She was amazing and lived a long, loving, happy life, but it was still sudden and it's been real hard.

So overall, this month has been pretty slow-going for me.

If you've got a cat or other fuzzy creature, give them cuddles and a big pet for me.

Continued infrastructure work

Well, with the time I did spend on Brioche this month, I mostly continued on with the infrastructure work I discussed last month. I made some good steps forward, and the foundation is pretty sturdy now! ...although a lot of it is "in the weeds" Kubernetes stuff (Longhorn for storage, Grafana for monitoring, etc.), so nothing really exciting to share yet.

Minor caching tweaks

I opened and merged #379, which added some very minor tweaks to the cache:

Retry PUT requests
Allow customizing cache timeouts through config / env vars

This change is pretty unassuming, but came up after a hair-pulling debugging session of failures in the brioche-packages repo, where builds were failing to upload to the cache sporadically. I think the root cause was related to some weird DNS quirks from some of my recent infrastructure work. But after getting this change in, then bumping up one of the timeouts in the brioche-packages workflow in brioche-dev/brioche-packages#2073, things have been running smoothly.

Housekeeping

New packages

Since the last update, there were 159 new packages. During this month, we blew past another major milestone:

<div class="font-bold text-xl">🎉 Brioche now has over 500 packages! 🎉</div>

(At the time of writing, we're sitting at 514, with about 2 dozen more in the build queue!)

As is tradition, here are all the new packages this month:

Brioche core updates

build: update dependencies in runtime NPM project (#375)
Tweak cache timeouts and retries (#379)
Update package-lock.json (#377)
build: update transient dependencies to latest versions (#378)
chore: resolve new warnings from Clippy 1.92 (#387)
feat: update typescript to its latest version (5.3.3) (#388)
Update Dependabot config to ignore major versions of Bincode (#391)
chore: revamp the command format, and add the ability to format files (#386)
Fix typo in dependabot.yml (#393)

Brioche Project Update - November 2025

Kyle Lacy — Sun, 30 Nov 2025 20:55:20 GMT

This month, there's been a new release, lots of infrastructure work, and a bunch more packages

Status report

Brioche v0.1.6 released

Shortly after the last status update, we released Brioche v0.1.6, so if you missed the news go check it out! The headline feature is official support for aarch64 Linux as a platform, but there are lots of other goodies packed in the release too.

Internally, this was the first time we got to use our new release process to cut the release (which I also talked about in last month's Project Update). Mostly, this was exciting because it didn't blow up!

`setup-brioche` action overhaul

The new release came from our new release process. Our new release process included a new installer script. The new installer script also enabled us to massively simplify our setup-brioche GitHub Action.

The initial version of the setup-brioche action bundled its own install script, which was nearly identical to the (old) install script. But, it started to grow out as we added features. That also meant any changes to the main install script needed to be ported over to the action too.

Now, the new install script (brioche-installer) includes all the custom functionality that was originally just added to the setup-brioche script (opt-in, of course). So now, setup-brioche can just pull the official installer and run it!

But, from a supply-chain security perspective, fetching and executing a live-updating shell script is... uh... somewhat risky! To help matters, the installer script itself is code-signed, and we explicitly check the signature before running the installer. I feel this helps us maintain a strong security posture for the action, along with massively simplifying the maintenance.

New aarch64 runner

For brioche-package builds, we use our own self-hosted runners. Specifically, we have a Minisforum mini PC as an x86-64 runner, and an M1 Mac mini as an aarch64 runner (running Asahi Linux).

For some time now, the aarch64 builds have been a bottleneck in our package builds (not that the Mac mini is a slouch! But it's not quite at the core count we'd want for building packages). To help it out, a new Nvidia DGX Spark joined the fray!

I might be one of the few people who bought this thing not for AI stuff. I've been watching this space for a while, and I believe it's the first ARM-based mini PC not made by Apple that reaches the performance level of Apple's M1.

...and for better or for worse, it was the best option I had for another aarch64 build machine. Ampere's stuff is interesting, but more targetted for enterprises I think and not really consumer-facing; Qualcomm has talked about ARM PCs for quite a while without really shipping anything I could get my hands on today; Minisforum shipped the MS-R1 recently but performance isn't stellar from what I've read; Apple's making some really solid stuff, but Asahi only supports M1/M2, so using Mac hardware would either mean buying older hardware for Asahi support or virtualizing Linux on top of macOS (more painful to manage).

Together, the DGX Spark and the Mac mini are keeping ARM builds roughly on pace with x86-64. That's good because that's cleared a bottleneck in our build pipelines!

(I was kind of hoping the DGX Spark alone could keep up with the x86-64 runner, but that hasn't really played out. That said, I've seen some slowness on both the DGX Spark and the Mac mini that makes me want to dig in more on general performance of aarch64 for Brioche + our CI pipeline. I'm not ready to make any claims about the DGX Spark's performance, to be honest!)

Python packaging fixes

For a long time, our meson has had an issue where it would try to use the wrong Python interpreter (issue #838). Basically, the bin/meson script ended up with a shebang with an invalid path (well, a path that was only valid within a single instance of the sandbox).

Well, @jaudiger opened #1669, which finally fixed this for good by using a proper shebang. Now, bin/meson uses the shebang line #!/usr/bin/env sh with a small polyglot script to execute the actual Python interpreter. This fix is also reusable and applies to other Python-based packages too!

Lots more foundational X11 packages

@jaudiger has been busy adding a bunch of new packages related to X11/X.Org. We don't have any GUI-based packages in Brioche yet, but all of these dependencies are prerequisites on the path to getting the first one added!

Infrastructure rework

Our GitHub Actions runners are managed with GitHub's (in)famous Actions Runner Controller for Kubernetes. Previously, I set things up the lazy way by creating a separate Kubernetes "cluster" for each runner-- not really ideal!

With the new DGX Spark runner, I used this as a chance to overhaul the Kubernetes setup used for Brioche stuff. Now, each of our 3 runners are part of a single cluster, which makes it easy to view and manage all the runners together!

Actually, I'm hoping this is a 2-for-1 deal. Today, a lot of Brioche's infrastructure is hosted by Cloudflare (plus some parts using Fly.io and Neon). I've honestly been unhappy with using Cloudflare for a long time[^cloudflare], and have long had the itch to move. Recently, I was inspired by Grebedoc and the modest infrastructure that powers it. So this month, I've been trying to get all the pieces in place for a "production-ready" Kubernetes cluster for self-hosting more Brioche infrastructure.

It's been... pretty painful getting everything working! But my hope is that this will be a long-term investment to keep Brioche's infrastructure small, sustainable, reliable, and fast! I'm hoping to write a lot more words about what I've been cooking once I've got more to share.

Housekeeping

New packages

Since the last update, there were 48 new packages.

New packages:

aicommit2
cargo_leptos
chezmoi
corrosion
datafusion_cli
delta
dumbpipe
font_util
fontconfig
fonttools
git_spice
harsh
helm
helm_docs
helmfile
helmify
httm
impala
leetcode_cli
libev
libfontenc
libimagequant
libpciaccess
libxcvt
libxft
libxinerama
libxkbfile
libxshmfence
libxxf86dga
libxxf86vm
lzlib
markdown_oxide
mergiraf
nexttrace
numcpp
ov
parqeye
pixman
rulesync
rustnet
stylelint
topiary
vfox
vtcode
xauth
xcb_util
xcb_util_errors
xcb_util_image

Brioche core updates

#363: Update Cargo configuration
#364: Work around race condition between "didOpen" and "diagnostic" LSP messages
#366: Update LSP to load documents atomically
#367: ci: remove old hack, now Brioche 0.1.6 is out

[^cloudflare]: Even from day one, I wasn't super enthusiastic about using Cloudflare. Their outage this month was timely-- although I'm not really concerned about their uptime, but I don't like how centralized they've helped make the Web. More concerning to me is their policies and who they openly support. They did stop protecting Kiwi Farms-- eventually (good). But there's also a notorious forum centered on self-harm-- there's a great video esssay about it by Tantacrul-- which Cloudflare still protects today. They also proudly sponsor the projects Omarchy (the Linux distro made by DHH, who is a white supremacist) and Ladybird (the browser started by Andreas Kling, who feels that gender-neutral langauge is "political" and, therefore, not welcome in his projects). Life's too short, and I'd rather support providers I like. Cloudflare does not spark joy.

Brioche v0.1.6: Now for ARM64!

Kyle Lacy — Sun, 02 Nov 2025 22:29:45 GMT

Well, it's been a while since the release of Brioche v0.1.5. We've got some exciting new features in this release, plus Brioche finally supports aarch64 Linux (a.k.a ARM64 Linux) as a platform!

Whether you're new to Brioche or want to try out the latest improvements, check out the installation docs for instructions on installing the latest release! Oh, and if you want a detailed breakdown of all the changes from this release, check out the release notes too.

(and small heads up: there were some major changes to the release process, so automatic updates with brioche self-update aren't available for existing installations for this release! sorry!!)

aarch64 support

If you're on an aarch64 machine, you can now install Brioche! It'd be hard to even tell that anything's different from an x86-64 machine.

Behind the scenes, it took a lot of work to reach this point. We needed to pull in aarch64 versions of our sandbox rootfs components, add a way for build scripts to distinguish the current platform, update the toolchain to for native aarch64 compilation, rebuild and fix all of our existing packages, and set up a CI runner for package builds. We've had all this up and running in nightly since June, so all existing packages in the Brioche Packages repo are supported on aarch64 now, and provided in the cache! <sup>(...with a few exceptions when an upstream package doesn't support aarch64!!)</sup>

This is another step on our goal towards great cross-platform and cross-compilation support in Brioche. With native compilation for aarch64, this is a stepping stone towards cross-compilation. Also, going from one to two platforms is a stepping stone to lots of platforms. (Check the GitHub Discussion on cross compilation too, if you're interested!)

Live updates

The Brioche Packages repo is growing steadily-- and we just crossed 300 packages! With only a small number of maintainers, we need to be thoughtful in how we stay on top of continual package updates over time.

That's where live updates come in: a tool for automated package updates. Basically, each package defines its "live update" script. Then, we have a CI job that runs weekly, calls each package's live update script, then creates a PR automatically for each out-of-date package. From there, the PRs go through review. We're able to merge most update PRs as-is, and the rest act as a starting point for manual changes.

Let's go through a little example to see live updates in action. Here's a Brioche project for building eza:

import * as std from "std";
import { cargoBuild } from "rust";

export const project = {
  name: "eza",
  version: "0.23.3",
  repository: "https://github.com/eza-community/eza.git",
};

const source = Brioche.gitCheckout({
  repository: project.repository,
  ref: `v${project.version}`,
});

export default function eza(): std.Recipe<std.Directory> {
  return cargoBuild({
    source,
    runnable: "bin/eza",
  });
}

...oh, but our package is already out-of-date! eza v0.23.3 was released at the start of October. eza seems to have new versions published on their GitHub Releases page, so we'll use the std.liveUpdateFromGithubReleases function:

// ...

export function liveUpdate(): std.Recipe<std.Directory> {
  return std.liveUpdateFromGithubReleases({ project });
}

Now if we run brioche live-update, we'll see that the version is automatically updated for us!

 export const project = {
   name: "eza",
-  version: "0.23.3",
+  version: "0.23.4",
   repository: "https://github.com/eza-community/eza.git",
 };

Not just that, but our lockfile brioche.lock was also updated, so it now points to the new commit for the v0.23.4 tag!

Under the hood, each piece is pretty simple:

brioche live-update builds and runs the liveUpdate export from our project-- which in turn just uses std.liveUpdateFromGithubReleases.
std.liveUpdateFromGithubReleases uses a small Nushell script to call the GitHub Releases API. It takes our starting project export, updates the version based on the API response, and prints the new value as JSON to stdout.
Brioche parses the JSON, then updates our project.bri, substituting the previous export const project with the JSON value.
Brioche updates the lockfile, if needed. This works because statics-- like Brioche.gitCheckout-- can reference the project export.

This approach is both simple and powerful, making it easy to add pre-built live-update scripts from a variety of sources, or project-specific ones as needed. And @jaudiger has been hard at work adding pre-built live-update scripts for a bunch of common package sources!

Faster bulk checking and formatting of packages

brioche check and brioche fmt can both take multiple packages to work on in bulk, since Brioche v0.1.2.

Well, now, both have been re-worked: both commands now load and handle all the projects in bulk too, rather than just processing them one by one. For brioche check especially, this can be a huge improvement! Back when PR #255 was opened, I measured the time it took to call brioche check with all the packages from the brioche-packages repo, and it went from 6 minutes 1 second to just 7.5 seconds-- that's 48 times faster!

(the actual speed improvement will be proportional to the number of projects you're checking, though!)

Lazy building (experimental)

The main build job in the Brioche Packages CI pipeline boils down to a simple loop-- basically:

for package in package/*; do
  brioche build -p "$package" --sync --locked
done

Unintuitively, this ended up wasting a lot of time and bandwidth. That's because, today, brioche build always ensures the build result exists locally (either building it, or pulling it from the cache). In other words, we were fetching the output for every package on every CI run, even if the package was already up-to-date in the remote cache!

I ruled out updating our CI pipeline to look at which files changed. It's not specific enough for our needs, since we'd also want to re-run if a dependent package changed too. It also creates awkward issues and failure cases if earlier CI builds failed.

Instead, I added a new flag for brioche build: --experimental-lazy. Basically, it first checks if the build result exists in the remote cache, and skips doing anything else if it does. It avoids the useless pulling from the cache that we don't want!

I added it as a new build flag for now; mostly because the implementation was a bit hacky and awkward, and partially because I felt like the old behavior of pulling the cache results could be useful in some cases. I also still feel this flag is experimental (hopefully that's clear from the name)! A few thoughts:

The name is not final of course. It could be renamed to --lazy (or some other name)
...or, in the future, we could make this the default behavior of brioche build (possibly with, say, an inverse --pull flag to always pull the result from the cache)
The behavior is also not final. It doesn't exactly play well with other flags like -o / --output today
We've been using this for a while now in the Brioche Packages repo, and so far nothing has broken terribly!

Lazy publishing

The basic flow of the Brioche Packages CI pipeline basically does this:

Check each package
Build each package
Publish each package

We've already talked about how we've sped up checking and building, but we've also sped up publishing!

Each time we ran brioche publish, Brioche would first run checks against the package. But by the time we reached step (3) in the workflow, we've already run brioche check on each package in step (1), so this check was redundant. And worse, brioche publish doesn't support taking multiple packages to publish (today), so each of these checks were run sequentially, just like we had been doing before.

So, @jaudiger added the new --no-verify flag for the brioche publish command (PR #326). As the name implies, it skips over the checks we do before publishing. This sped up the "publish" step in our CI pipeline roughly from 30 minutes to 3 minutes-- that's 900% faster!

(again, it's actually proportional to the number of projects you're trying to publish! also please don't use it unless you're already calling brioche check first!!)

Shorthand (non-function) exports

Let's look at a basic Brioche project (based roughly on our rust_backend example project):

import * as std from "std";
import { cargoBuild } from "rust";

export default function () {
  return cargoBuild({
    source: Brioche.glob("src", "Cargo.*"),
    runnable: "bin/rust_backend",
  });
}

The meat and potatoes of our build is wrapped within the export default function () { }. But having to wrap our build in an extra function doesn't seem to be super helpful here, does it?

Well, now you get rid of the function wrapper, if you'd like!

import * as std from "std";
import { cargoBuild } from "rust";

export default cargoBuild({
  source: Brioche.glob("src", "Cargo.*"),
  runnable: "bin/rust_backend",
});

Both snippets are effectively the same now, so it really comes down to personal preference for which style you go with. For Brioche Packages, we'll likely stick with the existing wrapper functions for a number of reasons (specifying the type of the recipe, giving the function a module-local name for use with tests, deferring work until a recipe is evaluated). But I have some ideas for future features where non-function exports could be a big improvement for ergonomics, plus I think this change makes things more consistent!

Release process changes

Our release process for Brioche went through a massive overhaul during this release, stemming from PR #281.

Unfortunately, the new release structure isn't compatible with the pre-existing self-update process-- sorry about that! Whether you have an existing installation of Brioche or not, check the "Installation" docs for details on installing the latest release.

This is mostly an internal change, and one that should make it easier to get new releases out faster. But there are a few interesting things that you might notice from the outside too:

Releases are now signed! Both the installer and self-updater will validate the signature against our "release" public key before installing.
Releases are now attested, too. We don't do any validation around the attestation (today), but this gives strong guarantees about how the release artifacts were built. Check out GitHub's docs on artifact attestation for more context.
We've now moved entirely over to "packed builds", finally! (Unpacked builds are still included with nightly releases though)
The release artifacts are distributed as .tar.xz files, meaning they're smaller (and therefore quicker to download)!
The install script now lives in its own repo: brioche-installer (still missing a README + docs at the time of writing, but hopefully that will change soon)

Well, this release ended up including a lot more than I was expecting... but it makes sense since it's been over 6 months since our last release.

Going forward, I'm aiming for smaller, more frequent releases, so expect less exciting announcements! The new release process should make it much easier to get new releases out the door. Plus, if you didn't see the news from the last project update, I'm now working on Brioche part-time!

Brioche Project Update - October 2025

Kyle Lacy — Thu, 30 Oct 2025 21:58:04 GMT

This month has been a really exciting one... both for Brioche, and for me personally!

Oh, and this month we also crossed 300 total packages! 🎉

Status report

Officially working part-time on Brioche!!

Even before it's initial announcement, I've worked on Brioche as a side project, working on it on nights and weekends. Making time to work on Brioche outside my dayjob as a full-stack developer while keeping enough downtime to not burn myself out has been a challenge, to say the least!

Well, as of this month, I've since switched from working from full-time to part-time at my dayjob (now as an independent contractor). So now, I work 3 days a week as my source of income, and I currently spend 2 days a week dedicated to Brioche! That frees up my weekends for resting or socializing or other side projects or other hobbies (or, to be honest, more work on Brioche-- but I'm trying to get better at letting myself not work on Brioche whenever I have free time)!

I'm really really excited about this and there's a ton more I could say!! But a few major ones:

I'm still funding Brioche out of my own pocket. Most of our recurring costs are from hosting the cache and registry (our build machines are self-hosted in our homelab).
Even with my reduced income and paying for Brioche resources, our household finances are still sustainable for the forseeable future.
...but this arrangement probably won't be sustainable for me forever. Realistically, I'll probably go back to working full-time again someday.

Now taking donations / sponsorships!

Since I'm now working on Brioche as my part-time job (effectively), I felt it was finally the right time to start accepting patronage. If you're interested in pitching in, I can take your money thorough either:

Liberapay (a Liberapay "team" for Brioche, which right now just has me)
GitHub Sponsors (my personal profile)

(there's already a few links on the website pointing to both platforms-- including on the new Get Involved page!-- but it's not consistent everywhere yet, so more links for both will probably show up in a few places)

I'm hoping that, as Brioche grows, a few folks would be willing to chip in enough to partially or fully cover Brioche's recurring costs, or even to pay for more self-hosted hardware-- like more build machines. Or if Brioche becomes a big success one day, it'd be a dream if those sponsorships could even sustain me for full-time work on Brioche indefinitely, or eventually be enough to pay to bring on other contributors part-time or full-time!

Small aside: this is something I've considered for a long time, but soliciting donations is something that I've felt a tinge of guilt around-- I'm not sure why to be honest! But switching to part-time was enough of a push for me to finally go and get it set up. I'm staunchly against taking VC funding or other investments for Brioche, so asking for donations is the first step on the path to making Brioche sustainable long-term.

New docs for contributors

I added a few docs geared towards potential contributors:

"Contributor's Overview": A breakdown of the different repos and structure of Brioche. Should help interested folks get their bearings and help them decide what to contribute to.
"Packaging Guide": Some general details on how to contribute new packages, with some specifics for a few common languages / ecosystems.

Both pages are sort of a "first pass", and I think both could use improvement / iteration (if you have ideas to improve them, contributions are welcome in the brioche.dev repo!)

I also added a new "Get Involved!" page. This was meant to be a catch-all for folks that would be interested in discussing Brioche (links to Zulip/Discord/GitHub Discussions), contributing to Brioche (links to docs pages on contributing), or supporting Brioche (links to Liberapay + GitHub Sponsors). I always wanted to link to more things in the header, but it's kinda space-constrained especially on mobile, so this was what I came up as an "umbrella" to cover multiple semi-related links at once.

Build Python with optimizations

@jaudiger opened a PR to enable compiler optimizations and LTO for our python package: brioche-dev/brioche-packages#1522. A small change (and one that's makes our CI runners sweat!), but a nice improvement for folks that use Python through Brioche!

More consistency fixes in `brioche-packages`

@jaudiger got a lot of PRs in to help fix miscellaneous issues in brioche-packages-- especially issues related to absolute paths and to live-update scripts:

Ensure std.pkgConfigMakePathsRelative and std.libtoolSanitizeDependencies follow symlinks (#1536)
Add functions for live-updates from Gitea, GitLab, Go modules, and Python PyPI packages (#1532, #1468, #1396)
Fix freetype so freetype-config uses relative paths (#1530)
Set ACLOCAL_PATH when relevant (#1411)
Add package test for linux (#1402)
Remove unused env vars from re2c (#1355)
Remove unused CMake from bubblewrap build (#1392)

Oh, and we had some discussions around the .pipe() method, and landed on our preferered style:

// Preferred:
recipe.pipe(function1).pipe(function2);

// ...instead of:
// recipe.pipe(
//   function1,
//   function2,
// );

See the discssion in #1594 for more context

Some packages CI / repo improvements

@jaudiger added some PR templates for adding / updating packages in the brioche-packages repo: https://github.com/brioche-dev/brioche-packages/tree/main/.github/PULL_REQUEST_TEMPLATE (permalink).

GitHub doesn't support templates for PRs (today), but this will be useful to help new contributors make sure everything that's needed is in place for a new package!

Also, @jaudiger got some fixes in to the CI pipeline itself to check for formatting (#1417), and to skip checks during the "publishing" step (#1413)-- which will help get new packages and package updates released faster!

Installer & release work

It's been a while since the last Brioche release back in April. There haven't been any huge new features, but there have been a few minor fixes-- and linux-aarch64 support!-- so it'd be nice to get a new release out.

...that's how I've felt for a while, but the current release process is pretty painful, with a lot of manual steps. So, I worked on cleaning up the mess that was our release process, and almost all the pieces are in place now:

Restructure build artifacts to unify packed / unpacked builds (#281)
Sign binaries we build in CI (#281)
Restructure release/update metadata in https://releases.brioche.dev
Update self-updater to use new metadata format and to check signatures during updates (#281)
Create separate repo for installer script: https://github.com/brioche-dev/brioche-installer
(In progress) Add automated CI pipeline for releases
(Post-release) Update website to serve new installer script
(Post-release) Update setup-brioche action to directly use new installer script

As part of that work, I did publish a new prerelease build of Brioche: v0.1.6-rc.2 (don't ask what happened to rc.1). This isn't intended to be used directly, but was meant to kick the tires on the new work-in-progress automated release pipeline. There'll also probably be one or more prereleases like this as a final pass.

Anyway, the hard parts should all be in place now. I'm really happy with how smooth the new release pipeline is turning out! Brioche v0.1.6 should be coming really soon!

Issues galore!

I spent some time just opening issues for miscellaneous Brioche ideas and improvements. For me, this is helpful to keep track on what I want to work on next, to have things to link to when discussing Brioche features, and to help give others an idea on the direction Brioche is heading. Plus, it's useful for collaboration in case others want to help out (I try to write pretty detailed issues!), and for gathering discussion.

If you have any feedback, please don't hesitate to comment on any open issues! A few that I'd like to call out that could use some discussion/feedback/ideas:

Dependencies / imports via git repos (#333)
Unified CLI syntax for projects/packages/exports (#320)
Spawn recipes within recipes (sccache-like builds) (#343)
Overhaul brioche jobs command (#339)
Support locking things in the lockfile using user-defined functions (#353)
WebAssembly-based sandboxing (#322, not opened by me but still wanted to highlight it!)

Housekeeping

New packages

Since the last update, there were 50 new packages

During October, we just crossed a huge milestone: we now have over 300 packages total!!! 🎉

New packages:

aks_mcp_server
bacon
cargo_binutils
cargo_bisect_rustc
cargo_c
cargo_deny
cargo_edit
cargo_expand
cargo_flamegraph
cargo_fuzz
cargo_generate
cargo_llvm_lines
cargo_machete
cargo_make
cargo_outdated
cargo_release
cargo_smart_release
ck_search
duktape
gitea_mcp_server
github_copilot_cli
github_mcp_server
json_c
lcms2
libint
libsrtp
libtiff
libvpx
libxaw
libxext
libxfixes
libxi
libxmu
libxrandr
libxrender
libxtst
lint_staged
mpdecimal
opus
promptfoo
rust_analyzer
sendme
skim
slang
terraform_mcp_server
typst
velero
wakatime_cli
xxhash
yasm

Brioche core updates

#314: Update GitHub CI
#315: ci: update workflow to use runs-on instead of runs_on
#316: ci: update workflow to use matrix include syntax
#317: refactor(ci): use latest version of brioche-dev/setup-brioche to simplify Brioche installation
#323: build: update object_store to version 0.12.4
#281: Sign and restructure built binaries
#324: Switch to xz for Brioche release artifacts
#326: Add publish args to skip some checks when used in CI
#335: build: update opentelemetry dependencies to latest versions
#336: build: remove assert_matches from dependencies, add to dev-dependencies
#337: Small tweaking here and there
#342: Add internal post-install step for Brioche installation and updates
#338: Update efficiency of format command when working with multiple projects
#345: build: remove target setup from rust toolchain toml file
#346: Update Dependabot config
#352: Update CI to attest artifacts

Brioche Project Update - September 2025

Kyle Lacy — Mon, 29 Sep 2025 01:30:29 GMT

This month has seen some very nice improvements that span multiple packages, some OpenTelemetry improvements, and some yak shaving

Status report

CMake build improvements

@jaudiger got a change in for CMake builds in #1144, which uses workDir for the CMake build directory. With this change, the source directory is now writable during the build-- previously, the source directory was read-only, which caused problems for some packages.

This resolved #677, which has been outstanding for a while

More build parallelism

@jaudiger opened #1149 and #1194 to update all of our make-based builds to build in parallel where possible, using make -j "$(nproc)". This can make a big difference both when testing changes locally and when getting changes deployed through CI!

And on the same track, @jaudiger opened #1149 opened #1195 too, which automatically does the same thing for all CMake-based builds-- well, those using the cmakeBuild function at least! This behavior can also be manually overridden still (see the PR for details).

libtool / pkg-config fixes

We've had the function std.pkgConfigMakePathsRelative() around for a while-- you can think of it like a post-processing / cleanup step to fix-up pkg-config files in a package, so paths are resolved properly when used as a dependency for other packages. This is needed because most software projects out in the world use absolute paths in their pkg-config files, while Brioche really needs relative paths. libtool files has the same problem: they often use absolute paths, while we'd really prefer they didn't.

So, I pulled a report of all libtool files across all packages, and @jaudiger opened some PRs to get those fixed as well: #1286, #1288, and #1290. Now, we use the new std.libtoolSanitizeDependencies() as needed, which automatically fixes any libtool files we find in a package!

Oh, and #1287 also added some calls to std.pkgConfigMakePathsRelative() for some packages that were missing it, too.

X11 libraries

@jaudiger has been on a tear adding a bunch of X.org packages lately. There's been incremental progress lately towards supporting GUI-based packages in Brioche, and this continues the same theme. A major milestone we just hit was with the libx11 package, which was just added recently in #1291!

OpenTelemetry improvements

At the end of last month (just after the window of the last Project Update), I spent some time cleaning up Brioche's built-in OpenTelemetry / tracing output in #305. @jaudiger also opened #306 to export logs as OpenTelemetry logs (previously, we wrote logs only as events instead). Together, these changes should make it much easier to trace Brioche using tools like Grafana's LGTM stack.

Yak shaving: instrumentation tools

I took a small detour this month a made a few small debugging / instrumentation tools. Are these related to Brioche? Not directly, but these tools were meant to help with Brioche-related tasks-- and I think there's a chance we'll have some of this tooling integrated in Brioche one day!

Background context: Brioche doesn't have much built-in tooling for debugging a failed build. We have process output logs, a debug shell, and direct access to the files from the failed sandbox. Beyond that, I often use strace-- a tool that records all the syscalls of a process-- which is extremely useful for figuring out which command deep in a build failed, what library or file couldn't be found, etc.

But strace can be pretty hard to parse! So I wrote a little tool called systrument. It basically parses strace's output, and converts it to a structured, graphical format:

<div class="bg-gray-200 dark:bg-gray-700 px-4 py-4 rounded-lg"> I also recently came across this list of "Process Tracing Projects", with other similar projects! </div>

I haven't gotten much use out of this tool yet, but from the little I've played with it, I'm hoping it'll pay dividends for diagnosing build errors. If it does, I think it'd be neat for a similar tool to be integrated in Brioche directly, similar in spirit to Ninja's log file.

Shifting gears, a totally unrelated problem: we have a very small pool of CI runners for building packages-- one aarch64 runner and one x86-64 runner. Since Brioche doesn't support cross-compilation yet, every package needs to be built on both runners before being published.

I wanted to visualize how our runners are running. In the short term, this could potentially help spot low-hanging fruit; in the medium-to-long term, it'll help to figure out build machine utilization, which will help us provision more build machines.

So I wrote ci-instrument, which calls the GitHub API and produces a file that can be loaded into the Perfetto UI to show GitHub Actions jobs over time.

It's really cool to be able to see visually where the bottlenecks are in our build pipelines, and to see which package builds are slowest!

Housekeeping

New packages

Since the last update, there were 19 new packages:

cargo_tarpaulin
eslint
go_task
igrep
libdnet
libnet
libx11
libxcb
libxdmcp
libxpm
libxt
tenv
terraform_docs
terraform_ls
traefik
tree_sitter
unixodbc
usage
xcb_proto

Brioche core updates

#303: Fix panic when enabling OpenTelemetry output
#305: Improve OpenTelemetry output
#306: feat(otel): refactor the code and add logging exporting
#307: Continue to resolve Clippy pedantic lints
#309: build: update Rust version to 1.90
#310: chore(clippy): add unreadable_literal lint to Cargo.toml
#311: Upgrade Brioche-in-Brioche dependencies
#312: Allow setting hash algorithm for HashSet/HashMap
#313: Resolve all the pedantic lints

Brioche Project Update - August 2025

Kyle Lacy — Fri, 29 Aug 2025 03:11:40 GMT

Not a lot of new headliner features made it in this month, but lots of little incremental improvements have been piling up!

Status report

Set up merge queues for CI

Each Friday, we run an automated GitHub Actions workflow for "live updates", where scripts check for new upstream package versions, then create separate PRs for each package. Each PR would trigger a build immediately, and could be merged after being reviewed and after a successful build.

But this setup started causing a "traffic jam", especially since the number of packages has been steadily growing! We only have one CI runner for each of x86_64 and aarch64 today, so triggering builds both when these live-update PRs open and when they were merged was slow, and could create a lot of wasteful / unused build results. Not to mention, if there were breakages between 2 new updated packages, we'd only find out when the main branch would fail-- which would then block any new packages or updates from going out as well!

We hit a particularly bad case starting August 8th: it took probably 36 hours to work through all the queued build jobs! That weekend, I updated brioche-packages to use merge queues instead. This was my first time using GitHub's merge queue feature-- I think it's kinda confusing... but getting this in has been really helpful for improving our CI!

Basically, live-update PRs never trigger builds anymore. Instead, every PR gets "queued" before merging, which is where we do the actual build. If the build passes, only then do we push to main, which then publishes the packages to the Brioche Registry. This avoids some "wasted" builds that are never published, prevents main from ever being broken due to conflicts between new versions, and gives us tools like limiting concurrent builds, rearranging PRs in the queue, etc. It's not a perfect setup, but it's a big improvement!

"Lazy builds" for CI

For builds, our CI pipeline basically just runs brioche build -p ./package/${package_name} for each package. Build results are cached of course, but this still ends up being pretty wasteful because brioche build always makes sure the build result is available-- downloading it from the cache if it doesn't exist locally.

Oh, and even though the new merge queue setup ensures PRs are built before reaching main, we still call brioche build for each package before publishing (that lets us still push fixes directly to main without going through the PR process, but that's a last-resort measure). But that also means each PR will fetch every package from the cache twice before publishing!

This was slowing down builds somewhat and wasting a little money each month for our cache cloud costs. As a quick-and-dirty workaround, I added a new --experimental-lazy flag in brioche#294. When running brioche build, this flag first checks if the build result exists in the remote cache, then returns early if it does. We now always use this flag for brioche-packages builds, so this saves a ton of time and bandwidth wasted fetching packages unnecessarily from the cache.

My gut feeling right now is that we should make this behavior the default eventually. Actually, the reason I didn't make it the default yet was because I want to do some internal refactoring to make this change more natural first (the current implementation actually checks the cache twice and can only short-circuit the top-level build-- these are both not ideal but are consequences due to how builds are currently handled, and I think reworking some stuff could make "lazy builds" the default, basically).

`object_store` update for better retry logic

Back in July, @nz366 opened brioche#286 to improve the retry logic when encountering network errors when fetching from the cache.

Well, it turns out that the the upstream object_store crate had a bug that prevented retrying requests due to connection errors, which was fixed in this PR. So once the new version of object_store was released, @nz366 opeend brioche#287, which just bumped object_store to the new version that included the upstream fix.

Sometimes it stings to see a code change thrown away because you tried to fix something too fast... but we got the retry improvements in the end either way!

Even more live-update improvements

@jaudiger has continued with lots of improvements to the live-update scripts, which helps us keep as many packages up-to-date with as little friction as possible! A few specific PRs:

#974: Add validation to version matching regex
#1098: Refactor projects with multiple versions for consistency and ease of updates
#1103: Update GitHub live-update scripts to support updating multiple simultaneous package versions

Housekeeping

New packages

Since the last update, there were 20 new packages:

coturn
cspell
dclint
deno
gopls
hugo
hunspell
isl
keep_sorted
libice
libsm
libxau
libxc
libxcrypt
mutagen
nuspell
socat
terragrunt
xorgproto
xtrans

This month, we had quite a few people to brioche-packages! Thanks to @jaudiger, @asheliahut, @easrng for PR contributions, and additional thanks to @rawkode and @nz366 for help in #366 towards adding deno!

Brioche core updates

Bump object store (#287 by @nz366)
Fix CI build for packed aarch64 artifact using x86-64 (#292)
Use Brioche nightly for building packed aarch64 build (#293)
Add new --experimental-lazy CLI option for brioche build (#294)
Update Rust version to 1.89 (#295 by @jaudiger)
Update clippy lints (#297 by @jaudiger)
Fix live-update removing some TypeScript annotations (#298)
Resolve all the automatic fixable lints from pedantic (#299 by @jaudiger)
Use lazy lock when it's possible (#300 by @jaudiger)
Remove features from tokio-util dependency (#301 by @jaudiger)

Brioche Project Update - July 2025

Kyle Lacy — Thu, 31 Jul 2025 10:11:48 GMT

Status report

Initial exploration of GUI packages

@jokeyrhyme started a Zulip discussion related to adding GUI packages in Brioche. That unveiled a bug in the meson package (#838) which is still blocking more progress. But in the interim, he was still able to add a new package: font_iosevka. I see this as an exciting milestone as the first step on the path towards support for GUI packages in Brioche!

The aforementioned meson bug is still an annoyance, and there are still a lot of steps until GUI apps are well-supported in Brioche. If you're interested in getting involved, feel free to reach out on Zulip, Discord, or open a GitHub Issue or Discussion!

Lots of live-update improvements

@jaudiger has contributed a ton of improvements and fixes in the brioche-packages around live-updates in the past month, including suppport for live updates via GitLab, crates.io, NPM, and GitHub tags; more options for automatically matching/sanitizing version numbers; and taking on the work to resolve any issues each week when automatic live-updates fail.

We have a triple-digit number of packages now, and all the improvements to live-updates has been crucial to making sure we're able to stay on top of upstream updates!

Infrastructure improvements

IRL, @asheliahut and I were pretty busy this month preparing to host our family over the last week (it went smoothly overall and it was great getting to spend time with them)!

Before the trip, we wanted to clear out the spare bedroom, where we had our our 22U server rack set up. The noise and heat made the room unlivable (we jokingly called it the sauna).

We ended up putting a new 42U server rack in the garage and moved our networking equipment there... plus got a new insulated garage door, an AC minisplit in the garage, a new electric subpanel, etc. It ended up being a big project!

Anyway, I use a mini PC and a Mac mini as GitHub Actions runners for Brioche. We moved those into the garage too, but the new server rack gives a lot more room to grow out to more build machines over time! Basically, we're future-proofed for lots more self-hosted homelab infrastructure for Brioche.

Housekeeping

New packages

🎉 This month, we crossed the 200 package milestone! We're at 216 total now! 🎉

Since the last update, there were 32 new packages added:

actionlint
binaryen
cargo_binstall
cargo_dist
cargo_hack
cargo_llvm_cov
cargo_minimal_versions
cargo_no_dev_deps
cargo_quickinstall
cargo_sort
cargo_spellcheck
dagger
eigen
font_iosevka
freetype
fribidi
gemini_cli
helix
iperf3
kubectl_view_allocations
libunistring
mpfr
pyrefly
tinycbor
trunk
vacuum
wasm_bindgen_cli
wasm_language_tools
wasm_pack
wasm_server_runner
wasm_tools
wrangler

Brioche core updates

Update Dependabot configuration (#274 by @jaudiger)
Remove unnecessary clone for Brioche.download (#279 by @jaudiger)

Brioche Project Update - June 2025

Kyle Lacy — Sun, 29 Jun 2025 06:39:51 GMT

In a surprising twist, this month has been all about getting aarch64 support in place!

Status report

aarch64-linux support

Last month, I wrote about my thoughts on getting aarch64 support in place. Actually, it was shortly after publishing that Project Update that I started re-thinking things a bit...

The end goal for cross-platform support is still the same as from the May Project Update. But I also know it might take a long time to implement everything needed for that! Meanwhile, the more packages we get that are only being built for x86-64 today, the more work it'll be to add support for aarch64 in the future. So I decided to do a very simple version of cross-platform support now. After all, the end-goal for cross-platform support will require a breaking change (probably), so now's the right time to get an implementation in place as an MVP.

The "cross-platform MVP" is very simple: there's a new std.CURRENT_PLATFORM const. It has the value "x86_64-linux" if you're running on x86-64 Linux, or the value "aarch64-linux" if you're running on aarch64 Linux. Pretty straightforward.

Most packages shouldn't need to use std.CURRENT_PLATFORM at all. std.toolchain and all the compilers / interpreters / etc. were updated to support both platforms, and everything cascades from there and "just works" (usually)!

The most exciting part though? (🥁 drumroll 🥁)

<div class="my-12"> <strong class="text-3xl block p-4 my-6 text-center border-4 rounded-lg border-gray-300 shadow-xl dark:border-gray-500 dark:shadow-black/80 ">All published packages have now been built for aarch64!<small class="text-sm align-super">*</small></strong>

<small class="text-sm text-gray-400">*with the exception of the package <code>bugstalker</code>. BugStalker doesn't support aarch64 (yet), but it markets itself as a "modern debugger for Linux x86-64", so fair enough!</small>

</div>

aarch64 support hasn't been included in a release yet, but if you install Brioche from source on an aarch64-based Linux machine, all packages can be installed and used from the registry!

The GitHub Actions workflow in brioche-packages now runs builds for both platforms. x86-64 and aarch64 are treated as equals, so all packages will need to build successfully for both platforms before packages are published (or with a magic comment to skip unsupported packages by platform, like for bugstalker).

TSDoc for package documentation

In brioche-packages#671 and brioche-packages#675, @jaudiger went through and updated all the doc comments across all packages to use TSDoc by convention.

The existing docs were basically Markdown. TSDoc has a ton of support in the TypeScript ecosystem, and it should play much nicer with existing TypeScript tooling. It also unlocks some options down the line, e.g. easily generating static doc pages for published packages.

Faster git cloning

brioche-packages#703 by @jaudiger uses the new git clone --revision option from git v2.49.0 when cloning a specific commit from a repo. GitLab's coverage of git v2.49.0 has a good explainer about the new option. Not only does this simplify git clones, but it's faster too! (Check the PR for some benchmark numbers)

More live-update updates

@jaudiger opened quite a few PRs to clean up, fix, and improve live updates in the past month. The biggest improvement came in brioche-packages#717, which added a new std.liveUpdateFromNpmPackages to easily live-update packages based on the version from the NPM registry!

Housekeeping

New packages

Since the last update, there are 30 new packages:

asdf
blake3
boost
cargo_chef
cyrus_sasl
dependabot
editline
gengetopt
gmp
gperf
iamlive
inframap
krb5
libevent
libfaketime
libyaml
libzip
lzo
lzop
markdownlint_cli2
ncurses
nlohmann_json
patch
readline
renovate
sqlx_cli
toml11
util_macros
wabt
yaml_language_server

Brioche core updates

Update OpenTelemetry crates (#257 by @jaudiger)
Fix another "file exists" bug when fetching projects from cache (#259)
Add JS op to get current platform (#260)
Resolve some Clippy pedantic lints (#263 by @jaudiger)
Upgrade all the transient dependencies (#265 by @jaudiger)
Update to Rust v1.88 (#267)

Brioche Project Update - May 2025

Kyle Lacy — Fri, 30 May 2025 08:21:36 GMT

This month has been a lot of work on ergonomics, future planning, and a lot of new packages!

Status report

Simplify recipes in `std`

brioche-packages#425 updated std to let you pass a function in most places that expected a recipe, rather than needing to call it first (e.g. passing openssl instead of openssl()):

import * as std from "std";
import meson from "meson";
import ninja from "ninja";
import cmake from "cmake";
import openssl from "openssl";

export default function (): std.Recipe<std.Directory> {
  // Previous:
  // return std.runBash`...`
  //   .dependencies(std.toolchain(), meson(), ninja(), cmake(), openssl())
  //   .workDir(source)
  //   .toDirectory();

  // New:
  return std.runBash`...`
    .dependencies(std.toolchain, meson, ninja, cmake, openssl)
    .workDir(source)
    .toDirectory();
}

I think this makes Brioche projects look simpler at first glance, and hopefully should be more intuitive too!

This was enabled by replacing the previous std.AsyncRecipe type (which could be either Recipe or Promise<Recipe>) with the new std.RecipeLike type (which can be Recipe or Promise<Recipe> like before, but also () => Recipe or () => Promise<Recipe>).

See the updated docs for more details on RecipeLike!

`pipe` utilities in `std`

brioche-packages#424 added a new .pipe() method to recipes, letting you combine together several utilities a little more ergonomically:

export default function () {
  // Previous:
  // let recipe = std.runBash`...`
  //   .workDir()
  //   .toDirectory();
  // recipe = std.setEnv(recipe, {
  //   /* env vars */
  // });
  // recipe = std.withRunnableLink(recipe, "bin/path");
  // return recipe;

  // New:
  return std.runBash`...`
    .workDir()
    .toDirectory()
    .pipe((recipe) =>
      std.setEnv(recipe, {
        /* env vars */
      }),
    )
    .pipe((recipe) => std.withRunnableLink(recipe, "bin/path"));
}

I found this to be especially helpful when using standalone utility methods like std.withRunnableLink, etc. Before, you'd have to shuffle between return ... to const recipe = ...; return ... to let recipe = ...; return ..., which could get kinda annoying!

We've adopted this style across the brioche-packages repo. But for your own projects, the old way still works if .pipe feels a little too functional for your tastes!

Oh, and there's also a standalone std.pipe function. It's basically the same, but as a standalone function rather than a method. Check the docs for more info about both versions of pipe!

Work on cross-platform support

The next big feature I'm aiming for is cross-platform support in Brioche. Specifically, my goal is to support aarch64 (ARM64) Linux as a build host + target. Since cross-compilation is also an eventual goal, I'm trying to build support in a way that will lead smoothly to cross-compilation too.

Unfortunately, I think there's still quite a bit of work remaining before we can get this implemented...

Design ideas

So, the "big idea" I'm working towards is to support dynamic bindings in recipes. Basically, you could have a recipe that looks like this (placeholder syntax):

export default function curl() {
  return std.runBash`
    # Build OpenSSL from source
    ./configure && make && make install
  `
    .workDir(source)
    .dependencies(std.toolchain, openssl)
    .hostPlatform(std.variable("hostPlatform"))
    .targetPlatform(std.variable("targetPlatform"));
}

The "trick" is that std.variable in this case doesn't return a string value like linux-x86_64 or whatever. Instead, it returns a "placeholder" value that we can substitute later. If you tried to console.log them, they would just show as variables named hostPlatform and targetPlatform. The Brioche runtime itself would substitute them-- say, based on a CLI option like brioche build --target ....

...why do we need that? Well, let's say I want to build curl for x86-64 and aarch64, and create a single tarfile for both of them. I'd like to write this recipe:

export default function () {
  // Build curl for x86-64 Linux
  const curlX64 = curl().setVariable({ targetPlatform: "x86_64-linux" });

  // Build curl for aarch64 Linux
  const curlArm = curl().setVariable({ targetPlatform: "aarch64-linux" });

  // Combine into a directory
  const curls = std.directory({
    "curl-x86-64-linux": curlX64,
    "curl-aarch64-linux": curlArm,
  });

  // Create a tarfile for the directory
  return createTarfile(curls);
}

The neat thing is that these hostPlatform and targetPlatform values can flow down into dependencies automatically. Since we're building curl for both x86-64 and aarch64, that means that openssl will be built for both as well.

This isn't just an arbitrary example either: since OCI / Docker container images are "just" tarfiles, we could use this to easily make multi-platform OCI images with very little extra work!

There's a lot of details I'm glossing over, and there's still even more work to iron out the details for this design, so I'll leave it at that for now.

Roadblocks

While starting work to implement and experiment with this design, I've basically fallen into a quagmire of issues:

To maintain backwards compatibility, I think the best option would be to handle these "dynamic bindings" as a preprocessing step on recipes.
Brioche has support "proxy recipes" today. A "proxy recipe" is a recipe that contains a hash for another recipe to use in its place. These were added as a performance optimization, since this lets us avoid (de)serializing duplicate recipes in the build graph.
Because proxy recipes are hashed eagerly, they basically conflict with having a preprocessing step. We also can't just "turn them off" for a number of reasons.
...but, I realized we could rely on referential equality in JavaScript as a simpler alternative to proxy recipes. This would be much cleaner and should be faster too.
...but that would require our own way to deserialize values from JavaScript to Rust, due to limitations with how we currently deserialize. But the Recipe type is huge and complex, and it'd be unwieldy to hand-write deserialization logic.

For (4), I've already got some prototype code that handles this properly on the JS side. For (5), I've been playing around with facet recently. Right now, I believe it's the best path forward for handling deserialization on the Rust side, which in turn should start to unravel these issues.

And just to be clear, cross-platform support is the catalyst for cleaning this up, but I think getting this cleaned up anyway will be a good change overall, even ignoring cross-platform support.

Breaking changes?

But, I'm also unsure if untangling the whole mess above can be done in a backwards-compatible way. Well, not so much if it can be done, but rather if it'd be better to do so as a breaking change...

I secretly hoped Brioche would graduate from v0.1.x straight to v1.0.0, but I've already got a wishlist of minor breaking changes I'd love to make to the data model. It might be time to rip the band-aid.

I'm still on the fence overall, but I'm leaning more towards it. That means cross-platform support might need to wait until Brioche v0.2.0.

Housekeeping

New packages

Since the last update, there were 64 new packages!

bash_language_server
biome
brotli
bubblewrap
bugstalker
cargo_about
cargo_audit
cargo_bloat
cargo_msrv
cargo_mutants
cargo_nextest
cargo_udeps
cmctl
codex
cosign
delve
difftastic
eks_node_viewer
expat
fzf
go_mockery
golangci_lint
gosec
grcov
gron
gtest
htop
hyperfine
jjui
kor
lerc
libcap
libdeflate
libgit2
libjpeg
libjpeg_turbo
libpng
libsodium
libssh2
libunwind
markdownlint_cli
mdbook
meson
mitmproxy
ninja
pcre
pnpm
rip2
sccache
snazy
starship
tailspin
tflint
trivy
uthash
uv
vegeta
xcaddy
xh
xz
yazi
zlib
zlib_ng
zziplib

Thanks to @jaudiger, who added a majority of these packages!

(the list in this section is so long now...! I feel like I'll need to eventually tweak the CSS a bit so it doesn't take up so much space on the page!)

Brioche core updates

Update live-update subcommand to use liveUpdate export (#237)
Speed up debug compilation (#238)
Remove use of legacy registry endpoints (#239)
Fix brioche live-update not showing output on error (#243)
Update Rust version to 1.87 (#248 by @jaudiger)
Upgrade Rust version for Brioche-in-Brioche (#249)
Add workaround for "file exists" bug in CI (#250)
Update CI macOS default images (#251 by @jaudiger)
ci: Add GitHub Actions updates through dependabot (#252 by @jaudiger)

Brioche Project Update - April 2025

Kyle Lacy — Sun, 27 Apr 2025 01:48:00 GMT

The biggest news this month was the release of Brioche v0.1.5! There were also 2 PSAs to be aware of from that announcement: registry support for Brioche v0.1.4 and earlier will be ending on 2025-04-30, and upgrading to Brioche v0.1.5 affects self-hosted registries pretty drastically, check the release announcement for more details.

Status report

Brioche v0.1.5 features

The Brioche v0.1.5 announcement includes all the details for the new release, so go check that out for more! Since the last project update, there were a few smaller changes that got merged prior to the release:

Support inheriting host CA certificates (#232). This adds a new special process template value that allows passing in the host's CA certificates into a recipe, but only when unsafe.networking is enabled! Once the supporting changes land in std, this means the ca_certificates package won't be needed for making HTTPS requests in most cases.
Implement currentDir option when baking process recipes (#231). This allows for changing which directory a process recipe starts in, rather than needing to use cd within a shell script
Implement the unsandboxed sandbox backend (#230). This is a very simple backend that can work as a "last resort" when other sandboxing wouldn't work.
Update custom cache to work as a layer with default cache (#229). The new cache was merged in February. When it was first merged, setting a custom cache would replace the default cache, which made it hard to use a custom cache in practice! Now it works as a layer on top of the default cache, which I made sure to get in before the v0.1.5 release.
Show download sizes in console (#227). Previously, the console would show the "blob count" when fetching from the cache, the "blob + recipe" count when fetching from the legacy registry, and would just show a percentage for downloads and archives. Now, it will consistently show a message with the current bytes and the total bytes (where possible). I found this to help give some context for large / slow downloads.
Fix V8 platform initialization (#225). I switched to daily driving Linux recently (from Windows + WSL) and ran into a weird crash in V8 when running Brioche tests. I dug in and found that it had to do with how the V8 platform got set up depending on your CPU flags. Fixing it was kinda dirty, but running test works for me again!
Merge various security fixes (various Dependabot PRs). There were several outstanding Dependabot alerts in Brioche, which have all been resolved. Most were in the runtime NPM project which has no security impact on Brioche due to the nature of how restricted our runtime is (I believe). But there was one fix that was relevant: an update to the zip dependency to resolve CVE-2025-29787. Support for unarchiving zip files was merged last month, but this feature only landed in v0.1.5, so there were no vulnerable versions of Brioche released!

Dropping support for earlier versions of Brioche

As mentioned in the v0.1.5 announcement, I'll be removing APIs used by older versions of Brioche from the registry after 2025-04-30, effectively ending support for Brioche versions <=v0.1.4.

Some packages in the brioche-packages repo have already started adopting cyclic dependencies, which already can't be fetched from the registry for prior versions. We've also started adding some features to std that depend on v0.1.5 features, so earlier versions of Brioche are already starting to lose out on the latest changes.

Package live updates

I merged #234 recently, which added the new brioche live-update subcommand. This new command allows us to define a special export per package, which allows packages to update themselves based on the latest upstream version. Thanks to the work of @jaudiger, 71 out of 90 packages already support live-updates, which will massively help with keeping packages up to date!

The name was based on Homebrew's livecheck subcommand. Originally, I was going to call it auto-update and so we've been using the name autoUpdate as the export, but I'm happier with the term "live update". The next step will be to rename all of the autoUpdate exports to liveUpdate.

I've also gotten a first-pass GitHub Action set up to create PRs from the live updates, many of which have started getting merged in brioche-packages!

`brioche.dev` updates

The source for https://brioche.dev is itself in a public repo, which includes automatic updates through Dependabot. Last week, I saw an exciting update come in that added support for using Tailwind v4 with Starlight!

...that update required a lot of fixes to get the site looking right again. But, I spent a lot of time fixing stuff, and merged the update. One really nice outcome is that the blog posts now use the same Markdown stylings as the doc pages, including header links. Nice!

Housekeeping

New packages

Since the last update, there were 4 new packages:

kubent (#308 by @jaudiger)
popeye (#300 by @jaudiger)
rclone (#322)
strace (#272)

Brioche core updates

Fix Deno Core / V8 platform initialization (#225)
Upgrade dependencies in runtime NPM project (#226)
Update console output to show download / archive sizes (#227)
Update Rust to v1.86 (#228 by @jaudiger)
Update custom cache to work as a layer with default cache (#229)
Implement the unsandboxed sandbox backend (#230)
Implement currentDir option when baking process recipes (#231)
Support inheriting host CA certificates (#232)
Add brioche live-update command to live-update/auto-update projects (#234)

std updates

Add support for zip archive format (#335 by @asheliahut)

Announcing Brioche v0.1.5

Kyle Lacy — Sun, 20 Apr 2025 06:20:05 GMT

Brioche is a new package manager and build tool that makes it easy to mix and match languages and tools for your own development projects. Check the installation docs to get started or run brioche self-update to update Brioche!

Since the release of v0.1.4, we've gathered a number of new features and quality of life improvements! Check the release notes for all the changes for this release, and thanks to @jaudiger, @paricbat, and @asheliahut for PR contributions for this release!

Before diving into the features, 2 quick PSAs:

Support for earlier versions of Brioche will phase out soon, so upgrading is strongly recommended. TL;DR: the registry will no longer support Brioche <=v0.1.4 as of 2025-04-30. See below for more details.
Self-hosted registries behave very differently now. If you're using a self-hosted registry, you'll need to update your configration and migrate your data after upgrading. See below for more details.

New cache

The biggest quality-of-life improvement day-to-day is the new cache system. Fetching large artifacts should now be much faster, and it's much easier to set up a self-hosted or private cache!

Since the initial release, Brioche has used a fairly naive means for storing and loading cached values: each individual file would be stored and fetched independently (although they were at least compressed and deduplicated by hash). That meant that fetching large directories with lots of small files would be painfully slow.

Now, artifacts are stored in the cache as a custom archive format. Plus, to reduce costs, archives are deduplicated using a content-defined chunking algorithm to avoid storing data redundantly.

But the best part? There are multiple supported backends for caching, so setting up a custom cache is very easy! Any S3-compatible storage provider should work with the custom cache, and it only takes a single env var to opt-in to the custom cache, which is much easier to use for CI/CD workflows. Check the docs for more details on caching.

Debug shell for failed build

Today, we have the brioche jobs logs subcommand to show all the output from a failed build. It's an indespenable tool for diagnosing build errors, especially when working on new packages. It's very useful! But sometimes though, I just wish I could reach into a build and poke around with it-- explore files, re-run commands with different parameters, etc.

The new brioche jobs debug-shell command lets you do just that. When a build fails, you can run brioche jobs debug-shell ${path_to_events_file}, and you'll get dropped right into an interactive shell, right from where the build failed! It'll run with the exact same sandbox semantics as the build did too.

Here's a quick recording to demo what this is like in practice:

Support projects with cyclic imports

This one was originally motivated by a long-standing draft PR in the brioche-packages repo: #228. Basically, this PR was blocked because there was a cyclic dependency between two projects: we wanted the curl package to use the libpsl package, but a helper function from libpsl dependends on curl (transitively).

In Brioche's JavaScript runtime (Deno Core / V8), this works without any hiccups. But for Brioche itself, I needed to make some adjustments to how projects were represented internally. It turned out to be a lot of work, but it works seamlessly now!

New `unsandboxed` sandbox backend

There might be times where the linux_namespace sandbox backend can't be used for builds or might be limited due to security policies.

So, I implemented the unsandboxed sandbox backend. As the name implies, it disables sandboxing. Generally, this isn't what you want, so this should only be used if you have no other options or if you know what you're doing! Check the docs for caveats and details on enabling this backend.

The coolest part though is that this sandbox only relies on Rust std, so it should work on any platform! This isn't very useful in general, but I think this will come in handy when standing up Brioche on new platforms.

Coming soon to `std`

Brioche itself includes several new features that will be used by the std package. These will be published sometime after release, and you'll need to update to the latest version of the std package to use them-- keep an eye on the std changelog for when they land!

Support for unarchiving `.zip` files

@paricbat implemented support for unarchiving zip files, so they can now be treated as first-class citizens just like .tar files!

Brioche.download(
  "https://github.com/brioche-dev/brioche/archive/refs/tags/v0.1.4.zip",
).unarchive("zip");

`process.currentDir` to change a process's starting directory

By default, processes start in an empty directory called work. You can pre-populate this directory using the .workDir() method, but this just writes files into this work directory.

Sometimes, it's useful to start in a different directory directly. Most commonly, it can be useful to start in $BRIOCHE_OUTPUT, which can be used to create or modify an artifact through a shell script.

Previously, you'd need to start your shell script with cd "$BRIOCHE_OUTPUT", but soon you'll be able to use .currentDir() to set the starting directory:

// Apply a sed script to each `.pc` file within `${recipe}/lib/pkgconfig`
recipe = std.runBash`
  sed -i 's|=/|=\${pcfiledir}/../../|' lib/pkgconfig/*.pc
`
  .outputScaffold(recipe)
  .currentDir(std.outputPath);

`Brioche.gitCheckout` to checkout a repo directly

Today, if you want to check out a git repo without specifying a commit hash directly, the easiest way is by combining the Brioche.gitRef static with git.gitCheckout, like this:

import * as std from "std";
import { gitCheckout } from "git";

const source = gitCheckout(
  Brioche.gitRef({
    repository: "https://github.com/brioche-dev/brioche.git",
    ref: "main",
  }),
);

Brioche.gitCheckout is a new static, which combines both functions together:

import * as std from "std";
import "git";

const source = Brioche.gitCheckout({
  repository: "https://github.com/brioche-dev/brioche.git",
  ref: "main",
});

PSA: Sunsetting support for older versions of Brioche

Projects with cyclic imports are represented very differently from projects without cyclic imports. As such, projects with cyclic imports won't work on earlier versions of Brioche.

On top of that, the new cache stores data independently from how the registry cached data previously. As of the Brioche v0.1.5 release, I'm paying costs for both the "legacy" storage as well as the new cache storage.

For both of these reasons, I'm planning on sunsetting support for prior versions of Brioche. On 2025-04-30 (or later), registry requests will fail for Brioche <=v0.1.4. The registry will no longer serve the endpoints used by prior versions of Brioche.

Existing data from the registry has already been migrated to the new cache, so this should be seamless when upgrading to Brioche v0.1.5 when using the official registry/cache. But I wanted to give a grace period before sunsetting prior versions.

If you have questions/comments/concerns about this timeline, please reach out via Discord, Zulip, or GitHub!

PSA: Self-hosted registry changes

With the new cache, project files are now stored and retrieved via the cache instead of via the registry. However, the registry is still used to associate project names with the artifact hash stored in the cache.

If you're using Brioche <=v0.1.4 with a self-hosted registry, this means a few things:

You'll need to update your config to use a custom cache along with a self-hosted registry, in order to store project files.
Since recipes/bakes/artifacts are no longer stored in the registry, you'll need to either start from a clean slate or migrate to the cache.

If you're using a self-hosted registry today and need help migrating to Brioche v0.1.5, please reach out via Discord, Zulip, or GitHub. This release includes some internal commands / options that can help with the migration, but these will likely be removed before the next release!

Brioche Project Update - March 2025

Kyle Lacy — Mon, 31 Mar 2025 07:35:49 GMT

Status report

Cyclic imports

Last month, I mentioned challenges with a package where we wanted cyclic/recursive imports. Well, I'm happy to report that that PR #211 added support for cyclic imports and has been merged in! It allows two projects to import each other as long as they're part of the same workspace.

Getting this feature over the finish line took a lot more work than I was expecting! To get into the weeds a bit: Brioche internally represented projects as a JSON structure that looked like this:

{
  // Value of `export const project`
  "definition": {
    "name": "curl",
    "version": "8.11.1"
  },

  // Hashes of the TypeScript modules from the project
  "modules": {
    /* ... */
  },

  // Any statics (downloads, etc) from the project
  "statics": {
    /* ... */
  },

  // Other projects imported by this project
  "dependencies": {
    "std": "<hash>",
    "openssl": "<hash>"
  }
}

This JSON structure is then normalized and hashed, which is used as the ProjectHash, which is used to uniquely identify the project. This hash is used for publishing and retrieving the projects via the registry, and is used in lockfiles when another project references it. Each value under dependencies is a ProjectHash value... which means that a ProjectHash depends on the ProjectHash values of its dependencies.

...do you see the problem? If fizz and buzz depend on each other, then fizz depends on the hash of buzz, and buzz depends on the hash of fizz, which... uh... you can't do!

To handle cycles, I introduced some indirection: projects get grouped into a "workspace", which basically just contains multiple of the JSON value from before:

{
  "packages/curl": {
    // Value of `export const project`
    "definition": {
      "name": "curl",
      "version": "8.11.1"
    }

    // ... same fields as before ...
  },
  "packages/openssl": {
    // ...
  },
  "packages/std": {
    // ...
  }
}

We hash this JSON value to get a WorkspaceHash. We still have a ProjectHash, but it instead comes from a JSON value that references the WorkspaceHash:

{
  "type": "workspace_member",
  "workspace": "<hash>",
  "path": "packages/curl"
}

So, as long as "workspaces" don't form cycles, we're golden.

I've also been using quotes around "workspaces". We do only group projects together when they're part of the same workspace on disk, i.e. they're under a directory containing a brioche_workspace.toml file. But, we only group projects together in a shared structure if we need to break a cycle: we build a graph of projects with their dependencies, and build a "workspace" from each strongly-connected component of the graph containing more than one project (and we validate that each grouped project contains the same brioche_workspace.toml file). This both ensures hash stability (since the hash only depends on the subset of projects part of the same workspace group) and backwards compatibility (if a project doesn't have any cycles, it still uses the "original" JSON format and so will keep the same hash as before, which is important for prior versions of Brioche).

Side note: I haven't been super happy with calling these grouped projects "workspaces" in this context, so I might switch to a different term before this feature gets shipped.

Debug shell for failed jobs

Okay, this one isn't super flashy, but it's something I've wanted for a long time and I'm really happy I finally had a chance to sit down and implement it! It landed in PR #215.

TL;DR: if a build fails, you can now run brioche jobs debug-shell $path_to_event_file and you'll get dropped into a shell in the sandbox for the failed build! It doesn't sound super exciting, but it can be a huge quality-of-life boost when trying to write a package with a slow build process.

The catalyst was LLVM, which I worked on packaging last week. The build failed consistently at 98%, after about an hour and a half and after consuming around 150 GB of disk space! Even worse: I tried running the build through strace to log all the subprocesses started by the build-- an invaluable tool for finding subtle build issues previously-- and something else went wrong earlier in the build process. My go-to debugging tool was interefering in some way with the build.

After I added the brioche jobs debug-shell, I could jump right into the LLVM build process. Since almost the whole build had already finished, running cmake build within the shell let me explore what went wrong-- basically doing an autopsy on the build itself. Along with strace, I was able to narrow down what went wrong with the build, short-circuiting the rest of the build that ran without problems.

The failure turned out to be from brioche-ld. I never implemented support for @file-style arguments, which was causing the build to fail. I got a fix ready pretty quickly in brioche-dev/brioche-runtime-utils#21. It would've taken much longer to track this down without the new debug shell command!

`Brioche.gitCheckout` groundwork

Currently, if you want to check out a git repo in Brioche, the recommended way is to combine the gitCheckout and Brioche.gitRef functions:

const source = gitCheckout(
  Brioche.gitRef({
    repository: "https://github.com/antonmedv/fx.git",
    ref: "35.0.0",
  }),
);

After some discussion about standardizing and improving how downloads from GitHub are handled in brioche-dev/brioche-packages#176, @jaudiger proposed having a single function to combine both steps.

Brioche.gitRef is a static, which means that it gets some super-powers from the runtime so it can push stuff into the lockfile. However, that also comes with some tight restrictions: it must be called with very simple arguments, or it'll return a runtime error. That also means that it can't just be wrapped in another function call.

So, to solve this, I added Brioche.gitCheckout as a new static in #218. The runtime itself treats it identically to Brioche.gitRef when it comes to the lockfile, but the brioche-packages repo can take care of the implementation so that it gets the commit and checks it out from the repo in one function call.

The implementation still hasn't landed yet, but expect to see it shortly after the next Brioche release!

New build machine

Whenever a build is triggered for brioche-packages, it runs on a server in my and @asheliahut's homelab. Unfortunately, this hasn't been a great setup for a number of reasons:

The server it runs on is used for other stuff too. Sharing CPU with the other stuff makes builds slower overall
The I/O on the server is really slow-- it's fine for low-usage stuff, but it does noticeably impact build times
There's some kind of issue in the kernel or firmware or something that causes disks to permanently disconnect until a reboot. It only happens about once a month, and seems to trigger more often with high I/O usage. Not a blocker, but it's definitely been annoying to deal with!

I finally decided to bite the bullet and ordered a new mini PC recently. I went for the Minisforum MS-A1, and it's now set up to run all the build jobs in the brioche-packages repo! Builds have been noticeably faster

Housekeeping

New packages

Since the last update, there were 10 new packages:

re2c (#241 by @asheliahut)
process_compose (#257)
icu (#258)
postgresql (#259)
libtirpc (#264)
rpcsvc_proto (#265)
php (#266 by @asheliahut)
caddy (#269)
nginx (#270)
llvm (#271)

Brioche core updates

Fix occasional error when an inline cache archive contains an empty blob (#196)
Rust 1.85 support (#197 by @jaudiger)
Handle projects with cyclic (circular) dependencies (#211)
Add a new command to start a debug shell within a job (#215)
Fix "File exists" error when fetching a project from the cache (sometimes) (#216)
Add support for Brioche.gitCheckout as a static (#218)

Brioche Project Update - February 2025

Kyle Lacy — Wed, 26 Feb 2025 08:41:51 GMT

Some highlights this month have been a major overhaul to build caching and more progress towards automatic updates!

Status report

Major overhaul (and speedup) for cached builds

Last month, I talked about progress on speeding up synced builds, which I still felt was a big win, but it involved some manual work at the packaging level. I wanted to come up with a way to improve syncing across the board without any extra manual work... which led to #179.

The problem is that even moderately complicated directory structures would take a long time to fetch from the registry-- each file and each subdirectory required a separate request. Things were downloaded in parallel to make this somewhat tolerable, but the sheer number of requests made fetching from the registry pretty slow.

Now, the new cache stores each artifact as a custom archive format. For small artifacts, that's basically it: fetch the entire archive, unpack it, and you're done. But, once the total data in the archive crosses some threshold (2 MB currently), we break the archive up into chunks using a content-defined chunking algorithm. Doing so means that two similar artifacts can share some underlying data, reducing the total amount of data stored in the the cache. Actually, the new cache uses less storage space than the data used by the registry for the same set of artifacts!

That said, the registry isn't going away either-- it's still used for mapping project names to project hashes. That is, when you run brioche run -r hello_world, the registry is used to determine which files to fetch from the cache for the name hello_world. But, if you were running a custom registry, you'll either want to migrate to a cache in parallel, or move entirely to using the new cache system.

Anyway, I plan on writing more in-depth on how the new cache works and especially the details of the custom archive format soon

Cache migration

Oh, and to follow up on the cache, I migrated all existing artifacts/bakes/projects from the existing registry to the new cache, which is already configured to be used by default when building Brioche from main today. The new code will also ensure all brioche-packages builds will be uploaded both to the legacy registry and to the new cache in parallel.

For the next release, migrating over to the new cache should be completely seamless

Support for unarchiving zip files

In #176, @paricbat updated Brioche's unarchive recipes to support unarchiving zip files! These work just like tar files, except the option to additionally decompress the file first isn't supported (since zip files natively handle compression internally).

The changes on the std side will be published following the next Brioche release, so the following will work:

const source = Brioche.download(
  "https://github.com/brioche-dev/brioche/archive/refs/tags/v0.1.4.zip",
).unarchive("zip");

LSP fixes and tweaks

While working on the new cache, I ended up causing a regression in the LSP code...

That was enough motivation to sit down and finally write some integration tests for the LSP server. Along the way, I found and fixed some other LSP bugs while fixing the regression, which all landed in #188. There were quite a few fixes, so check the PR for details.

I also made a minor tweak to the LSP: the LSP server will no longer remove unused dependencies from the lockfile on save. The main motivation is that I felt the current behavior could be surprising if you commented then uncommented part of a file. You can read more about the change in #192.

`brioche fmt` fix

@asheliahut opened #190 to fix a bug with brioche fmt. If you didn't explicitly specify a project path with -p, the command would just exit without doing anything! The PR fixed this, defaulting to the current directory if no project path is provided.

More work on auto-updates and tests

Since last month, @jaudiger has started adding a ton more test and autoUpdate scripts to tons of packages. Some of the newly-added packages have these functions too, so as we make progress on #94 and #165, a lot of packages should be ready to go on day one for automated tests and auto-updates!

Recursive import troubles

While working on adding libpsl support to curl, @jaudiger hit a case where two packages would need to import each other, which straight-up doesn't work today.

After a lot of trial and error, we came to the conclusion that there wouldn't be a satisfying way to work around this problem. So, I opened #175 to track work on adding circular project dependencies. I haven't gotten very far with the implementation yet, but it's my highest priority currently, and I'm hoping to get a PR up in short order.

Housekeeping

New packages

Since the last update, there were 9 new packages:

seaweedfs (#213)
libffi (#215)
aws_cli (#217)
libpsl (#224, thanks @jaudiger!)
github_cli (#225, thanks @asheliahut!)
nasm (#229, thanks @paricbat!)
steampipe (#234, thanks @asheliahut!)
restic (#240)
claude_code (#243, thanks @asheliahut!)

Brioche core updates

Use $BRIOCHE_DATA_DIR env var to override main Brioche path used for storing data (#171)
Add support for unarchiving zip files (#176 by @paricbat)
New cache (#179)
- Enable conditional PUTs for S3 object store (#181)
- Add config option to allow for HTTP-only cache requests (#182)
- Tweak how cache fetch jobs are displayed in plain output format (#183)
Enable some more Clippy lints (#185)
Fix miscellaneous bugs in LSP (#188)
Default brioche fmt to . (#190 by @asheliahut)
Tweak LSP to keep unused dependencies when saving (#192)

Brioche Project Update - January 2025

Kyle Lacy — Mon, 27 Jan 2025 07:19:42 GMT

It's been a while since I got a Project Update out. December was a turbulent month for me-- we had a loss in the family, we were out of town for a while, and I was sick for a while. And beyond all that, it was nice to unplug for a while to spend time with friends and family over the holidays too.

...anyway, since the start of the year, I've definitely picked up steam on Brioche again, and a lot has happened since the last update in November!

Status report

Brioche v0.1.4 released

About a week ago, I published Brioche v0.1.4! There were some fun headline features, but to me this release was more about finally getting a few incremental improvements out in the wild.

CMake

brioche-packages#144 introduced a new package for CMake. A simple project.bri for a CMake project might look something like this:

import * as std from "std";
import { cmakeBuild } from "cmake";

export default function (): std.Recipe<std.Directory> {
  return cmakeBuild({
    source: Brioche.glob("CMakeLists.txt" /* build files */),
    dependencies: [std.toolchain() /* other deps */],
  });
}

The biggest challenge here was to do with packed executables. It turns out CMake, in some cases, will try to update an executable's rpath after the build. Brioche's packed executable format interfered with CMake's logic here, so I ended up introducing a patch to the CMake build.

This patch updates CMake to look for Brioche's packed executables, and-- if found-- applies the rpath logic to the unpacked executable, then re-packs it. For normal non-packed executables, this logic works exactly the same, so you can use Brioche's version of CMake for totally normal builds too-- say, by running brioche install -r cmake!

Ubuntu troubles

I finally decided to update our GitHub Actions workflows from Ubuntu 22.04 to Ubuntu 24.04. That sent me down a rabbit hole, where I discovered that Ubuntu 23.10 started restricting user namespaces by default. Brioche uses user namespaces for build sandboxing, so this means Brioche builds don't work on Ubuntu out-of-the-box!

This led to a few outcomes:

I updated the main Brioche build to use Ubuntu 24.04, but added a sysctl config to "un-restrict" user namespaces (brioche#151)
I updated the setup-brioche GitHub Action to try installing an AppArmor profile, if it seems like it's needed (setup-brioche#2)
As part of the Brioche v0.1.4 release, I added a sandboxing fallback using PRoot. This has a performance penalty, but it makes sure Brioche can work out-of-the-box on Ubuntu 24.04 (brioche#159).

The work on the sandboxing fallback ended up taking me a lot longer than I expected. There were some false starts, but using PRoot seemed to be the most reasonable direction here. Specifically, we run PRoot within a namespace-- Ubuntu 24.04 lets you create an unprivileged user namespace, but prevents you from creating mounts within an unprivileged namespace.

So the idea is that we use namespaces just like we did before, except if using mounts fails for isolating the filesystem, we'll use PRoot instead (e.g. on Ubuntu 24.04).

PRoot definitely isn't a perfect solution, but it was the best solution that I could adopt easily. I also looked into using User-mode Linux, but everything I've read about it suggests it's not really meant as a sandboxing tool... plus I wasn't able to compile a working build of it. Longer-term, if Brioche is going to need sandboxing option like PRoot, I think it'd be interesting to build a DIY solution: a built-in ptrace-based sandboxing tool, but tailored for Brioche's sandboxing needs.

Shorter-term, I'd also like to add more sandboxing options. The next one would be an "unsandboxed" sandboxing mode, and later followed by a "qemu" sandboxing mode. I don't think either would be a good default, but would be a good option if you're using, say, Ubuntu 24.04 and want better performance than PRoot, but don't want to (or can't) lift the user namespace restrictions.

Sync speedup

@pzmarzly opened brioche#147, pointing out that syncing std.toolchain() for a minimal example build is really slow. I think solving this issue would be really valuable-- not just for first-time users, but also because brioche-packages builds start from a blank slate, meaning that this issue compounds a lot over time.

Today, things should be a lot better than when the issue was first opened, but it involved a few moving pieces: first, I added the new attach_resources recipe (discussed in the v0.1.4 announcement and from brioche#149). Then, I updated the std package to sync std.toolchain() as a tarball using the attach_resources recipe (brioche-packages#147). Combined, std.toolchain() will now get delivered from the registry as one big tarball instead of thousands of tiny files-- much faster!

I still have more ideas for speeding up syncing in general, so I didn't feel ready to close out brioche#147 yet-- expect more updates here for next month!

aarch64 work

brioche#150 introduced support for aarch64-linux builds in Brioche! This means you can now build packages targetting Linux on either aarch64 or x86-64 now!

...but, to make that usable, packages need to be built to support aarch64 as well. If you check out the wip/aarch64-linux branch, running brioche build -p packages/std -e toolchain works, and will create a toolchain for aarch64. I also verified that at least curl also builds with the toolchain changes!

But, the branch is not mergeable in its current state. There's no way for a recipe to choose between doing an x86-64 build or an aarch64 build right now. I've got an idea on how the host / target platform can get threaded down through recipes, but it'll take some work to implement. And, I'd like to hold that off until after doing some refactoring to the code first.

TL;DR: aarch64 works right now if you're willing to build from-source from a branch, but it's going to take some work to get everything playing nicely together.

Brioche-in-Brioche

As mentioned in the v0.1.4 announcement, Brioche can now build itself! For releases, this is useful because it leads to a packed build, which is fully portable despite Brioche itself targetting glibc.

(This also led to some changes to the install script, you can read about all the new options in the "Installation" docs)

For development, this is also pretty cool because you can now test your changes locally, and you'll get feedback beyond what you'd get just using cargo run! For example, if you update the JS runtime package under crates/brioche-core/runtime but don't rebuild it, you'll get an error during the build. We were already explicitly checking for this in our GitHub Actions workflow, but it's nice to have an easy way to run some CI-only checks locally without needing to install new tools. Oh, and you can also use brioche build -e runtime to rebuild / update the runtime too, even if you don't have your own NodeJS toolchain set up locally.

Performance optimizations

I spent some time cleaning up Brioche's Tracing / OpenTelemetry output in brioche#168, which makes it much easier to dig into performance issues.

That led me to find some low-hanging fruit for optimization, so I opened brioche#169, which can drastically speed up cache hits when preparing to do a build! For a (failing) test build, the time for the build to exit dropped from 15.51 seconds down to 1.12 seconds!

Progress on package auto-updates and tests

For a while, I've had in mind some ways we could automate simple package updates. Well, @jaudiger started picking up the work here and updated the alsa_lib package in brioche-packages#211 as the first step to landing this work!

The end goal will make it so routine package updates won't require any manual code changes: a simple CI pipeline should be able to handle the version bump in the project.bri file, then automatically run tests to catch any regressions. There will still be plenty of work for manual review and testing, but getting the routine work handled automatically should help a ton for keeping packages up-to-date.

Housekeeping

New packages

Since the last update, there were 13 new packages:

cmake (#144)
pstack (#145)
talloc (#150)
libarchive (#151)
proot (#152)
fx (#154)
dasel (#155)
scdoc (#157)
kmod (#158)
s2argv_execs (#159)
vdeplug4 (#160)
linux (#161)
just (#168, thanks @paricbat!)

There was also one extra package (jj) which was added in brioche-packages#148. @jaudiger pointed out that this was redundant with the existing jujutsu package, so jj was removed (brioche-packages#187). Oops...

And huge kudos to @jaudiger, who also updated 37 packages over the past week!

Brioche core updates

Add --display flag to explicitly choose an output format (#141)
- The new --display plain-reduced format is now used in the brioche-packages repo (see brioche-packages#143)
Add frame to process events after writing description (#144)
Add initial support for aarch64 (#150)
Fix OpenTelemetry runtime and shutdown (#142)
Add project.bri to build Brioche with Brioche (#157)
Update Linux sandbox to fallback to using PRoot for mounts (#159)
Clean up tracing output (#168)
Speed up outputs and fetching blobs (#169)
Disable self-update commands in packed Brioche builds (#161)
Update installation script in README (#163)
Update Cargo workspace and resolve a few lints on Darwin (#164)
Update Rust version to 1.84 in Cargo.toml files (#167)
Update tokio-utils features (#170)

setup-brioche action updates

Add AppArmor profile to fix Brioche builds on Ubuntu 24.04 (#2)
- Can also control this behavior explicitly with the new install-apparmor input value

Project update

I've been pretty loose with the structure of the project updates. I decided to try breaking it up into the "status report" with a little more in-depth discusion, and "housekeeping" with bulleted notes, mainly summarizing stuff from GitHub PRs.

Announcing Brioche v0.1.4

Kyle Lacy — Sun, 19 Jan 2025 07:26:00 GMT

Brioche is a new package manager and build tool that makes it easy to mix and match languages and tools for your own development projects.

Today marks the release of Brioche v0.1.4-- over 3 months in the making since our last release, and the fourth version after the public release in July!

New console output

My personal favorite feature, and the one that's most fun to show off, is the new TUI-style console output you see during a build. Compared to the old console output, the new format is colorful, cleaner, and more clearly organized. Plus, it runs at a higher refresh rate, so it feels a lot snappier. And here's what it looks like:

New format for process failures

Nothing's worse than following along with the instructions on building a program from source, only to get a build error despite doing everything the documentation says. Well, getting a build error and not being able to see what the error was might be worse...

Up until now, if a process recipe failed to bake, you'd get an error message helpfully pointing you to two files: stdout.txt and stderr.txt. As you might've guessed, these files contained the stdout and stderr streams of the failed process, respectively.

This definitely helped give some context on what went wrong. But, since stdout and stderr were written separately, if a process interleaved messages on both streams, it might be impossible to figure out when or where an error happened. While Brioche would print a stack trace on failure, it didn't persist the stack trace anywhere, so working backwards from the files stdout.txt / stderr.txt, it could be hard to even find out what failed!

In the latest release, build failures are definitely still painful, but a little less. Now, Brioche records a bunch more details on process failure in a new combined file format:

stdout and stderr get saved in the same file, preserving how output was interleaved
Details about the failed process recipe are recorded, including a full stack trace of your .bri code
We also save the process exit status and a timestamp for each output line for good measure
Oh, and the new format is compressed! I even ended up writing a new Rust crate (zstd-framed) to handle the compression while keeping the format seekable (which I wrote about in the November Project Update)

Anyway, the important part is, when a process fails, the error message will tell you to run a command like this: brioche jobs logs /path/to/events.bin.zst. If you do so, you'll see much more detail about the failed process than what you got before:

$ brioche jobs logs /home/user/.local/share/brioche/process-temp/01JCYC3PN9QHFJDD6R41KRTDMB/events.bin.zst
process stack trace:
- file:///home/user/.local/share/brioche/projects/97beeb18f169ced9073a4f542f4195ef675ca4b1407170a247485c21545987bf/extra/run_bash.bri:42:14
- file:///home/user/example-project/project.bri:4:21

[0.00s] [spawned process with pid 109069, preparation took 0.01s]
[0.01s] uh oh, build failed
[0.01s] [process exited with code 1]

CLI improvements

Like other package managers, several Brioche subcommands now have a --locked flag. This will prevent Brioche from updating your lockfile: if the lockfile is out of date, the command will instead error out. This is a perfect fit for CI/CD pipelines, where you want to make sure your lockfiles are committed to the repo properly.

Also, CLI commands take a new --display flag. With this, you can force Brioche to output plaintext only instead of using a TUI-style view (--display plain), or you can force it to output in TUI mode explicitly (--display console). Additionally, there's a brand new reduced display mode (--display plain-reduced), which is the same as plaintext, except it's less verbose. We now use this in the brioche-packages repo to avoid outputting an absurd amount of text during the GitHub Actions build, and it might be a good choice any time you have an overly-noisy or long-running build.

New recipe: `attach_resources`

This release includes a new attach_resources recipe. This is really a lower level tool that I don't really expect to be used outside the std package, but the goal is to help optimize how large recipes get served from the registry.

Okay, let's get in the weeds a little: file artifacts can have a "resource directory" attached, where the contents of the directory will be "nearby" at runtime. This is a core ingredient for packed executables.

The challenge is that resources are kinda lossy, and moving them into and out of processes can sometimes "lose" the resources if not handled properly. So, back in Brioche v0.1.1, we added the new collect_references recipe, which basically collects all the resources used in a directory and puts them within a subdirectory named brioche-resources.d, then detaches the files from their resources. attach_resources is the inverse[^attach-resources-and-collect-references-naming]: given a directory, it looks for each file's resources by looking under brioche-resources.d, and reattaches them. Together, these can be used to round-trip a recipe into a tar file and back out again, preserving the recipes on either side.

So, circling back to speeding up registry syncs: @pzmarzly opened #147 becuase, when trying to build one of the example projects, the initial registry sync took over 15 minutes! I had the idea to speed up the download of the std.toolchain() recipe by serving it as a single tar file, but to do so, Brioche needed a way to tar up a recipe and untar it while keeping the references in tact. With the new attach_resources recipe, this should be now be pretty easy!

Packed builds of Brioche

@matklad opened #51 following Brioche's public release, asking for a fully static build so it can work on non-glibc-based Linux distros-- and specifically for NixOS.

Not that long ago, I wrote about how Brioche builds portable packages on Linux, so it's a bit embarrassing that Brioche itself wasn't distributed that way...

Well, as of this release, Brioche has its own project.bri and can build itself! The result is fully portable and should work on any x86-64 Linux system, including distributions without glibc installed globally like NixOS and Alpine Linux.

Shortly after the release goes live, I'll work on updating the Brioche install script. For starters, this packed build will be used as a fallback, and the default will still be to install the glibc-based executable. Packed builds don't support automatic updates yet, so there's more work to do.

In the coming release(s), I'm hoping to build on this work: add support for automatic updates, switch to the packed builds as the default, and offer a migration path for existing installations during automatic updates. But this first milestone should make it possible for more people to try out Brioche today!

Linux sandboxing fallback with PRoot

Recently, I worked on upgrading all of our GitHub Actions pipelines from Ubuntu 22.04 to 24.04. I thought this would be frictionless, but I was very wrong...

In Ubuntu 23.10, Ubuntu started restricting unprivileged user namespaces by default, which is the basis for how sandboxing works on Linux. Ubuntu 24.04 loosened the restrictions a little, but the end result was that Brioche didn't work out of the box by default on Ubuntu 24.04.

For our internal GitHub Actions workflows, it was easy enough to use some sysctl commands to allow unprivileged user namespaces. For our public setup-brioche action, we also now try to install an AppArmor profile if needed (brioche-dev/setup-brioche#2), which should help avoid any issues for GitHub Actions workflows in most cases.

But for end users, I still wanted Brioche to work without any extra setup and without needing root to grant extra permissions. So now, Brioche will try to detect if it's able to use namespaces for sandboxing-- the same as before. If that doesn't work, it'll now fallback to sandboxing using namespaces along with PRoot (as introduced by #159). Failing that, it'll return an error. Using PRoot does come with a notable performance impact (see this comment), so it prints a warning (by default) if it needs to be used. You can also manually tweak the config for more control over sandboxing now.

Despite the performance impact, I feel pretty good about falling back to PRoot, so Brioche works without any extra setup on Ubuntu 24.04 (I believe Ubuntu 23.10 won't work, although I haven't tested it). And over time, Brioche will definitely gain more options for sandboxing, in order to balance the tradeoffs between performance, isolation, and host compatibility.

Anyway, Brioche v0.1.4 is live now! If you already have it installed, run brioche self-update and follow the prompts to update to the latest version. Or, if you haven't tried out Brioche yet, check out installation options in the docs.

Oh, and rolling out the new packed builds will take a little extra work post-release, so keep an eye on the docs for additional information shortly.

Questions, help, and feedback are always welcome on Brioche's GitHub, Discord, and Zulip!

[^attach-resources-and-collect-references-naming]: Yes, one of the recipes is named "collect references" and the other is named "attach resources", and these recipes are effectively opposites of each other. I don't like the split in the "references" / "resources" terminology, and I don't think "collect" and "attach" really make it clear these recipes are complimentary. I'd like to try to come up with better names in a future release (open to bike-shedding!)

Brioche Project Update - November 2024

Kyle Lacy — Mon, 18 Nov 2024 07:53:23 GMT

The last month (and change) has been one of rabbit holes!

Package updates

Since the last update, there were 11 new packages:

tcpdump
libpcap (thanks to @jaudiger for laying out the groundwork!)
python
sqlite
wasmtime
asciinema
libxml2
libxslt
moreutils
zx (thanks @asheliahut!)
amber (thanks @asheliahut!)
fd (thanks @asheliahut!)

Python was a big milestone! It was quite fun to puzzle out how package builds look when using Python. You can see asciinema for what a Python-based build looks like (although projects using Poetry/uv/pyenv/etc will probably end up being quite different-- asciinema just uses pure Python and pip)

moreutils proved to be a surprising challenge-- it's simple on the surface but it turned out to be hard specifically due to how docs are built. I needed to figure out what an "XML catalog" was, which I used to create this ~monstrosity~ beautiful inline XML file directly in the package definition. ~~(next up: adding JSX support for defining inline XML files more easily in Brioche?)~~

std updates

There were a few disperate changes in std since last update. Full changes in the changelog.

There was a breaking change around the std.process() function-- the unsafe options have been reorganized to be more composable. The rust package is now taking advantage of this as of brioche-dev/brioche-packages#123, where it passes through all unsafe options directly to the underlying std.process().

There were also two new functions used to create "runnables" (this also included a minor breaking change around process template symbols). The new functions std.withRunnable() and std.addRunnable() provide a nice alternative to the very bare-bones std.withRunnableLink() function, and the heavy-handed std.bashRunnable template function that pulls in Bash as a dependency.

Finally, there were a few minor changes around the toolchain to sand off a few more sharp edges:

Fix typo in $PERL5LIB env var
Fix broken library symlinks
Patch pkg-config files to use relative paths. We'll probably want to provide a general mechanism for this, but for now it's been copy/pasted to a few other packages (see brioche-dev/brioche-packages#135)
Set $BISON_PKGDATADIR to fix Bison
Set some more symlinks and env vars so more autotools builds "just work"

Brioche core updates

Okay, here's where things get exciting (and also a little complicated!)

This cycle, I've been interested in improving developer ergonomics in preparation to start on cross-platform builds, so I wanted to revisit some of the minor pain points in the flow for building and debugging packages before embarking on that journey.

New console output

I was getting pretty tired of the "pretty" output that the CLI currently uses... so I redesigned it!

Here's a side-by-side between the current version in Brioche v0.1.3 and the new one as of brioche-dev/brioche#137:

<div class="flex flex-col gap-x-4 content-between md:flex-row md:relative md:p-8 md:extrawide"> <div class="flex-1"> <a href="https://asciinema.org/a/9W15OfFdkIljSS2tVrIlWKIlX" class="text-center block">Before</a> <script src="https://asciinema.org/a/9W15OfFdkIljSS2tVrIlWKIlX.js" id="asciicast-9W15OfFdkIljSS2tVrIlWKIlX" async="true"></script> </div> <div class="flex-1"> <a href="https://asciinema.org/a/681816" class="text-center block">After</a> <script src="https://asciinema.org/a/681816.js" id="asciicast-681816" async="true"></script> </div> </div>

(You can probably guess why I added the asciinema package recently!)

I find the new output much easier to scan personally. The new output also uses a higher framerate, which makes it "feel" faster-- to my eyes at least.

New process logging

When a build fails in Brioche v0.1.3, you get a very ugly error message:

Error: process failed, view full output from these paths:
- /home/user/.local/share/brioche/process-temp/01JCYBMMGZ5QJD61R5Z0H7WZCQ/stdout.log
- /home/user/.local/share/brioche/process-temp/01JCYBMMGZ5QJD61R5Z0H7WZCQ/stderr.log: process exited with status code exit status: 1

The file paths listed though are pretty helpful! You can see the full stdout/stderr of the failed process (and pro tip: the directories containing those files contain the build's working directory, making it easy to find things like config.log files). Lots of process write to both stdout and stderr, but since they get written to separate files, these logs are lossy, since you lose how the messages get interleaved. In practice, this just makes trawling through failed logs harder than it should be.

As of brioche-dev/brioche#138, the error message is... still pretty ugly, but more useful!

Error: process failed, view full output by runing `brioche jobs logs /home/user/.local/share/brioche/process-temp/01JCYC3PN9QHFJDD6R41KRTDMB/events.bin.zst`: process exited with status code exit status: 1

As hinted by the error message, the separate stdout and stderr files have been replaced with a new mysterious events.bin.zst file, ooh! This is a pretty simple (binary) file format that includes both stdout and stderr, but with lots of fancy bells and whistles:

stdout and stderr are interleaved, but each chunk is tagged to indicate which stream it's from
Each chunk includes a timestamp
There's an event that includes the process exit code at the end (also timestamped)
There's an event at the start with lots of metadata about the build, including a stack trace

If we run the suggested brioche jobs logs command, we get a pretty detailed output:

$ brioche jobs logs /home/user/.local/share/brioche/process-temp/01JCYC3PN9QHFJDD6R41KRTDMB/events.bin.zst
process stack trace:
- file:///home/user/.local/share/brioche/projects/97beeb18f169ced9073a4f542f4195ef675ca4b1407170a247485c21545987bf/extra/run_bash.bri:42:14
- file:///home/user/example-project/project.bri:4:21

[0.00s] [spawned process with pid 109069, preparation took 0.01s]
[0.01s] uh oh, build failed
[0.01s] [process exited with code 1]

There's lots of untapped potential with the new process event format, but I'm pretty happy with this as a first pass! brioche jobs is an entirely new subcommand, and I think it'll be possible to add lots of handy new utilities for diving in when a build fails.

Compression woes

Oh, also, you may have noticed the filename above was events.bin.zst, implying zstd (Zstandard) compression. This is obviously good! If you have a long-running and verbose build that dumps a ton of output to stdout and stderr, compression can save a lot of disk space.

...but, I wanted to have my cake and eat it too. The new brioche jobs logs command also supports the flag --reverse, allowing you to print events starting from the end. For a super long build log, it'd be annoying to have to wait for the entire file to decompress just to show the last 100 or so log lines!

That's when I learned about the zstd seekable format, which basically chunks a zstd stream into multiple frames, and allows efficient seeking by only needing to decompress a particular frame: in the case of reading only a few lines starting from the end, you might only need to read the last frame for example (currently ~1MB of data). It's not a magic bullet though: dividing a stream into multiple frames does harm the compression ratio. But, in this case, I thought the trade-off was worth it.

Unfortunately, the zstd seekable format isn't yet supported in either the zstd or the async-compression Rust crates (it should be decodable by any compliant decoder, but it wouldn't allow seeking and it wouldn't be possible to encode the seekable format without explicit support). So, I ended up writing the zstd-framed crate to handle the compression and decompression with the seekable format. It should also gracefully handle edge cases, like if the process dies after having written a partial file, where it'll allow decoding even if the file wasn't closed properly during encoding. Good if, I dunno, your computer shuts down during a long-running build!

Anyway, the choice to use compression ended up stretching out development of the new process event format by several weeks, but I'm really happy with the end result!

Extension updates

@asheliahut just published v0.2.0 of the brioche-vscode extension, and added a new GitHub Actions workflow to simplify publishing of future versions! The new release doesn't have any feature changes, but bumps the versions of various dependencies (and additionally now requires a newer version of VS Code, hence why it was a bump to v0.2.0).

Also, we published the VS Code extension in the Open VSX Registry (although at the time of writing, our namespace is not yet verified, track this issue for progress). This means you can now use Brioche in other VS Code-based editors, such as VSCodium!

Infrastructure updates

In our homelab, we have an M1 Mac mini that has gone largely unused since we got it. I finally decided to provision it with Asahi Linux, which means I now have a powerful machine for doing native ARM64 Linux builds locally! This represents the first tangible work towards getting Brioche supporting cross-platform builds-- first up will be supporting native ARM64 (aarch64) Linux builds, followed by supporting ARM64-to-x86_64 and x86_64-to-ARM64 cross builds. The Asahi-ified Mac mini will be how I'll work on builds, and it'll eventually become a GitHub Actions runner for the brioche-packages repo.

Coming soon

Here's my shortlist of features that I'm semi-planning to work on soon:

CMake!
More work on improving the flow when debugging failed builds, including:
- Different build ID format (we currently use ULID, which turns out to be counter-productive as it takes a lot of characters to disambiguate failed builds)
- Command to show failed builds without using a full path, e.g. brioche jobs logs $BUILD_ID
- Record failed builds in Brioche database, so we can easily clean old failed builds and to allow e.g. brioche jobs logs last to show the last failed build
- Allow for starting a shell within a failed build to allow for debugging
Look for low-hanging fruit to speed up build times. brioche-dev/brioche#78 is the first candidate here
Make some progress on ARM64 Linux toolchain builds

Brioche Project Update - October 2024

Kyle Lacy — Sat, 05 Oct 2024 05:17:40 GMT

Well, it's been a busy couple of months! Since the last update in August, there have been two releases, some big changes in std, and more!

Blog posts

If you hadn't seen it yet, I published a post a couple of weeks ago called "Portable, dynamically linked packages on Linux"

It's a pretty in-depth behind the scenes look at how packed executables work in Brioche. I wanted to make it general enough that folks interested in packaging or distributing software more generally for Linux get some value out of it-- which is also why it's pretty technical and long

Brioche releases

Brioche v0.1.2 was released last week, with Brioche v0.1.3 close behind. v0.1.3 was pretty small release with a few bugfixes (most notably a fix for an LSP regression from v0.1.2), but the real meat and potatoes were in v0.1.2.

The release notes cover everything that changed, and there's a lot of exciting stuff! Especially the CLI changes that allow for running across multiple projects at once and the fix for the Brioche auto-updater (finally!) which will make future updates go more smoothly.

My favorite new features though are locked downloads and locked git refs.

Locked downloads

Up to this point, if you were adding a package in Brioche, you'd get something that looks like this package (dust). In the middle, you'd see something like this:

const source = std
  .download({
    url: `https://github.com/bootandy/dust/archive/refs/tags/v${project.version}.tar.gz`,
    hash: std.sha256Hash(
      "98cae3e4b32514e51fcc1ed07fdbe6929d4b80942925348cc6e57b308d9c4cb0",
      // ^^^ tedious, boring busywork ^^^
    ),
  })
  .unarchive("tar", "gzip")
  .peel();

To download the source code to build a project, we need its hash. So, how do we even get the hash for a URL? Well, before v0.1.2, there were two ways:

Copy the URL and paste it into a shell one-liner using curl | sha256sum, then copy the hash into the source code
Add an invalid hash, run the build, then copy the hash from the error message into the source code

In either case, manually copy/pasting hashes is busywork, which is the kind of thing we want to automate away. Which brings us to Brioche.download! Here's a snippet that's roughly the same as before:

const source = Brioche.download(
  `https://github.com/bootandy/dust/archive/refs/tags/v${project.version}.tar.gz`,
  // No more hash!
)
  .unarchive("tar", "gzip")
  .peel();

The "trick" is that, when you call brioche build on this project, Brioche scans for all occurrences of Brioche.download in the code, downloads each one, then saves each hash in the lockfile. This has all the same benefits as having the hash directly in the source code, except you don't need to manually figure out the hash yourself (plus Brioche only downloads the URL once, instead of making Brioche download it a second time after you manually hashed it)

Since Brioche needs to scan the script to find all URLs, there are some limitations on how you can construct the URL. Namely, the URL needs to be hard-coded as a string literal, or as a template that uses the project metadata (like project.version above). In the future, it should be possible to make this support a little more general, but the important thing is that Brioche needs to be able to figure out the URL statically.

Locked git refs

In the same vein, packages that clone from git used to need to reference the git hash directly in the source code.

If you squint, this is the exact same problem as with URL download hashes! So, Brioche can also apply the same trick: instead of referencing the git hash directly, you can call Brioche.gitRef({ repository, ref }), and Brioche will record the commit hash the currently corresponds with that git ref in the lockfile, such as in this snippet:

// Previously
// const source = gitCheckout({
//   repository: "https://github.com/extrawurst/gitui.git",
//   commit: "95e1d4d4324bf1eab34f8100afc7f3ae7e435252",
//   // Commit grabbed manually by finding the right tag in the repo
// });

// Now
const source = gitCheckout(
  Brioche.gitRef({
    repository: "https://github.com/extrawurst/gitui.git",
    ref: `v${project.version}`,
    // The commit hash for the version tag gets saved in the lockfile
  }),
);

Package updates

Since the last update, the only new package is terraform (thanks @asheliahut!), but quite a few packages were also updated across the board (kudos to @jaudiger and @asheliahut!)

The rust package was updated from v1.79 to v1.81, which did see a few packages break along the way. go was also updated to support more complex module layouts, namely to unblock the terraform package (under the hood, the go package now properly grabs all the go.mod and go.sum files before trying to download dependencies).

std updates

A lot of work went into std updates. The changelog has a full breakdown.

The std.glob(), Brioche.download(), and Brioche.gitRef() functions were added to support new features from Brioche v0.1.2. The std.setEnv() function had a breaking change that also enables support for more ways to set env vars (which also requires Brioche v0.1.2). Also, the new std.semverMatches() function allows for testing semver constraints, which is used by std itself to ensure you can't accidentally use new features on older versions of Brioche

std.toolchain() now sets a bunch of env vars that autotools use, so autotools builds should now work without needing to configure lots of env vars manually!

Also, both std.toolchain() and std.tools() are now "rewrapped", which ensures the final built version of glibc is used internally by all the different binaries. Previously, if you saved std.toolchain() as an output, you'd see that there were multiple different copies of glibc present in the output, due to different binaries being built at different stages and ending up with different versions of glibc! Rewrapping ensures everything is using a single copy of glibc for consistency (which also has the nice benefit that the output is now smaller too!)

Brioche core updates

Shortly after the v0.1.3 update, I fixed another LSP issue (#134). This one is certainly annoying as well, but I didn't feel that it was critical enough to cut another release immediately.

I also added a new --locked flag for the quartet of brioche build, brioche run, brioche check and brioche install (#133). Basically, it just ensures that the brioche.lock lockfile is up-to-date. This is useful for CI/CD (and was also sorely needed for the brioche-packages repo, especially now that downloads and git refs are being recorded in the lockfile)

Infrastructure updates

I've been doing my best running Brioche on a shoestring budget, and so far things have been going really well! Besides an outage due to an expired TLS cert, all of the infrastructure behind Brioche has been ticking along smoothly.

...well, with one exception. Builds are obviously super important for Brioche, and making it so updates to the brioche-packages repo get built and published in a timely manner is a core part of the project!

That entire pipeline is handled with a pretty boring GitHub Actions workflow, which first validates all the packages, builds and syncs the recipes to the registry, then publishes the project files to the recipe. We use GitHub Action's generous free runners where possible. But, for the actual slow, CPU-intensive builds, those get run on my and @asheliahut's homelab.

In early September, I decided to upgrade the Linux kernel on the hypervisor, plus all of the OS packages. The upgrade appeared to go fine. It was not fine.

To cut a long story short, the server would crash after about 24 hours of uptime (and, very annoyingly, would bring down our entire home network as well). We fought for weeks to triage the issue, from using older kernel releases, to trying out different kernel flags, to swapping the NIC in the server (which at least made it so the server crashes wouldn't bring down the network), to updating the BIOS. The last thing we did was disabling CPU C-states in the BIOS, and so far that seems to be holding strong

Coming soon

As with last time, here's a short wishlist of things I hope to make progress on soon

CMake and Python are still work in progress and haven't been touched since the last update. These will be pretty high priority!
There's still some work to do before merging brioche-dev/brioche-packages#80, but that should unlock more NPM packages as well
Brioche.download() and Brioche.gitRef() make it so many package updates are as simple as bumping a version number in project.bri. The next logical step would be to support package auto-updates, where bumping the version number itself is automated
Still no forward progress on cross-platform support. The immediate next step is to set up Asahi Linux on our homelab Mac Mini, which will make it possible to start tinkering with ARM64 Linux builds

Portable, dynamically linked packages on Linux

Kyle Lacy — Sun, 22 Sep 2024 00:46:12 GMT

Hey, want to see a magic trick? If you have Brioche installed, you can run this command:

brioche build curl -o ./output

...to get a copy of curl! Like you can then run ./output/bin/curl --version and it works!

*crickets*

Okay, not impressed? Well, like all of the best magicians, I will now explain why this is a good trick:

The output directory contains every file we need to run curl
The underlying curl binary is dynamically linked against glibc
There are no containers up my sleeve[^the-container-up-my-sleeve-kinda]

Which means we can just put this directory anywhere on the file system and it'll run... including in a container image. Presto, behold a Dockerfile!

FROM scratch
COPY output /output
ENTRYPOINT [ "/output/bin/curl" ]

Save it as Dockerfile (or Containerfile), build it with Podman / Docker / etc, and it still runs, even in the completely barren scratch environment:

$ podman build . -t magic-trick
$ podman run --rm magic-trick --version
curl 8.8.0 (x86_64-pc-linux-gnu) libcurl/8.8.0 OpenSSL/3.3.1 zlib/1.2.13 zstd/1.5.5
Release-Date: 2024-05-22
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS HSTS HTTPS-proxy IPv6 Largefile libz NTLM SSL threadsafe TLS-SRP UnixSockets zstd

Not just that, but the container we get out is actually pretty small[^small-container], since it's distroless!

so? who cares??

Well, I care, for one! I'm a weirdo trying to build a package manager. And building a package manager means... putting packages on people's computers, so you need to figure out where to put the package on the computer. Being able to put the package anywhere makes it easier!

Let's look at those those 3 facts about this version of curl from before:

The output directory contains every file we need to run curl
The underlying curl binary is dynamically linked against glibc
There are no containers here

Depending on how much you know about dynamic linking, it should seem impossible for all 3 of these things to be true. Let's ignore the how for now and talk about why I made it work this way:

You shouldn't need root permissions to install Brioche or to install new packages. Package managers usually install packages into a global, fixed directory (/nix for Nix, /home/linuxbrew/.linuxbrew for Homebrew), but that's just not an option if we want everything to work rootlessly. So we need to make everything portable so it works from anywhere on the filesystem
Brioche should be able to package software that uses glibc. The obvious option would be to just use musl libc for everything, which makes it easier to do fully static builds. The problem is musl isn't a drop-in replacement for glibc: as an example, musl's DNS resolution trips people up. I really wanted to support packages that wanted or needed to use glibc (Update: ifreund pointed out that musl now supports TCP DNS fallback! There are still differences from glibc, so it still won't work for programs that depend on glibc-specific behavior)
Brioche packages shouldn't use containers at runtime. snap, Flatpak, and AppImage all already use containers for packages, but I wanted to kick Brioche off with a focus on developer-facing CLI tools-- things that would be more awkward to try and use from a containerized environment. Since containers have their own view of the filesystem, using tools like find, du, etc. might not work like you expect! Plus, containers are a form of sandboxing and sandboxing is hard and I didn't want "solving sandboxing" to be a yak I needed to shave as a precursor[^sandboxing-future]

So hopefully you can see how the little party trick with curl came to be: to tick all the boxes for Brioche, I had to make that trick work. Every package in Brioche works this way, too. You can just put any package into a directory somewhere on your filesystem, and it'll run entirely self-contained even for a completely bare-bones Linux setup

Let's do it ourselves

So let's make our own portable package that checks all the same boxes that we get from Brioche. Let's start with a little Rust program to use as our candidate to package. First, run cargo new lil-demo, then cd lil-demo, then put this in src/main.rs:

// lil-demo/src/main.rs
fn main() {
    let location = std::env::current_exe().unwrap();
    println!("Hello from {}", location.display());
}

When a program calls std::env::current_exe(), it gets the path to itself (yes, using this particular function is important, we'll come back to it later). If you run it with cargo run, you'll see some output like this:

Hello from /your/path/to/lil-demo/target/debug/lil-demo

Wait, actually, I should mention... I'm in an Ubuntu Linux environment, so by default Cargo dynamically links my lil-demo program against glibc. This is true for most Linux distros, but if yours is different, you might not be able to follow along.

Finding all the dependencies

So we want to package our lil-demo up just like we had curl at the start. Remember, that means we need to put all of its dependencies into a self-contained directory. So, what does it depend on? Well, we can answer that using ldd:

$ ldd target/debug/lil-demo
linux-vdso.so.1 (0x00007ffd6c14a000)
libgcc_s.so.1 => /home/linuxbrew/.linuxbrew/lib/libgcc_s.so.1 (0x00007fa6b1856000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa6b1620000)
/lib64/ld-linux-x86-64.so.2 (0x00007fa6b18d2000)

Each line is a dynamic library our lil-demo depends on, which shows the name of the library that the program asked for, then the location it actually loaded it from (the hex is the address it got loaded to but we don't care about that):

linux-vdso.so.1: Okay, we start off with a free square! The "vDSO" is a special dynamic library provided by the Linux kernel itself, so it's not our responsibility to provide it
libgcc_s.so.1: Provided by gcc, and I think Rust links against it for stack traces or something. It's a very common standard system library (don't ask why mine comes from Homebrew)
libc.so.6: The main C library, glibc in my case. This is another system library
/lib64/ld-linux-x86-64.so.2: Okay, this one's pretty complicated. This is the dynamic linker. When you try to run lil-demo, the Linux kernel actually calls this ld-linux-x86-64.so.2 program instead (because its dynamically linked). It's job is to first find all the dynamic libraries lil-demo needs, load them into memory, then really run lil-demo

As you might've guessed, the name of that last one in particular varies per platform, so I'm going to use the generic name of ld-linux.so when I talk about it instead.

Setting up the portable directory

So in total, we need the lil-demo program itself, the 2 dynamic libraries it links against (not counting the "vDSO" one Linux gives us for free), and ld-linux.so, a.k.a. the dynamic linker. How do we combine these ingredients to make something portable?

Let's start with a little boilerplate first:

Make a new directory: mkdir lil-demo-portable
Copy lil-demo into it: cp target/debug/lil-demo lil-demo-portable/
Copy the 2 dynamic libraries and ld-linux.so into it too. These will be whatever paths you got from ldd. In other words, something like cp /path/to/libgcc_s.so.1 /path/to/libc.so.6 /lib64/ld-linux-x86-64.so.2 ./lil-demo-portable/
Create a new shell script at lil-demo-portable/run.sh with the following contents:

# lil-demo/lil-demo-portable/run.sh
#!/usr/bin/env bash

portable_dir=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &> /dev/null && pwd)

exec "$portable_dir"/lil-demo

Then, mark the script as executable and run it:

$ chmod +x lil-demo-portable/run.sh
$ ./lil-demo-portable/run.sh
Hello from /your/path/to/lil-demo/lil-demo-portable/lil-demo

Okay, so the script is pretty straightforward: first, it gets the directory containing the shell script itself (meaning lil-demo-portable/). Then, it uses exec to run the original lil-demo binary that we copied in[^exec]. Unsurprisingly, we see the path of this copied binary!

Okay, we've finally set the stage to actually make lil-demo into a lil' portable demo

Wrapping the binary

When lil-demo runs, we want it to use the copied libraries from within lil-demo-portable. The easiest way is to set the env var $LD_LIBRARY_PATH. When it's set, ld-linux.so uses it as a place to find dynamic libraries, which is exactly what we want! Here's an updated run.sh:

# lil-demo/lil-demo-portable/run.sh
#!/usr/bin/env bash

portable_dir=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &> /dev/null && pwd)
export LD_LIBRARY_PATH="$portable_dir"

exec "$portable_dir"/lil-demo

...basically, we just set $LD_LIBRARY_PATH to the directory containing the shell script itself. If we run run.sh again, the result is the same. But now, all the dynamic libraries get read directly from the lil-demo-portable-dir! Neat!

(If you want to see for yourself that libraries are now getting resolved correctly, change the last line of run.sh to ldd "$portable_dir"/lil-demo)

Wrapping the dynamic linker, too

So we're running lil-demo from our portable directory, and we're even loading all the dynamic libraries we need from it to thanks to $LD_LIBRARY_PATH. That just leaves ld-linux.so. To recap, it's not a dynamic library itself, but it's the thing responsible for loading all the dynamic libraries.

When we run lil-demo (either directly or via run.sh), the Linux kernel doesn't actually run lil-demo directly. Instead, it checks the header of the file-- specifically, the PT_INTERP element from the program header of the ELF file-- sees that it points to /lib64/ld-linux-x86-64.so.2, and runs that instead.

In other words, this:

$ ./lil-demo-portable/lil-demo

...effectively becomes this:

$ /lib64/ld-linux-x86-64.so.2 ./lil-demo-portable/lil-demo

But we don't want it to become that! We want it to call the ld-linux.so that's under lil-demo-portable!

...so instead, we can just call ld-linux.so directly ourselves. Here's an updated run.sh:

# lil-demo/lil-demo-portable/run.sh
#!/usr/bin/env bash

portable_dir=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &> /dev/null && pwd)
export LD_LIBRARY_PATH="$portable_dir"

exec "$portable_dir"/ld-linux-x86-64.so.2 "$portable_dir"/lil-demo

We just changed the last line to execute ld-linux.so with lil-demo as an argument. And, we can run it and see it works just like before, just using our copy of ld-linux.so from the portable directory instead!

$ ./lil-demo-portable/run.sh
Hello from /your/path/to/lil-demo/lil-demo-portable/ld-linux-x86-64.so.2

...

...wait, something feels... off somehow? Do you feel it? Let's zoom and enhance:

-Hello from /your/path/to/lil-demo/lil-demo-portable/lil-demo
+Hello from /your/path/to/lil-demo/lil-demo-portable/ld-linux-x86-64.so.2

...the path changed from lil-demo to ld-linux.so???

Checkov's path

Okay, remember a million words ago when I said std::env::current_exe() was going to be important later? Well later is now, and now we need to talk about it

How does lil-demo know where it is? ~~it knows where it is because it knows where it isn't~~ it knows where it is because it reads the path of the symlink /proc/self/exe[^proc-absolute-path], which is a special symlink that Linux sets up that always refers to the current executable.

For a demo that's kinda trippy, try this:

$ readlink /proc/self/exe
/usr/bin/readlink

When readlink reads the /proc/self/exe symlink, then by definition, it reads its own path, so it prints the path to readlink itself!

And of course, that's also exactly how std::env::current_exe() is implemented

So /proc/self/exe is some Weird Magic that gets controlled by the Linux kernel. It basically points to whatever file was passed to the underlying execve() syscall. Our shell script is now executing ld-linux.so instead of lil-demo, so that's what /proc/self/exe resolves to. And ld-linux.so then sets up and runs our program directly (i.e. no calls to execve()), so /proc/self/exe is "stuck" with ld-linux.so until the process ends or until it uses the execve() syscall itself

Wait but why do we care so much about /proc/self/exe?

You'd be surprised how many programs will break if they don't get the right value for /proc/self/exe! The Rust compiler itself reads /proc/self/exe to resolve the path to the Rust standard library. Off the top of my head, I believe both Node.js and gcc both read /proc/self/exe for resolving resources. It's just a pretty common thing that programs built for Linux end up depending on.

Fixing `/proc/self/exe`

Okay, so we know /proc/self/exe is important, and we know we're now "breaking" it, in a sense. Let's step through how we got here

At first, we started with just calling exec lil-demo. The flow basically worked like this:

The execution of lil-demo was implicitly calling ld-linux.so under the hood via the PT_INTERP field from the ELF header.

In our latest version, we changed it to explicitly call exec ld-linux.so, so we could use the dynamic linker from our portable bundle:

As we discussed before, /proc/self/exe is determined by the last call to execve(). And we can see the path ld-linux.so → lil-demo does not use execve(). So if we want to un-break /proc/self/exe, we need to change our execve() calls! ...somehow

Maybe we could somehow change ld-linux.so itself? Like, if we could make it so the path ld-linux.so → lil-demo uses execve(), that could fix our problem? But remember, any time anything uses execve() to call lil-demo, the Linux kernel itself will do the little ld-linux.so → lil-demo dance for us, so we can't do that...

The actual fix we're going for is to change the run.sh → ld-linux.so path. Specifically, we're going to explicitly call ld-linux.so without using execve():

So how do we execute ld-linux.so without using execve()? How can we execute something without using the special "execute something" syscall??

The keyword we're looking for is called "userland exec". It's a technique for executing a program without involving the kernel. The core idea is not too complicated: Linux programs use the ELF file format, which contains a description the exact memory layout a program expects when it runs. So we just parse the program as an ELF file, read the memory layout, then directly jump to the program's start address.

<a href="https://fasterthanli.me/" class="no-underline hover:underline">fasterthanlime</a> has a blog post describing exactly how to do that (part of a series of posts on building an executable packer), so be sure to give that a read if you want the fine details!

But we're gonna take the lazy path. and uhh... this is also not something you could do from a shell script[^userland-exec-shell-script]. So we're gonna handle all this "userland exec" stuff in Rust. That'll also let us ~~cheat~~ leverage the breadth of Rust's package ecosystem by using the userland-execve crate, which will handle the raw assembly and pointer manipulation for us. We're not gonna get our hands too dirty today.

Rewriting it in Rust

Let's start first by porting our current shell script to Rust as-is. Let's set up a second crate next to our lil-demo crate (so if you're in the lil-demo directory still, run cd ..). Run cargo new run to create the crate, then put this in run/src/main.rs:

// run/src/main.rs
use std::os::unix::process::CommandExt as _;

fn main() {
    let current_exe = std::env::current_exe().unwrap();
    let portable_dir = current_exe.parent().unwrap();

    let ld_linux_so = portable_dir.join("ld-linux-x86-64.so.2");
    let lil_demo = portable_dir.join("lil-demo");

    let error = std::process::Command::new(ld_linux_so)
        .arg(lil_demo)
        .env("LD_LIBRARY_PATH", &portable_dir)
        .exec();
    panic!("failed to exec: {error}");
}

A few notes on this version:

It uses std::env::current_exe() to find the "portable dir" path (equivalent to what we did in Bash)
It builds a command to call ld-linux.so, with the path to lil-demo as an argument
It sets $LD_LIBRARY_PATH to the portable dir for the command
It uses the (Unix-only) .exec() method, just like how we used exec in Bash. It's a little unintuitive, but this method should never return (it only returns if there was an error, which is why there's a panic!() right after)

While in the run directory, use these commands to build it and put it in our lil-demo-portable directory (which is where run expects to be):

$ cargo build
$ cp target/debug/run ../lil-demo/lil-demo-portable/

Alright, now if we run our new run program, we should see the same output as when we ran run.sh:

$ ../lil-demo/lil-demo-portable/run
Hello from /your/path/to/lil-demo/lil-demo-portable/ld-linux-x86-64.so.2

Userland or bust

We're ready to swap over to userland-execve. Add it to your dependencies by running cargo add userland-execve, then update the code to use it:

// run/src/main.rs
use std::ffi::CString;

fn main() {
    let current_exe = std::env::current_exe().unwrap();
    let portable_dir = current_exe.parent().unwrap();

    let ld_linux_so = portable_dir.join("ld-linux-x86-64.so.2");
    let lil_demo = portable_dir.join("lil-demo");

    let ld_linux_so_cstr = CString::new(ld_linux_so.to_str().unwrap()).unwrap();
    let lil_demo_cstr = CString::new(lil_demo.to_str().unwrap()).unwrap();
    let ld_library_path_env = CString::new(format!(
        "LD_LIBRARY_PATH={}",
        portable_dir.to_str().unwrap()
    ))
    .unwrap();

    userland_execve::exec(
        &ld_linux_so,
        &[&ld_linux_so_cstr, &lil_demo_cstr],
        &[ld_library_path_env],
    );
}

So pretty similar structurally to the last one, except it uses userland_execve::exec to run now-- the first argument is the file to execute, the second are the args[^userland-args], and the third are the env vars[^userland-env-vars]

Rebuild the run executable again, copy it over, and watch the magic unfold:

$ cargo build
$ cp target/debug/run ../lil-demo/lil-demo-portable/
$ ../lil-demo/lil-demo-portable/run
Hello from /your/path/to/lil-demo/lil-demo-portable/run

It's now a fully self-contained, portable Linux bundle! If you put this lil-demo-portable directory on another Linux machine (of the same architecture), calling run will run the same even if it doesn't have glibc or any other libraries installed globally, or even if the dynamic linker is missing

Then, when run gets called, it'll actually run lil-demo, which will think it's own path is run

wait but isn't this still wrong? don't we want it to think it's lil-demo? not run??

Ahh, well it's time to come clean...

Revealing the secret

Okay, here was the little curl example from a billion miles up the page[^billion-miles]:

$ ./output/bin/curl --version

There was a little sleight of hand earlier... ./output/bin/curl is not actually curl at all. It's actually a little substitute program, equivalent to the run program from above! The real curl is somewhere under ./output/brioche-resources.d with some unholy hash as part of its filename.

The important thing is that, from an outside perspective, ./output/bin/curl quacks like curl, waddles like curl, and eats bread like curl.

But won't this still break for programs that care about /proc/self/exe?

Nope! Let's take Rust as an example. If you install Rust through Rustup, it'll create a directory structure like this somewhere under ~/.rustup:

bin/rustc
lib/librustc_driver.xxxxxx.so
lib/libstd-xxxxxx.so
...

Rust uses /proc/self/exe to find its current folder, then grabs libraries from ../lib. So if we replace bin/rustc with a wrapper program like run, Rust will read /proc/self/exe and still think it's at the path bin/rustc. So it'll resolve ../lib to the right directory-- no matter where the real Rust binary lives-- because it still thinks it's at bin/rustc.

It also works for programs that execute themselves, since our little wrapper program runs exactly like the original program. The only thing that could break is if a program tries to read data from itself by opening its own path as a file, but that would obviously be very silly[^reading-executables]

Closing thoughts

So that's a peak behind the scenes for how "packed executables" work in Brioche on Linux. Hopefully you've walked away with the impression that there isn't too much dark magic going on (or maybe you've now seen horrors previously beyond your comprehension)

I think this is just a really cool technique, and I'd love to see it get adopted across other package managers or in other places where Linux executables get distributed! For Brioche, it means I can set up a fresh Brioche installation in a few seconds and start installing packages right away, without needing root permissions. It means that I can just run brioche build -o output ... to get a bundle, then just scp it to some remote Linux machine or send it to someone, even if Brioche isn't installed on the other side. It means the same bundle can be used both inside and outside a Docker container. It means I can just use glibc or whatever dynamically linked libraries I want, and not have to fiddle around with toolchains to make a fully-static build[^static-builds]

You might also wonder: what does making a nice, portable package look like if you use Brioche directly? Well, let's see what the config would look like to build a portable version of our lil-demo project:

// lil-demo/project.bri
import { cargoBuild } from "rust";

export default function () {
  return cargoBuild({
    source: Brioche.glob("src", "Cargo.*"),
  });
}

...yep, that's it! No need to set up any wrappers explicitly, it's all handled automatically. All the build tools within Brioche are set up to add all these little wrapper binaries automatically out-of-the-box, so all your builds will work fully portably by default. Just like magic.

[^the-container-up-my-sleeve-kinda]: Okay, technically, brioche build -r curl should basically always be a cache hit, but if it isn't for some reason and it has to build from source, the build itself would run in a container. The output doesn't use any containers though

[^small-container]: From commit 97cfcce, this example container came out to a size of 27 MB, which is pretty small (and glibc is 12 MB of that)! The official curl container image is still smaller at the time of writing (21 MB uncompressed) and that also includes both CA certificates and Busybox, but the big difference is that it uses musl instead of glibc. So getting a glibc-based curl container down to 27 MB is something I'm still proud of overall! ...but in the real world, you'd at least want to add CA certs to this image

[^sandboxing-future]: There is build-time sandboxing, but once a package gets built, the result doesn't itself run in a sandbox. But, it'd be possible to add a function in Brioche that "sandbox-ifies" another package, e.g. by using Bubblewrap. This would make sandboxing a composable piece, rather than being a core part of Brioche's design

[^exec]: Running exec ./some-program is almost the same as directly running ./some-program. The former replaces the current process with the new program to execute by directly using the execve() syscall, where the latter runs it as a subprocess. If you haven't used exec before, your homework assignment is to understand why it's the right tool for our lil-demo wrapper

[^proc-absolute-path]: Bonus points if you noticed that /proc/self/exec is an absolute path, the exact thing we're trying to get rid of! Well, Linux is very Unix-y, meaning "everything is a file", where "everything" includes several core APIs. These are implemented via special virtual filesystems, which get mounted at /proc, /sys, and /dev. Because they're so critical, you really can't avoid them for some tasks... but that also means even extremely bare-bones environments have them mounted (e.g. Docker's scratch container)

[^userland-exec-shell-script]: Super bonus points to the first person to write a userland exec implementation in pure POSIX shell

[^userland-args]: You may have noticed that we originally only passed one arg, but passed two when using userland exec. This is because the userland-execve crate expects you to pass argv0 explicitly, whereas std::process::Comamnd passes it implicitly by default.

[^userland-env-vars]: Note that the example code will clear every env var except for $LD_LIBRARY_PATH. If you want to inherit the rest of the env vars like std::process::Command does, you'll need to manually iterate over std::env::vars_os() and pass each one explicitly.

[^billion-miles]: 1,609,344,000,000 km

[^reading-executables]: Yep... horrifyingly, I have seen at least one package that directly reads an executable and looks for a specific byte pattern. I actually can't remember which package it was... but luckily it was only part of a test suite if I remember right

[^static-builds]: Don't get me wrong, I love a good static build! It's just that setting up the build tooling and getting the right libraries for it can be a pain

Brioche Project Update - August 2024

Kyle Lacy — Fri, 02 Aug 2024 03:50:42 GMT

So, it's been about two months since the initial public release of Brioche, so I thought it was about time to share what's been going on with the project! I quite like the newsletter-style updates that Rust and Dolphin put out regularly, so I think I'll start doing something similar for Brioche (expect ~monthly updates)

Brioche releases

Brioche v0.1.1 was released in early June, which helped smooth out some of the kinks from v0.1.0. The changelog includes all the details, but to give a few highlights:

Speed up registry syncing drastically and tweak registry timeouts
Update the JS runtime to report the current version of Brioche. This will allow std to detect the version before using a new recipe type (either returning an error or using a fallback implementation)
Add a new collect_references recipe type for use in the std package

I even included a subcommand to update Brioche itself, but uhhh... it's broken, so for now, you'll need to follow the installation instructions again to upgrade. Also unfortunately, I didn't catch this issue before v0.1.1 was released, so it's still broken and the next release will also require a manual upgrade... oops

Package updates

During the initial release, there were only 10 packages available: ca_certificates, curl, jq, miniserve, nodejs, rust, plus the std package and a few miscellaneous ones (hello_world, typer, smol_toml)

Now, there are 38, almost 4x as many as during the original launch! Here's what's available now:

alsa_lib
bat
broot
carapace
dust
eza
git
gitui
go
joshuto
jujutsu
jwt_cli
k9s
lurk
oha
oniguruma
openssl (split out from std)
opentofu
pcre2
pv
ripgrep
ruff
tcsh
tokei
xplr
xsv
zoxide

Massive kudos for these packages go to @jaudiger and @asheliahut, who opened 31 and 7 Pull Requests, respectively!

There were also several breaking changes made to Rust, NodeJS, and Go, to make building packages more consistent. See the PRs brioche-packages#65 and brioche-packages#67 for an overview of the changes.

std updates

The std package has also seen quite a few changes. Note that it's still pretty volatile, so keep an eye on the changelog to see how it evolves.

Since the initial release, there have been quite a few changes! Here's the full list of changes since the initial release, but here are the highlights:

Remove Python and OpenSSL from std.toolchain(). OpenSSL was added as its own package (openssl), and Python will join it soon!
Overhaul recipe casting. See brioche-packages#25 for more context
Replace the std.auotwrap() function with std.autopack(). Migrating to the new function should be fairly straightforward, and the new function provides a lot more options
Add std.BRIOCHE_VERSION to check which version of Brioche is being used. This will mainly be used by std itself for feature detection
Optimize container sizes made with std.ociContainerImage(). When using Brioche >=0.1.1, container images can now be much smaller!

Brioche core updates

I was hoping to have a v0.1.2 release ready to go before publishing the first project update, but alas, it's not ready to go quite yet...

...but that also means there are a few goodies in the main branch that you can look forward to for the next release!

Updates to several subcommands (fmt, check, publish, install) so they can take more than one package at a time. When working with multiple Brioche projects (say, if you're contributing a new package), you can now save time by passing the -p or -r flag more than once! Once again, huge thanks to @jaudiger for contributing these changes!

Add support for "locked downloads" with Brioche.download expressions. Normally when downloading something with Brioche, you need to provide the download hash explicitly in code. This has long led to an awkward dance where you need to get the URL, download it yourself first, get the hash, then paste it in the code. Well, as long as the URL is static in your source code, you'll no longer need to do this! Just change from this:

// Old way, need to specify the hash manually
const source = std
  .download({
    url: "https://github.com/jqlang/jq/releases/download/jq-1.7.1/jq-1.7.1.tar.gz",
    hash: std.sha256Hash(
      "478c9ca129fd2e3443fe27314b455e211e0d8c60bc8ff7df703873deeee580c2",
    ),
  })
  .unarchive("tar", "gzip")
  .peel();

...to this:

// New way, hash gets saved to the lockfile
const source = Brioche.download(
  "https://github.com/jqlang/jq/releases/download/jq-1.7.1/jq-1.7.1.tar.gz",
)
  .unarchive("tar", "gzip")
  .peel();

...and the URL will automatically be downloaded and recorded in the lockfile! Note that this will require an update to the std package too, which is still in draft.

Registry updates

Right after release, @matklad opened a Zulip discussion to ask why installing hello_world took so long (and then also contributed brioche#54 to help stop the bleeding).

After a lot of back-and-forth with testing, I got some general fixes in place that went out with Brioche v0.1.1, but it still wasn't a great experience. After a lot of investigation, @asheliahut and I ended up doing a major change to the registry.

To cut a long story short, the registry runs on ephemeral Fly.io machines, and I originally used LiteFS for the database (basically, SQLite with replication). LiteFS ended up causing a lot of latency and reliability issues, so we ended up migrating to a Postgres database hosted by Neon (which, so far, has been an amazing database host!)

Now, the registry should respond much faster, even when a Fly.io machine has to do a cold start!

Coming soon

To close this out, I wanted to share a shortlist of things currently in progress and things I'd like to make progress on in the near future! But... consider this more of a "wishlist" than a "roadmap"

@jaudiger opened brioche#105 to upgrade Deno Core to the latest version. There's still quite a bit of work left to get this working and mergeable, but I'm hoping this will make it in soon!
Add a new std.glob() function, which will take a recipe and keep only the files matching a glob pattern
More cleanup to std.toolchain(), especially around autotools (you need to do some hacky stuff to use autotools currently)
@asheliahut made some progress on CMake (permalink) and Python (permalink). Both open up lots of new packages that can be added!
Cross-platform support and cross-compilation are going to be fairly massive tasks, but I'd like to start making some progress towards these. ARM64 Linux will probably be the second supported target, and macOS will be further down the line.

Announcing Brioche!

Kyle Lacy — Mon, 03 Jun 2024 00:54:32 GMT

I'm super excited to announce the first public release of Brioche! Brioche is a brand new package manager and build tool that builds on top of the best ideas of other package managers, like Nix, Homebrew, and Cargo. It's designed to be flexible and easy to use, and it leverages TypeScript for fancy typechecking and autocompletions in your build scripts.

I'd consider this release a Technical Preview. Currently, it's limited to x86-64 Linux only, there's only a small number of packages, and there are still some outstanding issues, including pretty bad performance issues and bugs in the Language Server Protocol implementation. Brioche isn't really at the point I'd recommend folks use it for Serious Business™ yet... but, everything is working, and I'm finally at the point where I'm ready to get some more eyes on the project.

If you've heard of Tangram before, Brioche might looks pretty familiar because Brioche heavily borrows ideas from it too. There's a bit of a history there, which I touch on below.

...Anyway, without further ado, here's a fairly simple Brioche project file:

// project.bri

// Import dependencies
import * as std from "std";
import { cargoBuild } from "rust";

// This will get built by default
export default function app() {
  // Build a Rust project with Cargo
  return cargoBuild({
    // Import Cargo files to build
    crate: Brioche.glob("src", "Cargo.*"),

    // Define the path of the default binary to run
    runnable: "bin/hello",
  });
}

// Define an extra export that builds a container
export function container() {
  // Wrap the Rust build into an OCI container image
  return std.ociContainerImage({
    recipe: app(),
  });
}

The Brioche.glob line imports files from disk next to the project.bri file, so this particular script is meant to live alongside an existing Rust project. Besides that, hopefully it's pretty self-explanatory what's going on. A few things you can do with this:

You can run brioche run, which will call the default export function. This returns a recipe that will use the Rust compiler to build the project. Then, Brioche will call the bin/hello binary
You can run brioche build -e container -o container.tar to call the container function. This will build an OCI container image, ready to be imported into either Docker or Podman
You can run brioche build -o output/ to save a directory containing bin/hello. Not just that: this directory will contain all of the runtime dependencies for the project (including glibc)! That means you can send it to another computer, and it'll run using the exact same dependencies you just ran it with

If you want to check out some more examples or if you want to give Brioche a spin yourself, take a look at the documentation. If you want to see the currently available packages (or just want to explore some real-world Brioche code), check out the brioche-packages repo.

Why write another package manager?

There are lots of tools that cover some or all of the use-cases I had in mind for Brioche: Nix, Bazel, Earthly, Homebrew, asdf, direnv, devenv, tea, and too many more to list. So why spend the time building something new?

Well, first off, it's fun! Brioche has been one of my favorite projects to work on. Rust and TypeScript are my two favorite languages to write, and I love the fairly low-level nature of working with compiler toolchains (even if dealing with autotools is endlessly frustrating...)

Second, Nix. A long long time ago (circa 2016), I daily drove NixOS and even briefly maintained a package in the Nixpkgs repo. I was really sold on Nix's ideas, but I eventually abandoned it because I ended up feeling pretty frustrated when using it. I learned Nix-the-language pretty in-depth, but it never really felt intuitive to me. Derivations always felt weirdly rigid. I didn't like that the Nixpkgs repo as a whole is versioned as one unit instead of individual packages. I didn't like how much work I had to put in to get stuff that just works out-of-the-box everywhere else to work on NixOS: I felt like my tools work working against me instead of for me.

2016 was 8 years ago now, and I know the Nix ecosystem has evolved a lot since then. I know Flakes are supposed to be a game changer, there are lots more packages and resources for using Nix, and the core tooling has certainly been improved a lot since then. But it was this experience that planted the idea for Brioche in my head, and I couldn't shake the feeling that, starting from scratch, I could design something less quirky than Nix while keeping its best features.

What's in store for Brioche

In the short term, there's lots of things I want to get done: improve performance (I have a lot of ideas here), add more packages, make it easier to do more exotic builds, get Brioche working on more platforms, etc.

In the long term, I want Brioche to be the best way to manage your software projects. I end up finding myself mixing and matching lots of tools in weird ways: I've had projects that mix Rust and TypeScript, Rust and ffmpeg, Rust and Swift, Rust and the Windows API, Rust and Godot, Rust and SDL2, etc. (did I mention I like Rust?)

My dream is that I could use Brioche for all of these projects. I want to make a project and have all of its external dependencies tracked in a lockfile. I want to run one command that takes care of building, seeding my test databases, checking formatting, then starting a server in watch mode with my database running alongside. I want to use the same build configuration locally, for a container image, and for CI builds. And I want it to work without virtualization whether I'm using Windows on my desktop, macOS on my laptop, or Linux on the server (and Linux on the desktop when I can finally run all the games I want through Proton)

On the Nix community

The Nix community has recently been going through Something. My decision to build Brioche was in no way a reaction to the recent fallout around Nix. I still have a ton of respect for Nix as a community, and I'm genuinely rooting for them to establish a solid governance model that puts the community first so Nix-the-project can thrive.

On Tangram

Tangram is another package manager that has a lot of overlap with Brioche: it's heavily Nix-inspired, it uses TypeScript for build scripts, it's written in Rust, and generally Brioche build scripts look pretty similar to Tangram build scripts. At the time of writing, Tangram hasn't yet been released, but it's due to release very soon.

The similarities between Tangram and Brioche aren't a coincidence: I worked for Tangram, Inc. starting in October 2022 to April 2023, when I was fired (I felt this was an unfair decision, but I'm not here to tell that story). I had been working on Brioche before my time there, and I decided to resume it some time after I was no longer employed by Tangram.

On reflection, I felt there were some parts of Tangram's design that would work well for the goals I had for Brioche, and some that didn't align with my goals[^1], so I simply incorporated what I learned at Tangram into the current iteration of Brioche. I think it's fair to say that Brioche is, ethically, a fork of Tangram, even though haven't used any code from Tangram directly (although I am still using some prebuilt static Busybox/Dash/env binaries that Tangram built/published). This is also why the Brioche license includes a copyright notice from Tangram. Brioche was work-in-progress well before my time at Tangram, Inc. but I wanted to acknowledge that Tangram was the source for several design decisions I made in Brioche.

[^1]: In an earlier version of this post, I felt I worded my feelings about Tangram's design decisions poorly. I had said that I thought there were elements of Tangram's design that didn't work as well, but I should've clarified that I meant in the context of my goals for Brioche specifically. I've tweaked some sections where I talked about Tangram, and my apologies for anyone that walked away with a different impression on Tangram after reading the first version of this article.

Brioche Blog

Brioche v0.1.8: Preparing for the future

(Minor breakage) TypeScript v6 and linting changes

Show source paths during builds

Linux sandbox improvements

Reader support for new cache archive tag

Brioche v0.1.7: Simplified CLI

Simplified CLI syntax

Configurable split read/write cache

Sandbox: Resolve localhost

Experimental: Skipping cached artifacts

TypeScript v5.9.3 + ESLint upgrade

Brioche Project Update - December 2025

Highlights from the year

Status report

TypeScript 5.9.3 upgrade

brioche fmt improvements

More major groundwork packages

Sad personal news

Continued infrastructure work

Minor caching tweaks

Housekeeping

New packages

Brioche core updates

Brioche Project Update - November 2025

Status report

Brioche v0.1.6 released

setup-brioche action overhaul

New aarch64 runner

Python packaging fixes

Lots more foundational X11 packages

Infrastructure rework

Housekeeping

New packages

Brioche core updates

Brioche v0.1.6: Now for ARM64!

aarch64 support

Live updates

Faster bulk checking and formatting of packages

Lazy building (experimental)

Lazy publishing

Shorthand (non-function) exports

Release process changes

Brioche Project Update - October 2025

Status report

Officially working part-time on Brioche!!

Now taking donations / sponsorships!

New docs for contributors

Build Python with optimizations

More consistency fixes in brioche-packages

Some packages CI / repo improvements

Installer & release work

Issues galore!

Housekeeping

New packages

Brioche core updates

Brioche Project Update - September 2025

Status report

CMake build improvements

More build parallelism

libtool / pkg-config fixes

X11 libraries

OpenTelemetry improvements

Yak shaving: instrumentation tools

Housekeeping

New packages

Brioche core updates

Brioche Project Update - August 2025

Status report

Set up merge queues for CI

"Lazy builds" for CI

object_store update for better retry logic

Even more live-update improvements

Housekeeping

New packages

Brioche core updates

Brioche Project Update - July 2025

Status report

Initial exploration of GUI packages

Lots of live-update improvements

Sandbox: Resolve `localhost`

`brioche fmt` improvements

`setup-brioche` action overhaul

More consistency fixes in `brioche-packages`

`object_store` update for better retry logic

Simplify recipes in `std`

`pipe` utilities in `std`

`brioche.dev` updates

New `unsandboxed` sandbox backend

Coming soon to `std`

Support for unarchiving `.zip` files

`process.currentDir` to change a process's starting directory

`Brioche.gitCheckout` to checkout a repo directly

`Brioche.gitCheckout` groundwork

`brioche fmt` fix