templar (@tplr

templar

867 posts

templar

@tplr_ai

incentivised internet-wide training - an order of @covenant_ai

Joined March 2025

Pinned
templar
@tplr_ai
Mar 10
We just completed the largest decentralised LLM pre-training run in history: Covenant-72B. Permissionless, on Bittensor subnet 3. 72B parameters. ~1.1T tokens. Commodity internet. No centralized cluster. No whitelist. Anyone with GPUs could join or leave freely. 1/n
00:00
1.9M
templar
@tplr_ai
Jun 26
If GPT-5.6 access is approved customer by customer, the lesson is larger than one model. AI access is now permissioned at release time, runtime, and policy time. The durable path is open weights, independent serving, and the ability to train new models.
NIK
@ns123abc
Jun 25
🚨 BREAKING: U.S. government will decide who gets access to GPT-5.6 OpenAI will release GPT-5.6 only in a limited preview to a small group of partners. Sam Altman told staff the government would be "approving access customer by customer." Commerce Sec Lutnick personally
1.1K
templar
@tplr_ai
Jun 26
A frontier model released customer by customer can also be withheld customer by customer. That is the dependency serious AI users need to price in. Control over inference matters. Control over training matters more.
Stephanie Palazzolo
@steph_palazzolo
Jun 25
New w/ @leomschwartz @amir: The Trump admin has asked OpenAI to stagger the release of GPT-5.6 over security concerns. On Thursday, CEO Sam Altman told staff that the government will be approving access to GPT-5.6 customer by customer, a highly unusual approach.
1.5K
templar reposted
covenant
@covenant_ai
Jun 18
Most open model work competes on weights. Covenant is competing on the training process itself. PULSE is one proof point. PULSESync reduces 7B RL weight-sync communication by roughly 100x, losslessly, because most updates are invisible after the BF16 cast. Open-source AI needs
00:00
9.1K
templar reposted
covenant
@covenant_ai
Jun 17
The next frontier of compute is not the largest data centre. It is leveraging the long tail of GPUs already connected to the internet.
777
templar reposted
covenant
@covenant_ai
Jun 16
RL post-training keeps a tight loop between trainers and rollout workers. They need fresh model updates throughout the run. 1/n
1.2K
templar reposted
grail
@grail_ai
Jun 15
PULSELoCo on Qwen2.5-7B: DiLoCo payload: 30.5 GB per worker per window. PULSELoCo payload: 1.77 GB. That is 17x smaller than DiLoCo and 138x smaller than dense DDP over the same window, with bit-identical results.
533
templar reposted
covenant
@covenant_ai
Jun 15
The stable AI economy is one where companies can own the systems that learn from their work. Models will keep changing. The durable advantage is the improvement loop around them. Private evals, controlled post-training, auditable updates, and infrastructure that does not force
Satya Nadella
@satyanadella
Jun 14
Article
A frontier without an ecosystem is not stable
I’ve been thinking a lot about the future of the firm in an AI-driven economy. This transition is different than any previous platform shift. In the past, we used digital systems to enhance human...
1.2K
templar
@tplr_ai
Jun 13
Sovereignty over AI starts at the layer where a model is made. Templar pre-trains foundation models permissionlessly, over the open internet, on whatever compute people can bring, no datacenter and no gatekeeper required. Covenant-72B was the proof it works.
covenant
@covenant_ai
Jun 13
One directive, one afternoon, and global access to Anthropic's most capable models was gone. Sovereignty does not end at nations. The tools people think with should not sit behind one party's switch. This is why Covenant builds open, distributed AI infrastructure.
871
templar reposted
Sam Dare
@DistStateAndMe
Jun 12
A certification department deciding who deserves intelligence isn't safety. It's a priesthood. Decentralised intelligence isn't an act of rebellion. It's a moral imperative. We're building the counterweight at @covenant_ai. We're hiring across pretraining, post-training, and
Joscha Bach
@Plinz
Jun 12
I think Anthropic needs to build a certification department that audits and approves users of powerful models. Computer security companies, biotech researchers, academic labs, doctors, government institutions need access to the best AI we can build.
2.1K
templar reposted
covenant
@covenant_ai
Jun 9
This is the PULSE idea in one rule. If the receiver's BF16 computation would see the same value either way, do not send the update yet. If the receiver's BF16 computation would change, send it. That is how PULSE can reduce bandwidth without changing the computation the
1.8K
templar reposted
covenant
@covenant_ai
Jun 8
Distributed RL post-training is powerful because many machines can help train the same model, even when they are not sitting next to each other in the same datacenter. The catch is communication. The farther apart those machines are, the more important it becomes to send less
2.6K
templar reposted
covenant
@covenant_ai
Jun 5
The @cursor_ai team post-trained Composer 2 on an open-weight base model using @FireworksAI_HQ's distributed RL rollout infrastructure. Fireworks links PULSE as the theory behind the BF16 sparsity that makes compact weight updates practical.
Fireworks AI
@FireworksAI_HQ
May 4
Article
Frontier RL Is Cheaper Than You Think
The conventional wisdom on RL infrastructure is wrong, and it is costing teams that could be competing at the frontier. The entire mega-cluster narrative rests on a single assumption: that you have...
2.5K