Bittensor’s Subnet 3 Trains 72B AI Model on Decentralized Network

TLDR:

• Covenant-72B scored 67.1 on MMLU zero-shot, beating LLaMA-2-70B’s 65.6 under identical test conditions.
• SparseLoCo reduced communication overhead by 146x using sparsification, 2-bit quantization, and error feedback across nodes.
• Gauntlet scored every node’s contribution via loss evaluation and OpenSkill ranking, all recorded on the blockchain.
• $TAO rose 14% to $236 post-announcement, with Grayscale expanding its TAO trust for institutional investor access.
  

Bittensor’s Subnet 3 has trained a 72-billion-parameter AI model without a central data center. The model, named Covenant-72B, was built across more than 70 global participants.

All nodes are connected through a standard home internet. Covenant-72B outperformed Meta’s LLaMA-2-70B on the MMLU benchmark, scoring 67.1 against 65.6.

The test ran under identical zero-shot conditions. This outcome challenges long-standing assumptions about what decentralized compute can achieve.

Two Technical Innovations Drove the Decentralized Training

For years, AI crypto projects claimed decentralized compute could match centralized labs. Bittensor’s Subnet 3 now backs that claim with measurable results.

The training covered 1.1 trillion tokens across more than 70 nodes worldwide. Every node ran on 500 Mb/s commodity internet connections.

Two core innovations made this scale of training possible. SparseLoCo cut communication overhead by 146 times throughout the process.

It combined top-k sparsification, 2-bit quantization, and error feedback to keep all nodes in sync. No central server was needed to manage coordination across the network.

The second innovation, Gauntlet, handled trust and contribution scoring during training. It assessed each node through loss evaluation and OpenSkill ranking.

All scores were logged on the blockchain for full transparency. This gave every participant a verifiable record of their contribution.

Milk Road reported on the outcome via social media, noting that distributed networks can now train large models competitively. The model weights are available on Hugging Face under an Apache License.

Anyone can access, use, or build on Covenant-72B at no cost. That open approach separates it from many restricted, proprietary AI models available today.

$TAO Climbs as Market Responds to Covenant-72B Results

The market moved quickly after news of the Covenant-72B training spread publicly. $TAO, Bittensor’s native token, rose 14% to reach $236 following the announcement.

The token had also gained 36% over the prior 30-day period. Trading volume grew 167% across the past six months.

Grayscale expanded its TAO trust during the same week as the announcement. That move opened up broader institutional access to the token directly.

It came as investor interest in AI-linked crypto assets continued to grow. The timing added further upward pressure to the token’s price movement.

The combination of a technical result and institutional interest drew wide market attention. Covenant-72B’s MMLU score gives decentralized compute a credible, testable benchmark.

The result is measurable and can be reproduced under standard conditions. That distinguishes it clearly from many earlier unverified claims in the AI crypto space.

The Apache-licensed weights on Hugging Face allow any developer to verify the work independently. Bittensor’s approach shows a functioning framework for community-driven AI model training.

The network ran across 70-plus participants with no central coordination at any point. This sets a working precedent for distributed large-model training going forward.

The post Bittensor’s Subnet 3 Trains 72B AI Model on Decentralized Network appeared first on Blockonomi.