Four years into Zen’s development, it’s worth stepping back to assess where we are and where we’re going. Open AI development has made remarkable progress. It also faces significant challenges. Here’s my honest assessment.

What We’ve Achieved

Competitive Models

Open models now match or exceed proprietary alternatives in many domains:

  • Coding: Open models lead on HumanEval and MBPP
  • Reasoning: Competitive on GSM8K and MATH
  • General knowledge: Within 5% on MMLU
  • Multilingual: Often superior for non-English languages

The capability gap that seemed insurmountable in 2021 has largely closed for models under 100B parameters.

Real Adoption

Open models power real applications:

  • Millions of API calls daily through community inference providers
  • Thousands of fine-tuned variants for specialized tasks
  • Integration into major development tools
  • Deployment in production systems worldwide

This isn’t just research anymore. It’s infrastructure.

Community Scale

The open AI community has grown enormously:

  • Hundreds of contributors to major projects
  • Thousands of researchers building on open foundations
  • Millions of users accessing open models
  • Billions of inference requests monthly

Network effects are real. Each contribution makes the ecosystem more valuable.

Persistent Challenges

Compute Concentration

Training frontier models requires massive compute. This compute remains concentrated:

  • Top 5 cloud providers control 80%+ of AI-grade compute
  • Chip manufacturing bottlenecks persist
  • Training costs continue rising with scale

The Zoo Compute Network helps but isn’t yet large enough for frontier training. We need 10x current capacity.

Data Moats

Large organizations have accumulated unique datasets:

  • Proprietary user interactions
  • Licensed content
  • Internal documents
  • Real-world feedback signals

Synthetic data and creative commons content help but don’t fully substitute. Data remains a key differentiator.

Coordination Costs

Open development is harder to coordinate than corporate efforts:

  • Decisions require consensus
  • Resources are distributed
  • Timelines are uncertain
  • Accountability is diffuse

DAOs and governance frameworks help but add overhead. Closed organizations can simply decide and execute.

Safety Complexity

As models become more capable, safety becomes more critical:

  • Evaluation is hard (what are we even measuring?)
  • Mitigation often reduces capability
  • Adversarial robustness remains elusive
  • Dual-use concerns complicate openness

We can’t ignore safety. We also can’t let it become a barrier to open development.

The Path Forward

Frontier Open Models

Training truly frontier models openly requires:

  1. Scaled compute network: 100K+ GPU network with reliable coordination
  2. Novel architectures: Efficiency improvements that reduce compute needs
  3. Better data: Synthetic data, simulation, and permissioned partnerships
  4. Sustained funding: Multi-year commitments to training runs

This is achievable. It requires coordination the community hasn’t yet demonstrated.

Infrastructure Maturity

Open infrastructure needs to mature:

  • Training: Training Gym is solid but needs scaling features
  • Inference: Efficient serving remains fragmented
  • Fine-tuning: Needs better tooling for non-experts
  • Evaluation: Benchmarks don’t capture real-world performance

Each layer needs investment. Each layer enables the next.

Governance Evolution

Current governance works for small decisions. Bigger decisions need:

  • Faster processes: Weeks, not months
  • Expert input: Technical decisions need technical judgment
  • Accountability: Approved decisions must be implemented
  • Representation: Diverse stakeholders, not just token holders

ZIPs are a start. We need to evolve.

Safety by Design

Open development can lead on safety:

  • Transparency: Open models can be audited
  • Diversity: Multiple approaches reduce correlated failures
  • Iteration: Faster feedback loops on safety interventions
  • Community: More eyes on potential problems

We should demonstrate that openness enhances safety, not undermines it.

Predictions for 2025

Concrete predictions to be evaluated:

  1. Open models will match GPT-4 class on major benchmarks (>50% confidence)
  2. Zoo Compute Network will reach 50K GPUs (60% confidence)
  3. At least one open model will be in top-3 on major leaderboards (70% confidence)
  4. ZIPs will process >100 proposals (80% confidence)
  5. Foundation funding will exceed $50M (40% confidence)

I’ll revisit these next November.

A Personal Note

When we started Zen, open AI development was a niche concern. Most assumed the future belonged to well-resourced corporations. That assumption was wrong.

Open development has proven viable. Community coordination works. Distributed resources aggregate. Transparency accelerates progress.

But viability isn’t victory. The next phase requires:

  • Sustained commitment through multi-year projects
  • Patience with governance overhead
  • Investment in unsexy infrastructure
  • Honesty about limitations and failures

We’ve shown open AI development is possible. Now we need to show it’s preferable.

The future of AI is still being written. Let’s write it together.

Zach Kelling is a co-founder of Zoo Labs Foundation.