Product.ai / Join / Projects / Cost-Split Multi-Model Routing — make Haiku, Sonnet, and Opus do their actual jobs across one cron pipeline
Project Open to Alpha Team

Cost-Split Multi-Model Routing — make Haiku, Sonnet, and Opus do their actual jobs across one cron pipeline

Pick one Cortex or AIOS cron pipeline that currently calls Opus 4.7 on every step and refactor it to cost-split routing — Haiku 4.5 or Sonnet 4.6 for routine sub-steps, Opus 4.7 only on escalation, advisory, or synthesis steps. Use the existing `ops/scripts/model_router.py` infrastructure. Measure the before/after token spend, the failure rate, and the latency. Ship a routing primitive other pipelines can adopt without re-asking.
Project Overview
Discipline
AI Systems — AI Engineer · ai-systems-engineer · ai-systems-associate
Duration
2 weeks
Compensation
Your stated freelance rate
Surface
Engineering · Product.ai
Kernels
engineering · productai
Outcomes
team-visible · team-velocity · chat-expert
Tier
Applied
Alpha Team
Open to alpha members who want to take this on
Tooling
Claude Code or Co-work

Why we want this done

Token-cost awareness is the 2026 discriminator (Phase 3 axiom E2). Eval-first vocabulary has commoditized; the candidates who report token spend unprompted, describe when they switch between Opus/Sonnet/Haiku and why, and revert to manual coding when spend doesn't justify itself are the operators who run mid-market production AI economically. Multi-model routing is universal at frontier-adjacent firms — Vercel n=656 survey reports average of 2 model providers per team and 65% of teams switched providers in the prior 6 months. Single-model production is the anti-pattern (axiom C5). Today, several Cortex and AIOS cron paths default to Opus on every call. For a sub-agent fan-out where 80-90% of calls are routine triage or extraction, that is a 5-10x token-spend overcharge on work Haiku does just as well. Whichever pipeline gets routed first becomes the template; the rest follow.

Scope

  1. Audit Cortex/AIOS cron paths for token spend per run (Anthropic Enterprise Analytics + invocation logs)
  2. Pick the highest-leverage candidate — high call count, high routine-call percentage, low end-to-end risk if a sub-step fails
  3. Decompose the pipeline into steps; classify each step as routine (Haiku/Sonnet) vs. judgment (Opus advisor) vs. critical synthesis (Opus)
  4. Implement the routing via ops/scripts/model_router.py (extend if needed; do not inline single-provider API calls)
  5. Run a controlled A/B (legacy single-model vs. cost-split) for at least 3 invocations of each
  6. Measure: token spend delta, failure rate delta, latency delta, output quality delta (sample-graded by Opus on a frozen golden set)
  7. Document the routing primitive — when to use which tier, what failure modes triggered escalation, what spend is a reasonable floor

What success looks like

  • Token spend on the chosen pipeline drops by ≥40% with no increase in failure rate
  • The routing logic lives in model_router.py (or its successor) — no inline API calls in the pipeline script
  • The A/B harness is reusable for the next pipeline owner who wants to migrate
  • The candidate's routing decisions are defensible per step (not "I lowered everything to Haiku and saw what broke")
  • A pipeline owner who has never seen the work can adopt the same pattern from the docs

References

references.md
AI Engineering Phase 3 briefing axiom E2 (Token-Cost Awareness as Operational Instrument), C5 (Multi-Model Routing Universal)
ops/scripts/model_router.py and any existing routing helpers
Anthropic Enterprise Analytics dashboard for invocation cost data
VEC-34 Token ROI working doc for any prior cost analyses
Vercel State of AI 2026 (n=656) survey on multi-provider adoption

Constraints

  • Claude Code as primary substrate
  • All API calls through model_router.py — no inline single-provider API calls (Cortex and AIOS rule)
  • Anchor on Anthropic official $13/dev/active-day economics; reject Frontier $2-3K/day anchoring; reject under-adopter $50/month anchoring
  • Quality-first cost-aware: never downgrade Opus to Sonnet on critical synthesis without evidence; the rule is "right model for the job," not "cheapest model"
  • IP separation: cron paths in scope are application-layer; methodology paths (aios-methods/_tools/arc-autopilot/) are out of scope
Apply
01

Read the Codex (10 min)

The operating principles we work by. If they resonate, the rest of this will land. Open the Codex →

02

12-minute video screen

Hireflix, async. Questions are calibrated to this project specifically.

03

Chemistry call (30-60 min)

Direct call with the CEO. Strategic alignment and mutual fit. No problem-solving exercise.

04

Project begins within 2-3 weeks

1099 contractor agreement, NDA, paid at your stated rate. Day 1 in Santa Monica.

Alpha Team members can take this project without the screen-and-call sequence. Reach out via the Alpha Team channel.

Apply for this trial Browse other projects