Models commoditized. Context engineering, evaluation architecture, and verification certainty became the new moats. The 10x AI Systems operator architects the oracle that decides what is true — not the model that generates the candidate. Below: open projects, the physics of what is changing, and roles we are hiring against.
Real, paid 1-3 week engagements with the Product.ai team. Each one is a problem we are working on at the frontier of AI systems work — and the kind of work we hire against. Pick one that pulls at you and apply.
Last updated
In May 2026, AI Systems work at frontier firms (Anthropic, OpenAI, Cohere, Mistral, Inflection, Adept, Scale, Anysphere) has bifurcated cleanly. Three job families — Research Engineer, ML/Eval Engineer, AI-Native Data Engineer — share tooling but not craft. The shared physics: <strong>the model is a commodity; the context, the eval, and the verification protocol are the moat.</strong> Senior practitioners do not measure themselves by accuracy at training time. They measure by what holds up under adversarial paraphrase, distribution shift, and closed-loop deception. The 10x operator is the one who treats the evaluation framework as a shippable executable contract, not a slide.
The "AI Systems" label collapses three structurally distinct families. Research Engineers implement and run experiments at the model surface — pre-training, fine-tuning, RLHF, alignment. They're found at frontier labs (Anthropic, OpenAI), not at every product company. ML/Eval Engineers are the largest category — they build production inference, evaluation, monitoring, and deterministic-gate infrastructure around models. AI-Native Data Engineers architect the context substrate: structured commerce data, knowledge graphs, retrieval indices, embedding stores, the canonical truth that models read from.
These families share a stack (Python, PyTorch/JAX, vector DBs, LLM SDKs) but optimize for different objectives. Research Engineers optimize for capability. ML/Eval Engineers optimize for verified production behavior. Data Engineers optimize for context fidelity. A 10x Research Engineer who tries to ship product evals will under-invest in adversarial coverage; a 10x Data Engineer who tries to write a fine-tune will under-invest in distribution-shift defense. Hire to the family, not the umbrella.
The legacy ML deliverable was a trained model — a `.pkl` artifact you handed to the engineering team. In 2026 the deliverable is an executable evaluation contract. The eval framework is the specification of what "good" means. Anthropic's public eval philosophy, OpenAI's evals repo, and Scale's Spellbook all converge on this: the team that ships the better eval ships the better product, even if the underlying model is the same. The eval is what calibrates trust, exposes failure modes before users find them, and gates production deployments.
The senior practitioner spends more time designing the rubric than running the experiment. They distinguish glass-evaluator structural checks (path placement, schema validity — bright-line answers, regex works) from stone-evaluator semantic checks (is this content sovereign? does this answer cite real evidence? — meaning is not pattern-matchable). They build regression test corpora with paraphrase variants and prompt-injection attempts. The frontier moat is not "we have a better model." It's "we have a better oracle that decides what counts as a correct model output."
The legacy DS metric — accuracy on a held-out test set — has eroded as a meaningful signal. Verification certainty replaced it: the probability that an output is correct AND the system knows whether it is correct. A model with 92% accuracy and zero calibrated uncertainty is operationally worse than a model with 87% accuracy and reliable I-don't-know detection. The Trust Paradox documented at Product.ai (74% rate trust 4-5/5; 93% verify before acting) is not a survey artifact — it's the user's correct response to systems that don't signal their own confidence.
The 10x AI Systems operator instruments behavioral verification: citation-click rates, override rates, escalation rates, retry rates. These are the load-bearing trust signals. They beat survey-based scoring in every controlled study. The operator who can articulate which of their production systems systematically over-state their own confidence is higher-signal than the one who can't. Calibrated uncertainty is the new moat. Anyone who treats "model accuracy" as the primary metric is a generation behind.
Frontier AI Systems work in 2026 is increasingly about making models talk to other models reliably. Agent-to-agent protocols (MCP, OpenAI Agents SDK, Anthropic's Computer Use), tool-call routing, multi-agent orchestration, and verified-truth APIs are the fastest-growing area. Product.ai's thesis — that verified commerce knowledge is the agent-economy infrastructure layer — is one expression of this physics. Agents need ground truth that survives paraphrase and adversarial input; the firm that ships that infrastructure first wins the coordination layer.
This is why MCP server engineers, agent-tool integrators, and verified-knowledge graph architects have become some of the highest-leverage seats at frontier firms. The work is unglamorous from the outside (defining a tool schema, writing an authentication flow, designing a retry protocol) and load-bearing from the inside (every downstream agent that calls your tool inherits your design choices). 10x operators in this layer think about adversarial use, partial-failure modes, and protocol evolution — not just happy-path correctness.
The "MLOps" framing of 2021-2024 systematically under-invested in evaluation and over-invested in deployment automation. The result was a generation of teams who could ship a model in five minutes but couldn't tell you whether the model was getting worse over time. Anthropic's and OpenAI's 2025-2026 postmortems both cite "eval coverage gaps" as the dominant root cause of production incidents — not deployment failures, not infrastructure issues, but failure to measure what matters.
The companion mistake: treating the model as the bottleneck. In every observable case, the bottleneck is upstream — context quality, retrieval relevance, eval calibration, prompt design. Teams that swap GPT-4 for Claude or vice versa rarely see the gain they expect. Teams that improve their context substrate, their retrieval, or their eval rubric routinely see 2-5x quality improvements with the same model. The frontier moat is below the model, not at it. Practitioners who agree with this framing voice it specifically — context-engineering, retrieval-architecture, eval-design — not in slogans about "AI capability." The frontier is in the dissent, not the consensus.
Most of our best people came through projects, not interviews. If a project pulls at you and the trial goes well, the role conversation follows.
Twelve-minute Hireflix video, async. Then a 30-60 minute chemistry call. Then a paid 1-3 week project alongside the team. We will know within a week whether to move forward.