Product and Research Operations Manager

Ovii's Interpretation of the Role

The Product and Research Operations Manager will build and run a global, human‑in‑the‑loop evaluation workforce for AI models. You will own vendor strategy, workforce design, quality‑control systems, and day‑to‑day operational performance across product, engineering, and data teams.

Role Snapshot

• Lead global evaluation workforce
• Own vendor strategy & contracts
• Design scalable QA and metrics
• Drive capacity planning and throughput
• Collaborate across product, engineering, and data

Must-Have Requirements

• operations
• workforce management
• vendor management
• scale operations
• metrics analysis
• quality control

Nice-to-Have Signals

• AI/ML data operations
• audio/speech workflow knowledge
• multilingual operations
• marketplace platform experience
• familiarity with QA systems and annotation tooling
• audio/speech workflows
• AI/ML
• Speech technology

Work Setup

• Location: San Francisco, USA
• Work mode: ONSITE
• Employment type: Full-Time

Eligibility Gates

• Work authorization: Must be eligible to work in the United States
• Visa sponsorship: yes

Not Specified in JD

• Salary range
• Remote eligibility

What You'll Likely Work On

• Design and implement evaluator structures across languages, skill tiers, and use cases
• Build capacity models to support continuous evaluation pipelines and data production
• Negotiate rate cards, SLAs, and throughput guarantees with annotation vendors and contractor platforms
• Choose build, buy, or hybrid workforce models and benchmark cost and performance across regions
• Create multi‑layer QA systems with self‑checks, peer reviews, audits, and gold‑standard tasks
• Define and track inter‑rater reliability, error rates, and annotator‑level performance distributions
• Develop escalation and retraining workflows to maintain quality at scale
• Run day‑to‑day operations: task allocation, throughput tracking, and SLA adherence
• Build systems to reduce evaluator fatigue, rotate task types, and ensure consistency
• Partner with tooling and data teams to improve evaluator UX and deliver clean outputs for model training

Good Fit If You Have

• Experience in AI/ML data operations or evaluation pipelines
• Background in audio, speech, or language‑related workflows
• Familiarity with QA systems and annotation tooling
• Experience with marketplace platforms such as Upwork or Mercor
• Exposure to multilingual evaluation operations