Staff Software Development Test Engineer

Ovii's Interpretation of the Role

The Staff Software Development Test Engineer will lead AI‑focused test automation for Tekion’s cloud‑native automotive platform. You will design and run evaluation suites for LLMs, RAG pipelines, and ML models, embedding quality checks into CI/CD. Collaboration with engineering and product teams ensures AI‑driven features meet reliability and safety standards.

Role Snapshot

• Senior SDET (staff level)
• AI/ML testing specialist
• Automation framework owner
• Quality gate enforcer for CI/CD
• Collaborates with product & engineering

Must-Have Requirements

• Python
• SQL
• Pytest
• Postman / REST Assured / Requests
• RAGAS / DeepEval / Promptflow
• LangChain / LangSmith / LlamaIndex
• OpenAI / Anthropic / HuggingFace APIs
• Vector DB testing
• Pandas / NumPy
• MLflow
• Docker
• GitHub Actions
• Jenkins
• Grafana / Kibana / OpenTelemetry
• SDET

Nice-to-Have Signals

• AWS Bedrock / Azure OpenAI / GCP Vertex AI
• Kubeflow / Weights & Biases / Feast
• Scikit‑learn / TensorFlow / PyTorch
• Kubernetes / Terraform
• Playwright / Cypress
• Locust / JMeter
• Statistical hypothesis testing
• Synthetic data generation

Work Setup

• Location: Bangalore, India
• Work mode: ONSITE
• Employment type: Full-Time

Not Specified in JD

• Visa sponsorship
• Salary range
• Remote eligibility
• Education requirement
• Certifications

What You'll Likely Work On

• Build automated suites that detect hallucinations, bias, toxicity, and prompt‑injection in LLM‑powered products
• Implement RAG evaluation pipelines measuring relevance, groundedness, and answer faithfulness using frameworks such as RAGAS or DeepEval
• Design test beds for multi‑agent workflows, validating tool‑calling, reasoning, memory, and autonomous decision loops
• Run scripted and synthetic conversation simulations to stress‑test agents across intents, edge cases, and multi‑turn dialogs
• Create prompt regression frameworks to monitor output consistency when prompts, temperature, or sampling parameters change
• Statistically validate AI data outputs and audit data pipelines for schema drift, corruption, and vector‑DB indexing quality
• Maintain scalable test automation frameworks for APIs, backend services, and model endpoints, integrating them into MLOps and CI/CD pipelines
• Define AI quality KPIs, track ML metrics, and communicate release readiness to engineering and product stakeholders

Good Fit If You Have

• Familiarity with cloud AI services (AWS Bedrock, Azure OpenAI, GCP Vertex AI)
• Experience with MLOps platforms (Kubeflow, Weights & Biases, Feast)
• Knowledge of ML frameworks (Scikit‑learn, TensorFlow, PyTorch)
• Comfort with infrastructure‑as‑code tools (Kubernetes, Terraform)
• Exposure to UI automation (Playwright, Cypress) or performance testing (Locust, JMeter)