AI Engineer (AI labs)

Our client builds reinforcement learning (RL) environments and evaluations for leading frontier labs. They've reached $250M+ in annual recurring revenue within nine months of founding and are scaling the engineering team to meet growing demand from the labs building the next generation of AI models.

This role involves building and improving AI agents that generate and validate real-world tasks. You will be responsible for enhancing these agents using traces and evaluation results, turning enterprise workflows into executable tasks for AI training and evaluation, and ensuring their quality and behavior as frontier models evolve.

• Build and improve agents, prompts, tool interfaces, context handling, and orchestration across task generation and validation.
• Inspect model calls, tool use, trajectories, screenshots, state changes, and grader outputs for trace-driven debugging.
• Turn recurring failures into targeted code and agent changes.
• Review pipeline outputs, form hypotheses, implement fixes, and test on fresh runs and regression suites before rolling out improvements.
• Choose and instrument tracing, debugging, and experiment tools.
• Track quality, latency, token usage, and cost by agent and pipeline stage to improve model and tool choices.
• Improve validation logic and reward criteria for task and grader quality, ensuring shipped tasks are solvable and evaluations reflect genuine completion.
• Partner with Platform Engineers to turn agent improvements into modular components that can be evaluated and released reliably.
• Own a workflow from output review through debugging, evaluation, and release to find the actual failure.
• Build repeatable checks to distinguish agent failures from task or grader defects, making evaluation signals trustworthy.
• Fix recurring failure patterns and prove gains across task families, fresh batches, and regression suites.
• Establish a repeatable way to compare quality, latency, and cost across versions for fast iteration and measurable improvements.

• Strong software engineering fundamentals and hands-on Python experience.
• Ability to read unfamiliar code and debug a system across component boundaries.
• Experience building LLM agents or multi-step AI workflows with tool use, structured outputs, and state management.
• Comfort designing evaluations and using traces and data to distinguish a real improvement from a lucky run.
• Demonstrated ownership through implementation and verification, with clear communication about failures, tradeoffs, and evidence-supported decisions.
• Useful additional experience includes computer-use agents, RL environments, reward design, browser automation, or tracing and evaluation platforms.

• Meaningful ownership over technical decisions.
• Direct feedback from how our work performs with frontier models.

• Stage 1: Apply and submit the application form
• Stage 2: R0 with Talent Team
• Stage 3: F2F interviews at Bangalore office