Evaluations Team Lead

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for an Evaluations Team Lead based in Romania. As an Evaluations Team Lead, you will shape how advanced AI models are evaluated, validated, and improved in production environments. You will lead the development of a shared evaluation platform that enables research, engineering, and customer-facing teams to measure model performance with confidence. Your work will establish rigorous standards for benchmarking, statistical validation, data quality, and reproducibility. Combining hands-on technical leadership with people management, you will build a high-performing team while remaining closely involved in code, experiments, and methodology. You will assess predictive quality, inference latency, and cost efficiency to guide research priorities and product releases. This is an opportunity to define evaluation practices from the ground up and directly influence the reliability and impact of next-generation enterprise AI. Accountabilities: - Build and maintain a shared evaluation platform that supports research, engineering, and Applied AI teams, enabling independent benchmarking and investigation of model performance. - Develop and continuously improve benchmark suites, curated datasets, and model comparisons to assess predictive quality across a broad range of business use cases. - Establish rigorous evaluation standards covering statistical metrics, validation strategies, data splits, leakage prevention, calibration, uncertainty estimation, and benchmark contamination. - Design reproducible evaluation pipelines with version-controlled inputs, datasets, and artifacts, integrating automated regression checks into research and release workflows.

- Benchmark models against competing approaches using fair and consistent protocols for hyperparameter tuning, data access, computational resources, and end-to-end latency measurement. - Evaluate predictive performance, inference latency, and cost efficiency across deployment configurations, task types, and dataset characteristics. - Support customer proof-of-concept evaluations by providing reliable tooling, methodological guidance, and data-driven analysis to help demonstrate model value. - Translate evaluation findings into actionable research priorities, release recommendations, and evidence-based improvement plans for customer use cases. - Recruit, lead, and develop a small team of evaluation specialists, setting priorities, providing technical coaching, and delivering constructive performance feedback. - Remain hands-on with Python development, experimental design, statistical analysis, and evaluation methodology while fostering collaboration across research, engineering, and customer-facing teams. Requirements: - Proven experience owning the evaluation of tabular machine learning systems used in production environments or high-impact customer decision-making. - Strong statistical expertise, including selecting appropriate metrics and validation schemes, estimating uncertainty, comparing models across datasets, and accounting for repeated experimentation. - Practical experience identifying and preventing data leakage involving preprocessing, feature engineering, joins, temporal dependencies, and related entities across training and evaluation splits. - Strong proficiency in Python and SQL, with familiarity with scikit-learn, gradient-boosted decision trees, and the development of reliable ML tools or platforms used by multiple teams. - Demonstrated experience designing fair and reproducible model comparisons, including hyperparameter optimization, resource allocation, and end-to-end inference latency measurement. - Previous people management experience, including hiring, technical mentoring, team development, and performance management, combined with a willingness to remain actively involved in technical work.