Senior Python Engineer – Coding-Agent Safety Evaluation

About the role

We are looking for a senior software engineer to design and evaluate realistic coding‑agent benchmark tasks. The role blends production engineering, secure software development, repository validation, and rigorous testing to ensure agents operate safely and respect privacy and data‑integrity constraints.

Key responsibilities

• Build production‑like benchmark tasks using repositories, tests, configuration, documentation, and runtime scenarios.
• Create realistic engineering tasks such as bug fixes, feature additions, CI repairs, configuration migrations, integration updates, and runtime‑state fixes.
• Define utility, safety, and alignment requirements that verify correct task completion and protect system constraints.
• Develop visible and hidden test suites, and produce safe and unsafe reference solutions for controlled evaluation.
• Identify unsafe shortcuts (e.g., disabling tests, weakening assertions, leaking secrets) and package prompts, metadata, runnable repositories, evaluators, reference patches, scoring rubrics, and calibration notes.
• Analyze coding‑agent rollouts for utility completion and safety violations.
• Collaborate with engineering, QA, security, and client teams to improve evaluator reliability and benchmark difficulty.

Required profile

• Bachelor's or Master's degree in Computer Science or a related technical discipline.
• At least 8 years of hands‑on software engineering experience in tier‑one product companies or technology startups.
• Strong professional Python experience plus proficiency in at least one of Java, Go, or C++.
• Experience building and maintaining production software used by real customers.
• Solid software architecture, debugging, and performance‑optimization skills.
• Production‑engineering experience covering deployment pipelines, monitoring, logging, observability, metrics, distributed tracing, high availability, fault tolerance, scalability, and disaster recovery.
• Secure‑software experience identifying and fixing vulnerabilities such as injection, authentication/authorization issues, memory safety, race conditions, deserialization flaws, secrets management, and input validation.
• Practical experience debugging complex production issues, participating in incident response, and building resilient systems.

Required skills

• Python
• Java
• Go
• C++
• Production engineering
• Deployment pipelines
• Monitoring
• Logging
• Observability
• Metrics
• Distributed tracing
• High availability
• Fault tolerance
• Scalability
• Disaster recovery
• Secure software practices
• Injection prevention
• Authentication & authorization
• Memory safety
• Race‑condition mitigation
• Deserialization protection
• Secrets management
• Input validation
• Debugging
• Incident response

What we offer

• Remote contractor role for candidates located in Bangladesh, Egypt, Ghana, India, Indonesia, Nigeria, Turkey, or Vietnam.
• Start date around September 7 2026, lasting approximately 4–8 weeks.
• 8 hours per day with 6 hours overlap with Pacific Time.
• Compensation of $28 per hour (visible).