Senior Quality Engineer II (AI & Data)
Spot the Problem
- Find what's costing you interviews at Salla
- Get AI-rewritten bullet points
- Download Gulf-ready CV
60 seconds. $3.99 one-time.
Role Overview:
As a Senior Quality Engineer on the AI Team, you will be responsible for defining and executing the quality strategy for our AI-driven products. You will work at the intersection of data engineering and frontend application development, ensuring that our machine learning models are reliable, our data pipelines are accurate, and the final user experience is seamless.
Responsibilities:
• Data Pipeline Validation: Test the integrity of data ingestion and transformation processes to ensure models are trained on high-quality datasets.
• API & Model Testing: Conduct deep API testing using tools like APIDog to validate model responses, handling complex edge cases such as zero-value parameters or unauthorized access.
• Generative Model Output Accuracy: Define and implement evaluation frameworks to measure the quality of generative AI outputs — including relevance, factual accuracy, coherence, and hallucination rate — using techniques such as LLM-as-a-judge, human-in-the-loop scoring, and automated benchmark suites.
• AI Agent Response Validation: Design test strategies to assess the correctness and reliability of agentic AI responses, covering tool-call accuracy, multi-step reasoning integrity, goal completion rate, and graceful failure handling for out-of-scope or adversarial inputs.
• Security Testing: Proactively identify vulnerabilities like request interception and price manipulation to safeguard the system including prompt injection and jailbreak attempts specific to generative AI surfaces.
• UI Automation: Build and maintain scalable automation suites using low-code tools like mabl, aiming for high Mabl Test Coverage %.
• AI Tooling Advocacy: Champion the use of AI assistants like Jira Rovo for requirement analysis and test case generation to increase team velocity.
• Root Cause Analysis (RCA): Lead the RCA process for production bugs and developer-rejected tickets to continuously improve the "Definition of Done". Requirements
• Senior Experience: 8+ years of relevant work experience
• Data Proficiency: Strong SQL skills and experience testing data-heavy applications or ML models.
• Automation Mastery: Proficiency in automation tools (e.g., Selenium, mabl, Playwright)
• Security Mindset: Understanding of OWASP principles and practical experience in security testing (request interception, label tracking).
• Domain Knowledge: Experience in e-commerce and critical customer-flow analysis is highly preferred.
Requirements
- •8+ years of relevant work experience
- •Strong SQL skills and experience testing data-heavy applications or ML models
- •Proficiency in automation tools (e.g., Selenium, mabl, Playwright)
- •Understanding of OWASP principles and practical experience in security testing
- •Experience in e-commerce and critical customer-flow analysis is highly preferred
Responsibilities
- •Data Pipeline Validation
- •API Model Testing
- •Generative Model Output Accuracy
- •AI Agent Response Validation
- •Security Testing
- •UI Automation
- •AI Tooling Advocacy
- •Root Cause Analysis (RCA)
Browse Similar
- Find what's costing you interviews at Salla
- Get AI-rewritten bullet points
- Download Gulf-ready CV
60 seconds. $3.99 one-time.
Salla is an e-commerce platform that enables businesses to create and manage online stores. It provides tools for selling products online in Saudi Arabia.
Visit WebsiteView all jobs