Lead Data Quality Engineer

Job Purpose

The Chief Directorate Department at Emirates Airlines oversees strategic planning, operational excellence, and innovation across various divisions. It plays a key role in driving the airline's growth, enhancing customer experience, and ensuring seamless global operations.

As a Data Quality & Automation Engineering Lead you will be a fully participating member of a cross-functional team working autonomously on technology development and problem resolution in the Enterprise Data & Analytics space. The role involves leading and championing quality practices along with designing and implementing data quality and automation platforms. They will also provide support and maintenance to technical analytics solutions and products that support Emirates Airlines and the Emirates Group businesses.

In this role you will:

• Work with product owners, analysts, software engineers and architects to drive understanding of technical landscape and context of deliveries & refine complex functional and non-functional requirements and translate to acceptance tests.
• Support in building the technical design by contributing in the analysis of data application and ensuring quality related features are baked into the solution.
• Work on and guide engineers on problems of diverse scope where analysis of data requires evaluation of identifiable factors and select methods to validate and automate the solution.
• Drive & mentor creation of test strategies and test plans validating the core business problem and translating them into tests around code functionality, data quality, performance, and security.
• Build components & liaise with teams across the organization to manage cross dependencies for generating / mocking test data for exploratory analysis and running tests.
• Build platforms for automated tests / jobs of moderate to high scope and complexity across all stages of the data pipeline. Conduct code reviews to check for good coding & design practices and adherence to published coding standards and guidelines.
• Build and enhance data quality rules and platforms for observability across the data pipeline.
• Perform and support team debugging complex issues, resolve blockers.
• Advocate and perform data analysis activities such as source system analysis, data modelling, data dictionary collection, data profiling and source-to-target mapping ensuring delivery on business needs.
• Advocate and participate in updating data inventories and registries as required to keep metadata and data lineage up-to-date, following agreed Data Governance standards, guidelines and principles.

Qualification

To be considered for the role, you must meet the below requirements:

• 5+ years of experience in Information Technology Data Engineering with focus on quality & automation.
• Degree or Honours (12+3 or equivalent) in a relevant field such as Computer Science, Computational Mathematics, Computer Engineering or Software Engineering. Specialization or electives in a Data & Analytics field (e.g. Data Warehousing, Data Science, Business Intelligence)
• Minimum 4+ years of testing, automation and support experience in analytics applications such as Data Lake and Data Warehouse (preferably using the Big Data stack and Microsoft Azure cloud infrastructure.
• Worked in a team leadership role building culture and practices around quality and developing trustworthy data assets.
• Seasoned in building data quality platforms for highly complex data architectures performing automated checks running in a CI and in production for data observability.
• Experienced in leading team to build enablers / practices to identify quality issues across complex data pipelines running on big data technologies and defining rules for validating health of the data.
• Proficient in at least 1 language (Scala / Python) for building ETL jobs in a distributed computing environment.
• Strong SQL querying skills is required.
• Experienced in a wide variety of testing methods and tools covering functional, performance, security tests across individual jobs, pipelines and end to end across enterprise.
• Good understanding of data architecture concepts such as data modelling, Big Data storage, and dimensional modelling.
• Good experience of working with jobs in data pipelines and define metrics / measures to ensure correctness of the data.
• Experience with batch and real-time data ingestion/integration tools and technologies handling massive quantities of data (structured and unstructured).

Knowledge/Skills:

• Can collaborate with multiple teams / programs to deliver solutions interactively.
• Experienced with leading a team of engineers to achieve product and business goals and displays a positive attitude.
• Can mentor engineers and provide technical leadership in designing solutions.
• Strong ability to conduct data analysis (e.g. source system identification, data dictionary / metadata collection, data profiling, source-to-target mapping).
• Self-starter and has passion for exploring and learning new technologies, especially those in the Enterprise Data & Analytics space.
• Team player; able to collaborate with others to remove blockers, solve complex design problems and debug/resolve issues.
• Strong oral and written communication skills with demonstrable experience communicating cross-functionally with architects and product owners in formal and informal settings.
• Strong organizational, planning, and follow-up skills and ability to hold others accountable.
• Strong people management skills with a proven ability to hire and grow teams of Data Quality engineers.
• Exposure to Spark is highly desirable.
• Airline industry experience is nice-to-have.

Key Technologies/Tools

Big Data & distributed processing: Spark, Hadoop (HDFS, Hive, H-Base, Oozie), Airflow, Apache Nifi, Azure (ADLS, DataBricks, Azure Data Factory), Elasticsearch, AVRO / PARQUET file formats. Data Analysis, Modelling and Reporting: Snowflake, SQL, Data Vault 2.0, MicroStrategy, Power BI. Cloud Technologies: Microsoft Azure and Cloudera technology stacks. Integration and Messaging: Streaming (e.g. Spark Streaming), SnapLogic, TIBCO, Kafka. CI/CD: GIT, Bitbucket, Jenkins, Azure DevOps, Kubernetes, docker, SonarQube, Gatling. Languages: Scala, Python.

At Emirates, we're committed to providing our employees with opportunities to grow and develop in their careers. so if you're looking for a challenging and rewarding opportunity, apply today and join our team!

Salary & benefits

Join us in Dubai and enjoy an attractive tax-free salary and travel benefits that are exclusive to our industry, including discounts on flights and hotels stays around the world. Find out what it’s like to live and work in our fast-paced, cosmopolitan home city in the Dubai Lifestyle section of our website www.emirates.com/careers