Data Engineering Technical Lead

The Data Engineering Technical Lead will be part of the existing central Data Team, working to take the iiDENTIFii’s data practices to the next level. You will assist in refining our current platform while contributing to the development and implementation of our next-generation data platform using industry best practices such as the Medallion Architecture, best-fit collection of cloud-based data technologies, and automation through DevOps.

As technical lead, you will collaborate with leaders, senior engineers, and architects of the broader Engineering team to drive the technical data platform architecture roadmap, in alignment with the iiDENTIFii technology roadmap. Then you will lead the Data Engineers and Data Analysts to effectively implement the data platform architecture and build data and analytics use-cases to meet end-user requirements and demand.

In the Data team, we value delivering value on the quickest sustainable path, leaning on automation to make us more efficient & collaborative and detailed problem investigation and diagnosis. We operate as a shared services team to internal teams and stakeholders. We advocate and promote shared Data ownership and responsible usage of data through lean and practical data governance. These values guide our collaboration with our stakeholders to optimally meet their Analytics and Reporting needs and contribute to their strategic success because of the insights they access from iiDENTIFii’s Data 

ROLE TASKS AND RESPONSIBIITIES 

• Lead the design, orchestration, and implementation of robust data pipelines for batch and real-time processing.

• Ensure best practices in data engineering, monitoring, and observability are followed, utilising tools like Databricks, Azure Data Factory, Synapse or Airflow.

• Mentor junior team members and foster their growth into experts.

• Collaborate   with   executives   and   stakeholders   to   scale   data   platform   capabilities   and integrate them with other digital products.

QUALIFICATIONS AND EXPERIENCE

Leadership Experience: 

• 2+ years of experience leading teams, mentoring, and coaching, with a passion for leadership and team growth.

Data Engineering Expertise: 

• 5+ years of experience building modern, cloud-native data platforms, with deep hands-on expertise in:
  • Designing scalable and resilient pipelines for big data processing and real-time streaming using technologies like  Apache Spark  and Databricks.
  • Orchestrating data workflows using tools like Airflow, Databricks Jobs  Azure Synapse or Azure Data Factory.
  • Working with streaming platforms like Apache Kafka, RabbitMQ, or Azure Event Hubs for real-time data processing.
  • Developing metadata-driven pipelines for flexible and dynamic data workflows.
  • Strong   understanding   of  data   security,  governance,  metadata management, and data architecture.
  • Proficiency with  Linux,  Python  programming (object-oriented), and scripting for data automation.
  • Experience with DevOps, DataOps, and CI/CD pipelines, implementing best practices in version control, testing, and deployment.

Cloud Technology: 

• 3-5 years of experience with Microsoft Azure (Azure,   Azure   Data   Factory,   Databricks)   with   exposure   to   other   cloud platforms (AWS/GCP) being a plus.

Real-Time Processing:   

• Hands-on   experience   building   and   maintaining streaming architectures using Kafka, Event Hubs, or similar technologies.

BI & Visualization: 

• 2+ years working with tools like Power BI, Grafana, or other visualization platforms, focusing on secure and optimised deployments.

Monitoring & Observability:

• Familiarity   with   tools   like  Application Insights, Azure Monitor, and Grafana for tracking pipeline health, logging, and optimising resource usage.

Leadership Skills: 

• Proven ability to lead teams, drive performance, resolve conflicts, and align team.

ADDITIONAL SKILLS (BENEFICIAL)

• Data Model Design: Familiar with data lakes, data warehousing, and Kimball Dimensional Modelling principles.

• Data Development Lifecycle (DDLC): Experience guiding teams from requirements to deployment.

• Mentorship & Documentation: Ability to coach team members and create high-quality documentation to support knowledge transfer.

• DevOps   &   Security:   Experience   with   DevEx   in   Agile   environments   and   working knowledge of the ISO 27001 security framework.

CULTURE AND BENEFITS 

Why Join Us 

Our mission is to power innovators in remote biometric digital authentication, globally, and we’re well on our way there. You’ll join at a very exciting point in the journey, and your contribution over the next few years will directly and visibly impact iiDENTIFii’s global success.

Our team is incredibly friendly and collaborative. If you care about solving challenging

problems with people you enjoy being around, this is the place for you.

Benefits: 

Competitive salary offered.

Remote Working Conditions- Cape Town preferable.

Wellness perks.

Opportunities for career growth and development within the company.

Work-Life Balance is prioritized.

Diverse tech stack.