Data Engineering Lead – Data Science and AI
DVT
Centurion , Gauteng, South Africa
Salary not specified - Contract
Job Description
About the Role
DVT is a leading technology consulting and software engineering company delivering innovative solutions across Africa and internationally. We partner with clients to solve complex business challenges through software engineering, cloud platforms, data, artificial intelligence, and digital transformation. Our teams combine deep technical expertise with a strong consulting mindset to create measurable business value.
Responsibilities
- Define and maintain the target data engineering architecture for Data Science and AI workloads, including data preparation, model deployment, GenAI applications and agentic workflows.
- Design the architecture and integration patterns for bringing data from AWS into the Azure Data Science environment, working with IT, Data Platform, Cloud and Security teams.
- Own data engineering delivery, building reusable pipelines, transformations and curated datasets, and standardised frameworks for ingestion, transformation, orchestration and publishing.
- Define the approach to reusable features and data products, including feature-store capabilities, lineage, versioning and monitoring.
- Partner with Data Scientists and Engineers to establish production-grade MLOps and LLMOps capabilities, including model deployment, versioning, monitoring and retraining.
- Provide the data engineering foundations for Generative AI and agentic AI solutions, including ingestion, chunking, indexing and governance of enterprise information.
- Ensure data and AI solutions align with governance, privacy, responsible AI, security and data quality standards.
- Provide technical leadership to Data Engineers supporting the Data Science portfolio, and translate the roadmap into a prioritised engineering and architecture plan.
- Establish monitoring, operational support, incident management and cost-optimisation practices, and act as primary technical liaison with architecture, infrastructure, cybersecurity and governance teams.
Requirements
- Proven, senior-level experience (7+ years) in data engineering, architecture, and cloud platforms.
- Degree in Computer Science, Information Systems, Data Engineering, Software Engineering, Mathematics or a related field.
- Strong practical experience with Microsoft Azure data and AI services (e.g. Azure Data Factory, Azure Functions, Azure Storage, Azure Machine Learning), and experience integrating data across AWS and Azure environments.
- Experience building and operating production data pipelines (batch, incremental, streaming or API-based), with data orchestration, automated testing, monitoring and deployment practices.
- Experience supporting Data Science, machine learning or AI teams, including implementing MLOps or model deployment capabilities.
- Strong Python and SQL skills, with experience in data modelling and transformation frameworks.
- Experience with Git-based source control, CI/CD pipelines and Infrastructure as Code.
- Experience working with data governance, information security and enterprise architecture teams, and leading technical delivery or coordinating engineers.
- Experience with Microsoft Fabric, Synapse Analytics or Databricks.
- Experience with AWS services such as S3, Glue, Athena, Redshift, Lambda or SageMaker.
- Experience building or supporting feature stores, model registries and experiment-tracking platforms.
- Experience with containerisation, orchestration technologies, and relational, document, vector or analytical databases.
- Exposure to FinOps and cloud cost optimisation practices.
- This is a hands-on Data Engineering and Cloud Architecture leadership role embedded within the Data Science and AI function, not a pure infrastructure or platform-operations role. The focus is establishing the data architecture, pipelines and engineering foundations that enable Data Science, ML and AI delivery.
Preferred Qualifications
- Microsoft Azure and/or AWS architecture or engineering certifications advantageous.