Data Engineer

About the Role

The role forms part of the Client Engagement Delivery — the team that builds and maintains the data foundation powering our client engagement and campaign activity. We design data assets and pipelines that deliver trusted, accurate data to Salesforce Marketing Cloud and Data360, giving CRM Architects, Specialist Client Engagement teams and Product Owners what they need to run targeted, data-driven campaigns and client journeys.

Responsibilities

• Design, build and maintain robust, scalable and efficient data pipelines and systems that meet the needs of our data environment
• Analyse business and system requirements to build and enhance data platforms
• Provide input to the Data Architect during solution design for new and existing data flows
• Contribute to the design and development of new applications and systems
• Support and maintain data infrastructure to keep it reliable and efficient

Requirements

• Grade 12 National Certificate / Vocational
• Bachelor's Degree in Information Technology - Computer Science or Information Technology
• Advanced SQL
• modelling principles and query optimisation
• Hands-on experience with batch processing and basic implementation of stream processing
• Proficiency in multiple programming languages
• able to build reusable components and optimise code
• ETL pipeline design
• with error handling and recovery mechanisms
• Deep knowledge of at least one cloud platform's data services
• Implementation of data lake, data warehouse or other data patterns
• Implementation of data quality and governance
• Ability to performance-tune data systems and pipelines, balancing cost against performance
• Advanced version control and CI/CD principles
• including automated testing
• Implementation of containerised data solutions
• Infrastructure as code
• creation and setup of workflow orchestration tools
• 5 years in Data Engineering or closely related field
• Leading or significantly contributing to data engineering projects
• Using advanced data engineering tools
• SQL, Python, Java, Apache Spark, Hadoop
• Cloud platforms
• AWS, Google Cloud or Azure
• and Big Data technologies
• Building and optimising data pipelines for financial data analytics
• Working with cross-functional teams to understand business needs and deliver data-driven solutions
• Engaging stakeholders
• communicating technical details and project progress to non-technical audiences
• Understanding of various financial products and services
• Awareness of market trends and their impact on data engineering
• Analytical Skills
• Communications Skills
• Interpersonal & Relationship management Skills
• Problem solving skills
• Clear criminal and credit record

Preferred Qualifications

• Honours Degree in Information Technology - Computer Science or Information Technology