Cloud Data Engineer

Location: Gauteng, Johannesburg Metro, Johannesburg

Job Type: Contract, Full-Time

Primary Industry: Banking and Finance

Job Description

The Cloud Data Engineer is responsible for designing, developing, and maintaining scalable data pipelines and cloud-based data solutions to support business intelligence and analytics within the banking and finance sector. This role involves working with large volumes of data, ensuring data integrity, security, and availability while optimising cloud infrastructure and adhering to compliance standards.

Key Responsibilities

• Staging design,pipelines,storage optimisation.
• Design, build, and maintain robust and scalable cloud-based data pipelines and architectures.
• Develop and implement ETL (Extract, Transform, Load) processes to efficiently process data from various sources.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver actionable solutions.
• Optimise data storage and retrieval performance using cloud-native technologies and best practises.
• Ensure data quality, security, and compliance with relevant regulatory frameworks and internal policies.
• Monitor, troubleshoot, and resolve issues related to data pipelines, databases, and cloud infrastructure.
• Create and maintain technical documentation for data workflows, architectures, and processes.
• Participate in code reviews and contribute to continuous improvement of development standards and practises.
• Stay informed on emerging cloud data technologies and recommend improvements or innovations to existing systems.

Required Qualifications

• Spark, cloud storage/computer, security.
• Proven experience as a Data Engineer or Cloud Data Engineer, preferably within the banking and finance sector.
• Strong knowledge of cloud platforms such as Microsoft Azure, Amazon Web Services, or Google Cloud Platform.
• Proficiency in programming languages including Python, Java, or Scala.
• Experience with data pipeline and workflow management tools such as Apache Airflow or similar.
• Hands-on experience with big data technologies such as Apache Spark, Hadoop, or equivalent.
• Solid understanding of relational and non-relational databases, including SQL and NoSQL systems.
• Familiarity with containerisation and orchestration technologies such as Docker and Kubernetes.
• Comprehensive understanding of data security principles and regulatory compliance requirements pertinent to financial data.

Education

• Bachelor’s degree in Computer Science, Information Technology, Engineering, Mathematics, or a related discipline.
• Relevant certifications in cloud technologies or data engineering are advantageous.

Experience

• Minimum of three years’ experience in a data engineering role with a focus on cloud environments.
• Experience working within the banking, finance, or a similarly regulated industry is preferred.
• Proven track record of delivering data solutions that support business analytics and decision-making.

Knowledge and Skills

• Excellent analytical and problem-solving skills.
• Strong communication skills, capable of explaining technical concepts to non-technical stakeholders.
• Ability to work independently as well as collaboratively in a team-oriented environment.
• Attention to detail and commitment to delivering high-quality work.
• Understanding of agile development methodologies and version control systems.

Preferred Qualifications

• Advanced certifications such as Microsoft Certified: Azure Data Engineer Associate or equivalent.
• Experience with financial services data standards and frameworks.
• Knowledge of machine learning frameworks and integration with data pipelines.

Working Conditions

• Full-time contract position based in Johannesburg, Gauteng.
• Standard office hours with potential flexibility depending on project requirements.
• Occasional requirement to engage in remote work or attend meetings outside regular hours.
• Work environment adheres to all health and safety regulations relevant to the banking and finance sector.