Location: Bedfordview, Johannesburg | Hybrid
Employment Type: Permanent | Full-Time
Industry: Data Engineering | Applied AI | Financial Services

WatersEdge Solutions is partnering with an established financial services business to find a technically strong and curious Data and AI Engineer who enjoys building reliable data solutions and exploring how artificial intelligence can create practical business value.

This opportunity combines hands-on Data Engineering with Applied AI, offering the chance to work across modern Microsoft technologies while contributing to the development and ongoing operation of a governed AI platform.

The role will initially be approximately two-thirds Data Engineering and one-third Applied AI, with the AI component expected to grow as adoption increases.

We're looking for someone with strong fundamentals in SQL Server, Microsoft Fabric and Python, a genuine interest in AI, and the confidence to take ownership of technical solutions from development through to production support.

About the Role

As a Data and AI Engineer, you'll join a collaborative Data and AI team responsible for developing, maintaining and improving the data infrastructure supporting business reporting, analytics and AI applications.

You'll work within a Microsoft-centred environment spanning SQL Server, Azure SQL, Microsoft Fabric, Python, Power BI, SSRS and Azure DevOps.

Your responsibilities will include building reliable data pipelines, transforming and validating datasets, supporting data models and contributing to AI-powered solutions.

You'll also help operate an internally governed AI platform, supporting data ingestion, retrieval indexes, automation and AI use cases such as search, summarisation, classification and information extraction.

This is a production-focused engineering role, where data quality, security, documentation and responsible AI practices are essential.

The business values strong technical foundations and is willing to develop your expertise across the broader technology stack. However, candidates must already have practical experience with SQL Server, Microsoft Fabric and Python.

Key Responsibilities

• Data Engineering & Pipeline Development

  • Design, develop and maintain batch and incremental ETL/ELT pipelines.

  • Build and support data integration solutions across SQL Server, Azure SQL and Microsoft Fabric.

  • Write, optimise and troubleshoot complex T-SQL queries, stored procedures, views and transformation logic.

  • Apply SQL techniques including joins, aggregations, Common Table Expressions (CTEs) and multi-step transformations.

  • Develop and maintain Microsoft Fabric pipelines, Dataflow Gen2 flows and Lakehouse/Warehouse solutions.

  • Support data warehousing, staging processes and incremental loading.

  • Contribute to dimensional data modelling and Medallion Architecture across bronze, silver and gold layers.

  • Implement metadata-driven and idempotent data processing patterns.

  • Manage pipeline orchestration, scheduling, monitoring, logging and alerting.

  • Investigate and resolve data pipeline failures and production incidents.

  • Implement data validation, reconciliation, row-count and parity checks.

  • Ensure data is accurate, consistent and reliable for reporting, analytics and AI workloads.Applied AI & Automation

• Applied AI & Automation

  • Develop and maintain curated datasets used by AI applications.

  • Support retrieval indexes, data interfaces and AI platform integrations.

  • Assist with the operation of an internally governed AI platform, including ingestion, indexing, configuration and access controls.

  • Develop Python scripts for data preparation, automation and integrations.

  • Support AI workflows involving embeddings, indexing and evaluation.

  • Contribute to practical AI use cases involving search, summarisation, classification, extraction and workflow automation.

  • Evaluate AI-generated outputs against defined quality and accuracy criteria.

  • Apply human-review checkpoints to AI-assisted processes.

  • Maintain traceable records of AI evaluations and results.

  • Follow responsible AI practices, including data minimisation, anonymisation and appropriate security controls.

• Governance, Development & Production Support

  • Use Git and Azure DevOps for source control and collaborative development.

  • Participate in peer reviews and structured deployment processes.

  • Support controlled releases across development, UAT and production environments.

  • Maintain technical documentation, source-to-target mappings, runbooks and support procedures.

  • Apply data security, confidentiality and governance standards.

  • Ensure sensitive business information is handled appropriately and never exposed to unapproved AI tools.

  • Collaborate with analysts, technical colleagues and business stakeholders to deliver reliable solutions.

  • Take ownership of assigned data and AI assets from development through deployment and ongoing support.

What You'll Bring

Essential Requirements

• Minimum 3 years of hands-on experience in Data Engineering, ETL/ELT, Analytics Engineering, Database Development, Applied AI or a closely related production-oriented technical role.

• Strong practical experience with SQL Server and T-SQL.

• Ability to write and troubleshoot SQL queries involving joins, aggregations, CTEs and multi-step transformations.

• Understanding of SQL query performance and optimisation.

• Demonstrable hands-on Microsoft Fabric experience. You must have built, supported or substantially prototyped at least one of the following:

  • Microsoft Fabric pipelines

  • Fabric Lakehouse or Warehouse solutions

  • Dataflow Gen2 flows

  • Fabric notebooks

• Practical understanding of data warehousing, staging, incremental loading and dimensional modelling.

• Working Python knowledge, including the ability to read, write and modify scripts for data preparation, automation or integrations.

• Practical experience using AI or LLM-assisted tools in real work, with the ability to critically evaluate their outputs.

• Familiarity with Git or similar source-control tools.

• Understanding of structured development, peer review and controlled deployment practices.

• Strong awareness of data quality, confidentiality, security and responsible data handling.

• Good analytical thinking and troubleshooting ability.

• Strong attention to detail and a commitment to producing reliable, maintainable solutions.

• Ability to work independently while collaborating effectively with a technical team.

• Willingness to learn new technologies and take ownership of technical responsibilities.

Important: A formal Machine Learning or Data Science background is not required. The business is willing to develop your Applied AI expertise, provided you already have the required SQL, Microsoft Fabric and Python foundations.

Nice to Have

• Advanced Python or PySpark experience.

• Microsoft Fabric Dataflow Gen2, Mirroring or Change Data Capture (CDC).

• Fabric deployment pipelines.

• Power BI semantic modelling and DirectLake.

• SSRS development.

• API integrations and enterprise data ingestion.

• Retrieval-Augmented Generation (RAG).

• Vector search, semantic search and embeddings.

• Azure AI Services.

• AI evaluation frameworks, prompt engineering or agent tooling.

• MLOps exposure.

• Azure DevOps CI/CD pipelines.

• Microsoft DP-700 certification or another relevant Microsoft certification.

• Relevant tertiary qualification in Computer Science, Information Systems, Engineering or a related discipline.

• Experience working in financial services or another regulated environment.

• Understanding of POPIA and sensitive data handling requirements.

What's On Offer

• Permanent hybrid opportunity based in Bedfordview, Johannesburg.

• Opportunity to work across both Data Engineering and Applied AI.

• Hands-on exposure to Microsoft Fabric, SQL Server, Azure SQL, Python and Azure DevOps.

• Involvement in the development and operation of a governed AI platform.

• Opportunity to develop practical experience with AI automation, retrieval systems and emerging AI technologies.

• A production-focused environment with established engineering and governance standards.

• Training and development across the broader technology stack.

• The opportunity to take increasing ownership of data and AI solutions.

• A collaborative team environment that values technical curiosity and continuous learning.

Company Culture

You'll be joining a team that values curiosity, accountability, integrity and quality.

The environment is collaborative and technically focused, with a strong emphasis on doing things properly. Source control, peer reviews, documentation, data validation and responsible AI practices are part of everyday work.

The team is looking for someone who asks good questions, welcomes feedback and takes responsibility for delivering reliable solutions.

You'll be encouraged to explore new technologies and develop your technical skills, while maintaining the discipline required to work with sensitive information in a regulated environment.

This role is particularly well suited to someone who enjoys building things, solving problems and taking ownership from initial development through to ongoing production support.

Please Note: If you have not been contacted within 10 working days, consider you application unsuccessful.