Location: Bedfordview, Johannesburg | Hybrid
Employment Type: Permanent | Full-Time
Industry: Data Engineering | Applied AI | Financial Services

WatersEdge Solutions is partnering with a business in the financial services sector to find an experienced Data and AI Engineer who combines strong production Data Engineering expertise with an interest in practical, responsible AI development.

This opportunity is ideal for a hands-on Data Engineer who enjoys building reliable data infrastructure, solving complex technical challenges and exploring how AI can create meaningful business value.

The role will initially focus on approximately two-thirds Data Engineering and one-third Applied AI, with the AI component expected to grow over time.

Important: This is a dedicated engineering position, not a Business Intelligence, reporting or Data Analyst role. Candidates must have at least 3 years of professional, hands-on experience in a dedicated Data Engineering role.

About the Role

You'll work within a Microsoft-centred technology environment, using SQL Server, Azure SQL, Microsoft Fabric, Python and Azure DevOps to develop, maintain and improve production data solutions.

Your primary focus will be designing and supporting robust data pipelines, data transformations, orchestration processes, dimensional models and data quality controls.

Alongside your core engineering responsibilities, you'll contribute to the development of an internally governed AI platform, supporting trusted datasets, retrieval indexes, automation and practical AI applications.

This is a role for someone who understands what it takes to build and maintain production-ready data solutions, takes ownership of technical deliverables and is keen to grow their expertise in Applied AI.

Key Responsibilities

Data Engineering & Pipeline Development

• Design, develop and maintain production-grade batch and incremental ETL/ELT pipelines.

• Build and support data integration solutions across SQL Server, Azure SQL and Microsoft Fabric.

• Write, optimise and troubleshoot complex T-SQL queries, stored procedures, views, CTEs and transformation logic.

• Develop and maintain Microsoft Fabric pipelines, Dataflow Gen2 flows, Lakehouse and Warehouse solutions.

• Implement data warehousing concepts, staging processes and incremental loading strategies.

• Design and maintain dimensional data models.

• Apply Medallion Architecture across bronze, silver and gold data layers.

• Manage pipeline orchestration, scheduling, monitoring, logging and incident resolution.

• Implement data validation, reconciliation and quality controls to ensure reliable data processing.

Applied AI & Automation

• Develop curated datasets and data interfaces to support AI workloads.

• Build and maintain retrieval indexes for AI-driven applications.

• Support AI platform operations, including data ingestion, indexing, configuration and access controls.

• Develop Python scripts and automation solutions for data preparation, integrations and AI-related workflows.

• Contribute to practical AI use cases involving search, summarisation, classification, extraction and workflow automation.

• Use modern AI and LLM-assisted tools while critically evaluating outputs.

• Apply responsible AI practices, including appropriate human review, evaluation and data security controls.

Development, Deployment & Production Support

• Use Git and Azure DevOps for source control, peer reviews and controlled deployments.

• Troubleshoot production pipeline failures and resolve data integration issues.

• Maintain technical documentation, pipeline designs, runbooks and support procedures.

• Support the ongoing performance, reliability and maintainability of data solutions.

• Take ownership of engineering deliverables from development through deployment and ongoing maintenance.

• Collaborate with technical teams and business stakeholders to deliver scalable, well-governed solutions.

What You'll Bring

Essential Requirements

• Minimum 3+ years of professional, hands-on Data Engineering experience in a dedicated Data Engineering role. This is a strict, non-negotiable requirement.

• Proven experience designing, developing, maintaining and supporting production ETL/ELT pipelines and data integration solutions.

• Experience primarily focused on Data Engineering rather than reporting, dashboards, Business Intelligence or data analysis.

• Strong SQL Server and T-SQL expertise, including complex queries, stored procedures, data transformations and performance optimisation.

• Demonstrable hands-on Microsoft Fabric experience, including building, supporting or substantially prototyping pipelines, Lakehouse/Warehouse solutions, Dataflow Gen2 flows or Fabric notebooks.

• Solid understanding of data warehousing, staging, incremental loading and dimensional modelling.

• Working Python skills for data preparation, automation and integrations.

• Practical experience using modern AI or LLM-assisted tools, with the ability to critically assess generated outputs.

• Familiarity with Git or equivalent source-control practices.

• Strong understanding of data quality, security, confidentiality and production support.

• Ability to independently manage engineering deliverables through development, deployment, documentation and maintenance.

• Strong analytical thinking, troubleshooting ability and attention to detail.

• A proactive, accountable approach to technical problem-solving.

Please note: Experience focused primarily on Power BI reporting, dashboard development, Business Intelligence or data analysis will not satisfy the minimum Data Engineering experience requirement.

Nice to Have

• Advanced Python or PySpark experience.

• Microsoft Fabric Dataflow Gen2, Mirroring or Change Data Capture (CDC).

• Fabric deployment pipelines.

• Power BI semantic modelling and DirectLake.

• SSRS development experience.

• API integrations and enterprise data ingestion.

• Retrieval-Augmented Generation (RAG).

• Vector search and embeddings.

• Azure AI Services.

• AI evaluation frameworks, agent tooling or MLOps.

• Microsoft DP-700 certification or another relevant technical certification.

• Relevant tertiary qualification or equivalent practical experience.

• Experience working in financial services or another regulated environment.

What's On Offer

• Permanent, full-time opportunity with a hybrid working arrangement in Bedfordview, Johannesburg.

• The opportunity to combine established Data Engineering responsibilities with emerging Applied AI technologies.

• Hands-on exposure to Microsoft Fabric, SQL Server, Azure SQL, Python and Azure DevOps.

• Involvement in practical AI initiatives within a governed, production-focused environment.

• The chance to contribute to AI solutions while maintaining ownership of critical production data infrastructure.

• Opportunities for continued technical development as the Applied AI component of the role grows.

• A collaborative environment that values engineering quality, accountability and continuous improvement.

Company Culture

You'll be joining a technically focused, collaborative environment where curiosity, ownership and attention to detail are highly valued.

The team prioritises reliable, maintainable and well-governed solutions over quick fixes. Engineers are encouraged to challenge assumptions, contribute ideas and take responsibility for the quality of their work.

This is an excellent opportunity for someone who enjoys the practical side of Data Engineering and wants to develop their Applied AI capabilities without moving away from hands-on engineering.

Please Note: If you have not been contacted within 10 working days, consider you application unsuccessful.