We are looking for a specialised Cloud Engineer to build, secure, and maintain the underlying infrastructure for modern data platforms. Unlike traditional cloud engineering roles focused on general application hosting, this position is dedicated to the unique requirements of data workloads — including large-scale storage, high-throughput networking, and specialised compute clusters for Spark and SQL.

You will bridge the gap between DevOps and Data Engineering, ensuring data scientists and analysts have a robust, secure, and automated platform for building pipelines and models. The role focuses on the core infrastructure of the data ecosystem, primarily within Databricks, AWS, and Azure environments.

Responsibilties

Platform Architecture: 

• Design and deploy scalable cloud infrastructure for data lakes and analytics platforms using Infrastructure as Code (Terraform)
  

Security & Governance: 

• Implement identity management (IAM / Entra ID), network security (Private Link / VPC), and governance controls (Unity Catalog)
  

• Build CI/CD pipelines for infrastructure and data products.
  
• Automate provisioning of compute resources and workspaces
  

Observability:

• Monitor cost, performance, and reliability of data workloads
  

Enablement:

• Create self-service infrastructure patterns for Data Engineers
  

Requirements

Requirements

• Minimum of 5 years professional cloud engineering experience
  
• Experience in data engineering and Databricks is highly desirable
  

Core Skills, Tools & Frameworks

• Terraform (modules, state, workspaces)
  
• CloudFormation, Bicep, or Crossplane advantageous
  

Containerisation & Orchestration:

• Docker and Kubernetes (EKS, AKS, or self-managed)
  

CI/CD & Version Control:

• Git, Azure DevOps, GitHub Actions
  

Scripting & Automation:

• Python and Bash
  
• DBX CLI, AWS CLI, Azure CLI, REST APIs
  

Cloud Security:

• Least Privilege, RBAC, encryption, secrets management
  

Cloud Networking:

• VNET/VPC design, Private Link, DNS, Firewalls
  

Data Platform Architecture:

• Medallion architecture, Delta Lake, Data Vault
  

Observability:

• Cost management, logging, alerting
  

Data Workload Understanding:

• PySpark, SQL, dbt, Spark Structured Streaming
  

Azure Platform Skills:

Databricks Platform & Infrastructure:

• Workspace automation via Terraform
  
• Unity Catalog configuration
  
• Cluster policies and instance profiles
  
• Private Link, VNet Injection, IP Access Lists
  
• Workflows, DLT, Airflow integrations
  
• MLflow, Mosaic AI, Vector Search infrastructure
  

General Cloud & DevOps:

• AWS Solutions Architect (Associate/Professional)
  
• Azure Solutions Architect Expert
  
• Terraform Associate
  
• Certified Kubernetes Administrator
  

Data & Platform:

• Databricks Data Engineer Professional
  
• AWS Data Engineer Associate
  
• Azure Data Engineer Associate (DP-203)