We are seeking an experienced Technology Specialist – Kafka to design, administer, automate, secure, and optimise enterprise Kafka platforms supporting mission-critical banking applications.
This role is responsible for ensuring the stability, scalability, resilience, and security of Kafka environments while driving automation, DevOps best practices, and cloud adoption.
The successful candidate will combine deep Kafka platform engineering expertise with Linux administration, Infrastructure-as-Code, observability, and cloud engineering to deliver highly available event streaming infrastructure that supports modern banking systems and real-time data integration.
Key Responsibilities
Kafka Platform Administration
- Install, configure, upgrade, patch and maintain enterprise Kafka clusters.
- Administer Confluent Platform and/or Confluent Cloud environments.
- Manage brokers, topics, partitions, replication and consumer groups.
- Optimise Kafka cluster performance, throughput and latency.
- Perform capacity planning and scaling of Kafka infrastructure.
- Troubleshoot broker failures, replication issues and performance bottlenecks.
- Support production incidents and perform root cause analysis.
Platform Engineering
- Design highly available Kafka platforms across multiple environments.
- Implement resilient architectures for business-critical messaging workloads.
- Configure multi-broker clusters with appropriate replication and fault tolerance.
- Support disaster recovery and high availability strategies.
- Maintain platform documentation and operational runbooks.
DevOps & Automation
Develop and maintain automation to reduce manual operational effort.
Responsibilities include:
- Infrastructure as Code
- Platform provisioning
- Automated deployments
- Configuration management
- CI/CD integration
- Environment standardisation
Typical technologies include:
- Terraform
- Ansible
- Python
- Bash
- Azure DevOps
- Git
Linux Platform Administration
Provide specialist Linux administration supporting Kafka infrastructure.
Responsibilities include:
- Performance tuning
- Storage optimisation
- Network troubleshooting
- Service management
- Operating system maintenance
- Capacity management
- Log analysis
Kafka Security
Design and maintain secure Kafka environments.
Responsibilities include:
- TLS/SSL encryption
- SASL authentication
- ACL administration
- Certificate lifecycle management
- Identity integration
- Secure client connectivity
- Compliance with enterprise security standards
Cloud Engineering
Support Kafka deployments across cloud platforms, with a focus on Microsoft Azure.
Responsibilities include:
- Azure networking
- Virtual Machines
- Managed services
- Storage
- Identity integration
- Hybrid deployments
- Cloud security
- Platform optimisation
Experience with Confluent Cloud is highly advantageous.
Monitoring & Observability
Build comprehensive monitoring and operational visibility across Kafka platforms.
Responsibilities include:
- Health monitoring
- Performance dashboards
- Alerting
- Capacity monitoring
- Log aggregation
- Distributed tracing
- Incident diagnostics
Typical tools include:
- Prometheus
- Grafana
- ELK Stack
- Splunk
- Azure Monitor
Incident & Problem Management
- Support production platforms within SLA.
- Lead technical investigations.
- Perform root cause analysis.
- Recommend permanent fixes.
- Participate in major incident resolution.
- Drive platform stability improvements.
Collaboration
Work closely with:
- Platform Engineers
- DevOps Engineers
- Infrastructure Teams
- Cloud Engineers
- Security Teams
- Application Developers
- Architecture Teams
- Operations Teams
Minimum Requirements
Experience
- 6+ years' experience in Systems Engineering, Platform Engineering or Infrastructure Engineering.
- 4+ years' hands-on Kafka administration.
- Enterprise production support experience.
- Experience supporting highly available distributed systems.
- Experience automating infrastructure and operational processes.
- Experience working within DevOps environments.
Required Technical Skills
Kafka
- Apache Kafka
- Confluent Platform
- Confluent Cloud
- Broker administration
- Topic management
- Consumer groups
- Partition management
- Replication
- Cluster upgrades
- Performance tuning
- Capacity planning
Automation
- Python
- Bash
- Terraform
- Ansible
- CI/CD Pipelines
- Azure DevOps
- Git
Operating Systems
- Linux Administration
- Shell scripting
- Performance tuning
- System troubleshooting
Cloud
- Microsoft Azure
- Azure networking
- Azure infrastructure
- Azure Monitor
- Hybrid cloud environments
Kafka Security
- TLS/SSL
- SASL
- ACLs
- Certificates
- Encryption
- Authentication
- Authorisation
Monitoring & Observability
- Prometheus
- Grafana
- ELK Stack
- Splunk
- Azure Monitor
- Log analysis
- Alerting
- Performance monitoring
Preferred Experience
- Banking or Financial Services experience.
- Confluent Cloud implementation experience.
- Enterprise messaging platforms.
- Event-driven architecture.
- Kubernetes or container platforms.
- Docker.
- Site Reliability Engineering (SRE).
- Platform Engineering.
- Infrastructure as Code at enterprise scale.
Nice-to-Have Certifications
- Confluent Certified Developer or Administrator
- Microsoft Azure Administrator
- Microsoft Azure Solutions Architect
- Azure DevOps Engineer
- Red Hat Certified System Administrator (RHCSA)
- Terraform Associate
- Kubernetes Certifications (CKA/CKAD)
Ideal Candidate Profile
The ideal candidate is a senior Platform or Systems Engineer with deep expertise in Apache Kafka and Confluent Platform, capable of administering large-scale event streaming platforms in enterprise environments.
They should have strong Linux administration skills, experience automating infrastructure through Terraform, Ansible, Python, and CI/CD pipelines, and a solid understanding of Kafka security, cloud infrastructure (particularly Azure), and modern observability practices. Experience within a regulated environment such as banking or financial services would be highly advantageous.
