Job Summary
As a Data Engineer, you will be the backbone of our data strategy. You won't just be building pipelines; you will be designing the scalable infrastructure that powers our analytics, machine learning models, and business intelligence. We are looking for a mentor and a builder who thrives on optimizing complex data environments and ensuring 24/7 data reliability.
What you'll do:
- Architecture & Design: Design, build, and internalize large-scale batch and real-time data pipelines (ETL/ELT).
- Infrastructure Management: Optimize our data warehouse (e.g., Snowflake, BigQuery, Redshift) and lakehouse architectures for performance and cost.
- Data Governance: Implement rigorous data quality checks, monitoring, and alerting to ensure "one version of the truth."
- Tooling & Automation: Automate manual processes and improve developer productivity by implementing CI/CD for data workflows.
- Collaboration: Work closely with Data Scientists and Analysts to transform raw data into "analysis-ready" formats.
- Mentorship: Lead code reviews and provide technical guidance to junior members of the engineering team.
Your Expertise:
- Experience: 3-5+ years of experience in data engineering or backend software engineering.
- Coding: Mastery of Python or Scala/Java.
- SQL: Expert-level SQL (window functions, optimization, and complex joins are second nature to you).
- Big Data Tech: Experience with Spark, Flink, or Kafka.
- Orchestration: Hands-on experience with Airflow, Dagster, or Prefect.
- Cloud: Proficiency in AWS, GCP, or Azure data ecosystems.
- Modeling: Deep understanding of data modeling techniques (Star Schema, Snowflake Schema, Data Vault 2.0).
- Experience with dbt (data build tool).
- Knowledge of Infrastructure as Code (Terraform).
- Background in managing containerized environments (Docker/Kubernetes).
Soft Skills
- Pragmatism: You know when to build a "quick and dirty" fix and when to invest in a robust, long-term solution.
- Problem Solving: You enjoy deconstructing a broken pipeline like a puzzle.
- Communication: You can explain the "why" behind a technical architecture to non-technical stakeholders.