Software Engineer III - Senior Data Engineer

Job description

We are seeking a Senior Data Engineer to lead the design, build, and operation of scalable data pipelines and domain-aligned data products on Databricks using Python . This role is accountable for delivering governed, discoverable, high-quality datasets and services aligned to data mesh principles (domain ownership, product thinking, and federated governance), enabling analytics and ML use cases with strong reliability, security, and performance. Key Responsibilities: - Architect and deliver end-to-end data engineering solutions on Databricks (Spark/Delta Lake) using Python and SQL. - Build data products aligned to data mesh principles: clear contracts, documentation, metadata, lineage, access controls, and measurable SLAs/SLOs. - Develop and optimize batch and streaming pipelines with strong data quality checks and observability. - Implement curated data layers and models (e.g., bronze/silver/gold as appropriate), including partitioning, file sizing, clustering strategies, and query optimization. - Establish engineering standards: code quality, unit/integration testing, CI/CD, release governance, and reusable frameworks. - Partner with governance, security, and platform teams to e...