Senior Data Engineer - Full Stack

Job description

Role Summary We are seeking a highly skilled Senior Data Engineer – Full Stack to build and maintain internal tools, automation frameworks, and workflows that enhance the efficiency, reliability, and scalability of our data and machine learning platforms. This role will work closely with Data Engineers, Data Scientists, and ML Engineers to streamline operations across the data lifecycle. Total Exp: 8+Years: Location; New York City(Hybrid) Key Responsibilities: - Design and develop CLI tools, scripts, and internal utilities to automate repetitive tasks across the data platform, including: - Pipeline execution and orchestration - Data governance workflows - Metadata synchronization - Environment setup and configuration - Test harness development Automate workflows on Databricks, including: - Job deployment and scheduling - Environment provisioning - MLOps processes using APIs, Terraform, or Databricks SDK Build and implement robust testing frameworks: - Integration testing for pipelines - End-to-end validation of ETL/ELT workflows - Testing and validation for ML inference workflows - Improve overall productivity, scalability, and reliability of the data and ML engineering ecosystem -...