Databricks Data Engineer

Job description

Job Family: Data Engineering & Architecture Consulting Travel Required: Up to 10% Clearance Required: Ability to Obtain Public Trust What You Will Do: Develop and implement CI/CD pipelines for Databricks notebooks and jobs. Develop ETL pipelines using PySpark and Databricks. Implement Delta Lake for ACID transactions and data reliability. Optimize ingestion from APIs, streaming, and batch sources. Ensure compliance with data governance and security standards. Collaborate with data engineers and scientists to support data pipelines and ML workflows. Conduct ETL and data quality analysis using various technologies (i.e., Python, Databricks). Ensure data governance and quality assurance standards are met. Organize and lead meetings, including scheduling meetings; drafting and delivering agendas and meeting minutes; providing and archiving required documentation; and documenting, tracking, and following up on action items. Summarize and present information and reports to the team and make recommendations (both oral and written). What You Will Need: US Citizenship is required A bachelor’s degree is required: Minimum THREE (3) years of total experience in cloud-based data platforms Minim...