Lead Software Engineer - Databricks/Spark/AWS

Job description

This is your chance to change the path of your career and guide multiple teams to success at one of the world's leading financial institutions. As a Lead Software Engineer at JPMorgan Chase within Corporate Sector, Chief Technology Office, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. As a core technical contributor, you are responsible for conducting critical technology solutions across multiple technical areas within various business functions in support of the firm’s business objectives. Job Responsibilities: - Lead architecture and delivery of high-throughput, low-latency data pipelines using Databricks and Apache Spark (Core, SQL, Structured Streaming). - Establish lakehouse patterns with Delta Lake (ACID transactions, schema evolution, time travel, Z-ordering, compaction) and ensure performance at scale. - Own Databricks cluster strategy and setup: runtime selection, autoscaling, driver/executor sizing, Spark configs, unit scripts, cluster policies, pools, and instance profiles. - Orchestrate jobs with Databricks Workflows; integrate with AWS eventing and or...