Data Engineer - Warsaw

Job description

About the Role We are looking for a Data Engineer to design and maintain scalable data solutions that power advanced analytics and AI-driven insights. This role combines expertise in big data engineering and web scraping , enabling you to work on high-impact projects involving large, complex, and unstructured datasets. You will architect data pipelines, enforce governance standards, and build tools for extracting and processing data from diverse sources, including websites and external vendors. If you thrive in solving complex data challenges and want to work with cutting-edge technologies, this is the role for you. What You’ll Do: - Architect, develop, and maintain high-throughput data pipelines in Databricks and AWS (Glue, EMR, Fargate, Step Functions). - Ingest, normalize, and enrich large volumes of structured and unstructured data, including internal data, market data, vendor feeds, and alternative sources. - Collaborate with AI Engineers , ML scientists , and software teams to translate requirements into scalable data architectures, schemas, and APIs. - Optimize pipeline performance and cost using distributed processing techniques ( Spark, Delta, Arrow ) and AWS best practice...