Lead Data Engineer - Las Vegas, NV
Photon · US
Greetings Everyone Who are we? For the past 20 years, we have powered many Digital Experiences for the Fortune 500. Since 1999, we have grown from a few peop...
Job description
Greetings Everyone Who are we?: For the past 20 years, we have powered many Digital Experiences for the Fortune 500. Since 1999, we have grown from a few people to more than 4000 team members across the globe that are engaged in various Digital Modernization. For a brief 1 minute video about us, you can check https://youtu.be/uJWBWQZEA6o . What will you do? What are we looking for? Responsibilities: - Develop and maintain data pipelines, ELT processes, and workflow orchestration using Apache Airflow, Python and PySpark to ensure the efficient and reliable delivery of data. - Design and implement custom connectors to facilitate the ingestion of diverse data sources into our platform, including structured and unstructured data from various document formats . - Collaborate closely with cross-functional teams to gather requirements, understand data needs, and translate them into technical solutions. - Implement DataOps principles and best practices to ensure robust data operations and efficient data delivery. - Design and implement data CI/CD pipelines to enable automated and efficient data integration, transformation, and deployment processes. - Monitor and troubleshoot data pipelines...