Senior Specialist - Software Engineering

Job description

Key Responsibilities Design and Build Data Pipelines Develop and maintain scalable data pipelines that automate the extraction transformation and loading ETL of data from various sources into centralized repositories Data Management Create and manage data management systems ensuring they meet business requirements and industry practices Data Quality Assurance Implement validation rules and cleansing procedures to maintain high data quality and integrity ensuring accurate and reliable data for analysis Collaboration Work closely with data architects data scientists and analysts to understand their data needs and tailor solutions accordingly Research and Development Explore opportunities for data acquisition and new uses for existing data integrating new technologies and tools into existing structures Documentation and Reporting Document data processes and workflows and report on data quality and pipeline performance to stakeholders Required Skills and Qualifications: Technical Proficiency Strong programming skills in languages such as Python Java or Scala and experience with SQL and NoSQL databases Big Data Technologies Familiarity with big data tools and frameworks like Hadoop Spar...