Data Flow Engineer
ARHS · Warsaw, Mazovia, PL
Company Description Arηs Group, Part of Accenture, specializes in the management of complex public sector IT projects, including systems integration, informa...
Job description
Company Description Arηs Group, Part of Accenture, specializes in the management of complex public sector IT projects, including systems integration, informatics and analytics, solution implementation and program management. Our team helps lead clients through digital and information systems design, bringing expertise in a variety of areas ranging from software development, data science and security management to machine learning, cloud, and mobile development. Arηs Group was acquired by Accenture in July 2024. Job Description: - Design, develop, and maintain complex data flows within Cloudera DataFlow (Apache NiFi), ensuring scalable, reliable, and high-performance data movement across systems. - Develop and optimize real-time and near real-time data pipelines leveraging NiFi, Kafka, and CDC technologies (e.g., Debezium, SQL-based connectors). - Implement integrations with internal and external systems using REST APIs, JDBC, Kafka, and other communication protocols, ensuring secure and resilient data exchange. - Design and manage data schemas (Avro), metadata, and lineage using Apache Atlas, ensuring full traceability and governance of data flows. - Define and enforce data securit...