Director of Software Engineering (Node.js & Web Scraping Expert)

Job description

We are seeking a Director of Software Engineering with deep expertise in Node.js development and large-scale web scraping . This role will lead the engineering team , designing and optimizing high-performance, distributed web scraping systems . The ideal candidate has extensive experience in handling anti-bot measures, data pipeline optimization, and scalable cloud-based architectures . Key Responsibilities- Software Engineering & Web Scraping Leadership: - Architect, develop, and maintain scalable and distributed web scraping systems using Node.js . - Design and implement data extraction pipelines to process large volumes of structured and unstructured data. - Develop solutions to bypass anti-bot mechanisms , including CAPTCHA handling, session management, fingerprinting, and IP rotation . - Optimize scraping processes for performance, reliability, and efficiency while managing proxy services (residential, datacenter, rotating).Oversee data storage and processing strategies , ensuring high availability and consistency. - Collaborate with Product, DevOps, and Data Science teams to integrate extracted data into analytics and business applications. - Implement best practices for micr...