Web Scraping Specialist

Job description

Location: Remote with a 6 hour overlap with EST Remote | Full-time: Compensation: $75K - $150K: We are hiring on behalf of our client who is seeking a Web Scraping Specialist to join a specialized technical team focused on building the infrastructure that delivers massive amounts of web data for the training of advanced AI models. This organization operates a massive distributed crawler and manages complex pipelines for ingesting, segmenting, and annotating billions of data points, including videos, transcripts, and audio files. The successful candidate will lead efforts to gather and analyze data, optimize scraping processes, and support the scaling of high-quality public web data accessibility. This role is ideal for a lean, technical builder who thrives in a fast-paced environment without bureaucratic red tape. Key Responsibilities: - Code Development: Write, test, and refine high-performance code to extract data from various online sources, ensuring maximum reliability and efficiency. - Data Retrieval: Manage complex data retrieval tasks, including handling pagination and dynamic content loaded via AJAX. - Data Quality: Clean and format extracted data to ensure it meets rigorou...