CS - Sr. Web Scraping Engineer (Antibot Specialist) - Job7207

Summary

We are seeking a senior Web Scraping Engineer focused on automation and anti-bot techniques to build resilient data-collection systems. This role is critical for reliably extracting data from dynamic web applications while navigating anti-bot defenses and maintaining ethical, compliant practices.

The position offers deep hands-on work with headless browsers, proxy and CAPTCHA solutions, and cloud-deployed scraping architectures using JavaScript/Node.js, Python, and .NET. You will lead technical design, solve complex evasion challenges, and transfer knowledge to internal teams.

Responsibilities

• Design and implement web scraping and browser automation solutions using headless browsers.
• Develop strategies to handle dynamic content (AJAX, infinite scroll) and complex DOM interactions.
• Implement anti-bot evasion techniques, including CAPTCHA integration, fingerprint spoofing, IP rotation, and proxy management.
• Emulate human-like behavior (mouse movement, delays) to reduce detection.
• Build reliable authentication flows and manage secure credential storage and sessions (OAuth, SAML, MFA).
• Integrate and manage third-party CAPTCHA solvers and proxy services.
• Deploy and containerize scraping services for scalable execution in cloud functions or containers.
• Analyze network traffic and reverse-engineer web app interactions to identify stable data sources.
• Collaborate with stakeholders to transfer knowledge and ensure ethical, compliant scraping practices.

Requirements

• Senior-level experience in web scraping, browser automation, and anti-bot strategies.
• Proficiency with headless browsers and automation libraries: Puppeteer, Playwright, or Selenium.
• Strong skills in JavaScript/Node.js and Python; experience with .NET also required.
• Experience parsing and extracting content using BeautifulSoup, lxml, or Scrapy.
• Experience performing direct HTTP requests using requests or httpx.
• Practical experience with CAPTCHA solving workflows and services such as 2Captcha, Anti-Captcha, or CapMonster.
• Experience with proxy services and IP rotation strategies using providers like Bright Data, Oxylabs, or Smartproxy.
• Familiarity with deploying scraping workloads to cloud functions (AWS Lambda, Azure Functions) and containerizing with Docker.
• Deep understanding of authentication flows (OAuth, SAML, MFA), secure credential handling, and session management.
• Awareness of legal and ethical boundaries for scraping and familiarity with data privacy considerations (e.g., GDPR, CCPA).

Nice to Have

• Experience reverse-engineering web applications and analyzing API calls via browser dev tools.
• Experience with browser fingerprinting tools such as Multilogin.
• Familiarity with bot detection platforms like Cloudflare Bot Management or PerimeterX.