Senior Python Data Scraping Engineer (Freelance)

Mindrift is recruiting a Senior Python Data Scraping Engineer for a contract freelance position based in Serbia. You will lead data extraction workflows within the Tendem project, collaborating with AI agents while providing technical expertise, validation, and domain knowledge to deliver high-quality, structured datasets.

About This Position

This contract role focuses on advanced web data collection using Python and specialized extraction tools. Working within a hybrid framework where AI handles routine processing, you'll provide critical judgment and technical oversight to ensure accuracy and completeness. Projects typically require 10–20 hours per week during active phases. You'll have access to established platforms including Apify and OpenRouter, complemented by your own proven methodologies and custom solutions. This position offers freelance compensation up to $40 per hour equivalent, scaled according to contribution level and expertise.

Your Responsibilities

• Design and execute complete data extraction pipelines from initial specification through final structured dataset delivery, maintaining comprehensive coverage and accuracy across target sources.
• Combine provided extraction platforms with custom Python workflows to optimize data collection velocity while maintaining quality standards and meeting project specifications.
• Extract data reliably from complex web environments, including dynamically-rendered content, AJAX-driven interactions, and infinite scroll implementations, adjusting approaches as site structures evolve.
• Implement systematic data validation, cross-source consistency verification, and format standardization to guarantee dataset integrity before delivery.
• Optimize and scale collection operations for large datasets using efficient batching, parallelization, and monitoring systems that sustain performance against minor platform variations.

Requirements

• Minimum 5+ years of professional experience in data engineering, web scraping, system automation, or software development.
• Advanced proficiency in Python for web scraping, with hands-on expertise in libraries such as BeautifulSoup, Selenium, or equivalent frameworks.
• Demonstrated capability extracting data from intricate structures—nested hierarchies, archived content, malformed HTML—with systematic troubleshooting.
• Strong expertise in data cleaning, normalization, validation, and formatting (CSV, JSON, Google Sheets) to produce publication-ready datasets.
• Proven experience circumventing anti-bot defenses and managing dynamic website architecture at production scale.
• Hands-on experience with cloud platforms (AWS or comparable) and containerization technologies (Docker) in operational workflows.
• Applied experience with large language model integration frameworks (LangChain, OpenRouter, or similar) for automation purposes.
• Methodical, detail-oriented mindset with commitment to data precision and independent problem-solving capability.
• Professional English communication at B2 level or above.
• Bachelor's or Master's degree in Engineering, Applied Mathematics, Computer Science, or related technical discipline (advantageous).
• GitHub portfolio or public code samples (advantageous).

More About Mindrift

Mindrift operates a specialized marketplace connecting technical experts with AI-driven initiatives developed by leading technology companies. The platform bridges human expertise with artificial intelligence, enabling professionals to contribute deep domain knowledge and validation to advanced projects that shape the future of AI capability.

Next Steps

To apply, complete your application directly on this page, or you'll be redirected to the employer's application platform to finish submitting there.