Python Developer: Web Scraping. Scrapy in iPublisher

Python Developer: Web Scraping. Scrapy remote iPublisher continues development. We are relaunching the Clever mobile real estate project. This is a CRM for managing the work of real estate brokers. Part of this project is the collection of data from real estate websites. Currently, we want to expand this project to other cities. As part of the team, you will work on the collection, storage, processing and analysis of huge data sets. The main task will be to develop the parser. Technology stack: Python; Zennoposter to know scrapy (proper tool for parsing) Django; Celery; MySQL; Docker. About the role: As a Python engineer, you will join a team that is passionate about their work, and together you will implement advanced technologies into an easy-to-use application. Unas white application taparsera. We need to fix some of the broken parsers (for example, olh), then increase the number of sites from which we parse data. What you will do: —build and troubleshoot web assembly tools; — expansion of data collection sites (parsing); circumvention of protection against parsing — improvement of parsing algorithms; Fixing Broken Parsers —Analysis of target websites, including reverse engineering, detection of defenses, and best practices for data acquisition; —creating and supporting solutions for storing collected information; — analysis of the data you received, selection of named objects, performance of classifications, comparative analysis, etc. Our goal is to collect data, optimize its storage and use, and be able to analyze it. Working conditions: - part-time; —remote work; — a truly wonderful collective of such leadership.