Senior software engineer (scraping) in Competera

InCompetera, weare building aplace where optimal pricing decisions can bemade easily. Webelieve thatAI technologies will soon drive all challenging decisions and are capable ofhelping humans bebetter. Weare now looking for aSenior Software Engineer tohelpus turn business ideas into cutting-edge tech solutions that explicitly drive the product and the company towards its strategic direction.Our stack:Programming Languages: Python.Tech Stack: PostgreSQL, asyncio, FastAPI, Apache Kafka, ClickHouse, BigQuery, GCS, Scrapy, Selenium, PyppeteerInfrastructure: Kubernetes, Helm, ArgoCD, Grafana, Prometheus, Sentry, OpsGenie.You could beaperfect match for the position ifYou wantto:Design and implement scalable and efficient services for aweb scraping system using Python and relevant libraries.Lead the development team inovercoming anti-scraping measures such asCAPTCHAs, rate limiting, and dynamic content loading.Mentor and guide junior engineers inthe team, fostering their growth and development.Automate existing scraper development processes using tools and frameworks toimprove efficiency and maintainability.Monitor system performance and proactively identify and address any issues.Design and implement microservices architecture for modularity and scalability.Apply Domain-Driven Design (DDD) principles toensure clean and maintainable code.Stay up-to-date with the latest advancements and best practices inweb scraping.You have:5+years ofexperience asaSoftware Engineer with astrong track record inweb scraping.Proven expertise inPython and relevant libraries like Scrapy, Selenium, Pyppeteer, Playwright.In-depth understanding ofweb protocols and how modern web works (SSL, proxies, cookies, headers, browsers).Experience with anti-scraping detection and bypass techniques.Experience with Kafka, Redis, and microservices architecture.Familiarity with Domain-Driven Design (DDD) principles.Eng— interm +Pleasant extras:Experience with cloud platforms like AWS orGCP.Experience with data quality and cleaning techniques.Knowledge ofmachine learning and data analysis techniques.Soft skills:Ability tolead and mentor junior engineers inovercoming technical challenges.Proactive approach toidentifying and solving problems, seeking out new opportunities.Ability towork independently and aspart ofateam inafast-paced environment.Ability tolearn new technologies and adapt tochanging web scraping landscapes.You’re gonna loveit, and here’s why:Rich innovative software stack, freedom tochoose the best suitable technologies.Remote-first ideology: freedom tooperate from the home office orany suitable coworking.Flexible working hours (westart from 8to11am) and notime tracking systemson.Regular performance and compensation reviews.Recurrent 1-1s and measurable OKRs.In-depth onboarding with aclear success track.Competera covers70% ofyour training/course fee.20vacation days, 15days off, and uptoone week ofpaid Christmas holidays.20business days ofsick leave.Partial medical insurance coverage.Wereimburse the cost ofcoworking.Drive innovations withus. BeaCompeterian.