Data Engineer / Web Scraper Specialist in Smart Orange

About the company:

We are looking for a talented web scraping and data collection specialist to help us build an automated company information collection system. The main goal is to collect a database of companies, their websites and short descriptions from open sources for further analysis.

Main responsibilities:

• Company data collection via D7 Lead Finder:

  • Use web scraping or API (if available) to collect company data such as name, web -site, description.
  • Configure script for proper navigation and authentication on D7 Lead Finder for automatic information collection.

• Additional data search through other sources:

  • Hunter.io API, LinkedIn API / Sales Navigator: Use to search and enrich data about companies.
  • Google Custom Search API: Automate the search for companies by keywords to supplement the database.

• Processing and data verification:

  • Clean and combine data from different sources, remove duplicates, ensure correctness and relevance of information.
  • Save data in format CSV or Excel for further analysis.

• Documentation and process automation:

  • Develop documentation on configuring and running scripts for data collection.
  • Configure periodic scripting to run semi-annually/annually or as needed.

Requirements for the candidate:

• Experience in web scraping and working with dynamic pages: Knowledge Python and Selenium, Beautiful Soup, Scrapy libraries.
• API skills: Integration experience with Hunter.io , LinkedIn, Google Custom Search to automate data collection.
• Data processing and structuring: Ability to work with CSV, JSON, Excel formats for data cleaning and processing.
• Attention to detail and data verification: Ability to verify received information, remove duplicates and correct errors.

< strong>Would be an advantage:

• Experience with data processing tools (Pandas).
• Understanding the principles of processing large volumes of data and their optimization.
• Skills in automating data collection processes and experience with HTML parsing.

We offer:

• Flexible work schedule with the possibility of remote cooperation.
• Project work with the prospect of further cooperation on a long-term basis.
• The possibility to independently choose tools and methods of work.

If you are passionate about data processing, have experience with web scraping and want to join our team, please send your resume