Data Engineer in ANVI

Project description:

• Establishment of a cloud data platform on which data-driven solutions are developed with the help of analytics, machine learning and artificial intelligence.

• The forecasting project aims to develop modules for automated, AI-based, linear TV reach forecasting so that better decisions can be made to place the right content on the right channel at the right time.

Scope of the service:

• The implementation of data pipelines for the preparation, provision and versioning of data for model training

• Advising the Data Scientists on the development of machine learning models, especially regarding the productive operation of these models

• Design and implementation of microservices to provide the models via REST API, including functions for monitoring these models in productive operation

• Deployment of the microservices in the productive cloud environment, accounting for high availability requirements.

Technology Stack used

• Google Cloud Platform, Terraform, GitLab

• Kubernetes, Docker, Airflow, MLflow

• BigQuery, BigTable

• Python, Pyspark, SQL

• REST APIs