Senior Data Engineer - ElasticSearch in Globaldev Group

Kanda Software is a dynamic company based in the US, known for delivering innovative software solutions and technology services. We are currently seeking a highly skilled Senior Data Engineer to join our team and contribute to our exciting projects.Responsibilities:Lead the design, development, and optimization of complex data systems, including data flows, lakes, warehouses, and ETLs, across various cloud platformsArchitect and manage Elasticsearch clusters, focusing on performance optimization, security, and scalabilityCreate and optimize indices for performance and storage efficiency, leveraging index templates and mappingsBuild and optimize Elasticsearch queries using the Query DSL, including complex aggregations and full-text searchImplement data ingestion pipelines to load data from various sources, including relational databases (RDBMS), into Elasticsearch Oversee performance monitoring, cluster management, security protocols, backup and restore procedures, and version management of Elasticsearch clustersDevelop and maintain data pipelines using industry-standard tools such as Apache Spark, Apache Hive, Apache Airflow, and StitchImplement data pipeline observability strategies to ensure reliable data flow and system transparencyApply modern software development practices, including Agile methodologies, Test-Driven Development (TDD), and Continuous Integration/Continuous Deployment (CI/CD), to enhance data engineering workflowsRequirements:A minimum of 5 years of experience as a technical lead, building and extending complex data systemsAdvanced proficiency in Python, with at least 3 years of experienceStrong SQL writing and tuning skillsExpert-level knowledge of Elasticsearch, including configuration, administration, and optimization techniquesDemonstrated experience with building and maintaining data pipelines using tools such as Spark, Hive, Airflow, and StitchProven experience with data pipeline observability strategies and toolsStrong understanding of software development practices, including Agile, TDD, and CI/CDWill be a plus:Experience building or maintaining streaming platforms using Apache KafkaProficiency in PySpark for distributed data processingSkills in implementing and managing containerization solutions using Kubernetes, Docker, etc., including observability of containerized applicationsExperience with Big Data as a Service (BDaaS) tools, such as AWS EMR or Azure HDInsightExperience in building and extending Machine Learning and Data Science platformsFamiliarity with integrating data flows with third-party Business Intelligence (BI) platforms like Domo, Tableau, Sisense, etcWhat we offer:Competitive salary and benefits package;Flexible remote work arrangements;Opportunities for professional development and growth;Periodic review of the salary;Paid events attendance; Show more Show less Посадовий рівень Старший середній рівеньType of employment Full-time Job responsibilities Information Technology Industries Primary and secondary education and Non-profit organizations