Senior StreamSets Engineer/ Data Engineer IRC170963 in GlobalLogic

Description:

We are looking for a Senior StreamSets Engineer/ Data Engineer to join our ambitious project dedicated to one of the Clients of our Fin-tech portfolio. You will be working as a part of the distributed cross-functional Scrum team based in Poland.

You will be working in a Reporting team as well as domain experts, developers, controls engineers, and data scientists. Their primary responsibility will be to develop reliable and instrumented data ingestion pipelines that land inbound data from multiple processes and operational data stores throughout the company to on-premise and cloud-based data lakes. These pipelines will require data validation and data profiling automation along with version control and CI/CD to ensure ongoing resiliency and maintainability of the inbound data flows supporting our advanced analytics projects.

We are an enthusiastic team, tightly integrated with the Customer and empowered for business and technical decisions. This Project is a part of the long-term Program Plan.

Don’t hesitate – to join us and create your world!

Requirements:

• At least 3 years of hands-on experience in designing, developing, deploying, and troubleshooting StreamSets pipelines within data pipeline eco-systems
• Strong working understanding of Big Data ETL solutions
• Data Streaming/Pattern skillset is a must
• Software – StreamSets Data Collector, StreamSets Transformer, StreamSets Control Hub, Kafka, Mongo DB, CDC
• Experience using StreamSets Data Collector with a focus upon design, testing, deploying, operating, and maintenance of pipelines that flow streaming and batch data
• Experience with using the Streamsets API/logs to create alerts for team awareness of issues related to the environment
• Strong understanding of relational database application development
• Strong understanding of development and deployment on a cloud platform in Snowflake or AWS
• Proficiency in scripting – Python language
• Good level of English (both written and spoken). Ability to take part in day-to-day communication and lead a conversation with other stakeholders
• Agile Practices

Nice to have:

• Oracle, Unix scripting, Java, Cloud, NoSQL
• Modern code development practices (git, CI: CD, QA, and Documentation)
• PySpark/Spark, and Data Wrangling
• Experience in a financial services/banking environment

Job Responsibilities:

• Manage Critical Data Pipelines that power analytics for various business units
• Develop different pipelines in the Stream sets according to the requirements of the business owner
• Deploy the Stream sets pipelines into the server
• Architect and build pipeline solutions to integrate data from multiple heterogeneous systems using Stream sets data collectors
• Integrate multiple years of data from files and databases like DB2, SQL Server, Oracle, Teradata, JSON, and MySQL
• Integrate data from multiple topics to the database. Manage RESTful API, and integrate with Stream sets to move data
• Integrate data from Cloudera Big data stack, Hadoop, Hive, Hbase, and MongoDB. Build Stream sets pipeline to accommodate change
• Responsible for sending quality data thru secure channels to downstream systems using role base access control and Stream sets

What We Offer

Exciting Projects: With clients across all industries and sectors, we offer an opportunity to work on market-defining products using the latest technologies.

Collaborative Environment: You can expand your skills by collaborating with a diverse team of highly talented people in an open, laidback environment — or even abroad in one of our global centers or client facilities!

Work-Life Balance: GlobalLogic prioritizes work-life balance, which is why we offer flexible work schedules.

Professional Development: We develop paths suited to your individual talents through international knowledge exchanges and professional certification opportunities.

Excellent Benefits: We provide our employees with private medical care, sports facilities cards, group life insurance, travel insurance, relocation package, food subsidies and cultural activities.

Fun Perks: We want you to feel comfortable in your work, which is why we create good working environment with relax zones, host social and teambuilding activities and stock our kitchen with delicious teas and coffees!

About GlobalLogic

GlobalLogic is a leader in digital engineering. We help brands across the globe design and build innovative products, platforms, and digital experiences for the modern world.By integrating experience design, complex engineering, and data expertise—we help our clients imagine what’s possible, and accelerate their transition into tomorrow’s digital businesses.Headquartered in Silicon Valley, GlobalLogic operates design studios and engineering centers around the world, extending our deep expertise to customers in the automotive, communications, financial services, healthcare and life sciences, manufacturing, media and entertainment, semiconductor, and technology industries.GlobalLogic is a Hitachi Group Company operating under Hitachi, Ltd. (TSE: 6501) which contributes to a sustainable society with a higher quality of life by driving innovation through data and technology as the Social Innovation Business.