Senior Technical Lead

Job Summary

Job Code

15182

Locations

Bengaluru

Minimum Experience

4

Maximum Experience

8

Mandatory Skills

Pyspark

Skill to Evaluate

Pyspark

Experience

4 to 8 Years

Location

Bengaluru

Job Description

·       4-6 years of experience in data engineering tools and technologies.

·       Designing and implementing highly performant data ingestion pipelines from multiple sources using Apache Spark and/or Azure/AWS Databricks

·       Integrating the end-to-end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is always maintained

·       Willing to work in 2nd shift

·       Willing to work in Hybrid mode

·       Strong knowledge of Data Management principles

·       Good Experience in SQL or any native SQL like query languages.

·       Experience in ETL / data warehouse transformation processes

·       Direct experience of building data pipelines using Apache Spark (preferably Databricks).

·       Good to have experience in writing code in python.

·       Good to have experience in Power BI.

·       Working with other members of the project team to support delivery of additional project components.

·       Evaluating the performance and applicability of multiple tools against customer requirements.

·       Working within an Agile delivery / DevOps methodology to deliver proof of concept and production implementation in iterative sprints.

·       Able to Work Independently

·       Good oral and written communication skill

Key Responsibilities

·       Designing and implementing highly performant data ingestion pipelines from multiple sources using Apache Spark and/or Azure/AWS Databricks

·       Integrating the end-to-end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is always maintained

·       Support Biz for adhoc issues

Create and maintain optimal data pipeline architecture

Assemble large, complex data sets that meet business requirements

Identify, design, and implement internal process improvements

Optimize data delivery and re-design infrastructure for greater scalability

Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS technologies

Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics

Work with internal and external stakeholders to assist with data-related technical issues and support data infrastructure needs

Skill Requirements

   Designing and implementing highly performant data ingestion pipelines from multiple sources using Apache Spark and/or Azure/AWS Databricks

·       Integrating the end-to-end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is always maintained

Other Requirements