Post a job

Senior Data Lead - Azure Data Factory (ADF), Databricks

On-site United States

Job Summary

We are seeking experienced Data Engineers to lead and support the migration of large-scale analytical and reporting workloads from Amazon Redshift to Databricks Lakehouse Platform. The ideal candidate will have hands-on experience in data warehouse modernization, ETL/ELT migration, performance optimization, cloud-native architectures, and big data processing.

Key Responsibilities

Lead the migration of data pipelines, data models, and reporting workloads from Redshift to Databricks.Perform assessment and discovery of existing Redshift environments, schemas, workloads, and dependencies.Design and implement scalable data solutions using Databricks, Delta Lake, and Spark.Convert SQL-based ETL/ELT processes into Databricks notebooks, workflows, and pipelines.Optimize data ingestion, transformation, and storage for performance and cost efficiency.Develop batch and streaming data pipelines using PySpark and Spark SQL.Implement data quality, reconciliation, and validation frameworks during migration.Collaborate with business stakeholders, architects, and analytics teams to ensure successful migration and adoption.Troubleshoot performance bottlenecks and tune Spark jobs, Delta tables, and SQL workloads.Establish CI/CD processes and infrastructure-as-code practices for Databricks environments.Document migration strategies, architecture, and operational procedures.

Skill Requirements

Strong experience with large-scale data warehousing and data lake architectures.Hands-on experience with Amazon Redshift and Databricks.Expertise in PySpark, Spark SQL, and Python.Strong SQL development and query optimization skills.Experience with data modeling (Star Schema, Snowflake Schema, Dimensional Modeling).

Other Requirements

Migration ExperiencePrior experience migrating from:Redshift to DatabricksTeradata/Snowflake/Hadoop to Databricks (preferred)Data validation, reconciliation, and cutover planning.Performance tuning and workload modernization.

Source: the employer's careers page. Last checked 2026-11-17. Posted 2026-11-11.

Applications happen on HCLTech's own site. View role and apply

Get jobs like this by e-mailAt most one e-mail a day, only when something new matches. Unsubscribe in one click.

More from HCLTech

All jobs at HCLTech

Similar jobs in United States

Hiring for a role like this? See how employers fill a role fast.