Development Lead - Advanced RPA

Job Summary

Role Overview

We are hiring an experienced Data Engineer to design, build, and optimize scalable data solutions across AWS and Microsoft Azure. The role requires strong hands-on expertise in Databricks, data lakes and lakehouse architecture, ETL/ELT pipelines, data warehousing, and the delivery of business-ready reporting solutions using Power BI.

Key Responsibilities

Responsibilities (Brief)

• Design, develop, test, and maintain scalable batch and streaming data pipelines across AWS and Azure.
• Build and optimize Databricks lakehouse solutions using PySpark, Delta Lake, and medallion architecture.
• Integrate structured and unstructured data from multiple sources while ensuring reliability, scalability, and cost efficiency.
• Develop curated data models and semantic layers to support Power BI dashboards, KPIs, and self-service reporting.
• Apply data quality checks, security controls, governance standards, metadata management, and lineage across the data lifecycle.
• Monitor and troubleshoot pipeline failures, improve query and workload performance, and support production data operations.
• Collaborate with architects, analysts, BI developers, and business stakeholders to translate reporting needs into dependable data products.

Skill Requirements

Key Skills Required

• Programming & Querying: Advanced SQL, Python, and PySpark
• AWS Data Services: Amazon S3, AWS Glue, AWS Lambda, Amazon Redshift, IAM, and related data integration services
• Azure Data Services: Azure Data Lake Storage, Azure Data Factory, Azure Synapse Analytics, and Azure Key Vault
• Databricks: Apache Spark, Delta Lake, notebooks, jobs, workflows, performance optimization, and Unity Catalog
• Data Engineering: Batch and streaming ingestion, ETL/ELT development, orchestration, monitoring, and reusable data pipelines
• Data Architecture: Data lake, lakehouse, cloud data warehouse, and medallion architecture
• Data Modelling: Dimensional modelling, star and snowflake schemas, and semantic data models
• BI & Reporting: Power BI, DAX, Power Query, data modelling, dashboard development, and performance tuning
• Data Governance: Data quality, security, metadata management, lineage, access control, and cataloguing
• DevOps: Git, CI/CD, infrastructure deployment practices, and environment management for data solutions

Other Requirements

Preferred Qualifications

• 6–10+ years of data engineering experience, including hands-on delivery on AWS and Azure.
• Strong experience with Databricks and Power BI in enterprise-scale analytics environments.
• Exposure to Snowflake, Fivetran, HVR, Tableau, Atlan, or similar data integration and governance tools.
• Understanding of cloud security, cost optimization, deployment automation, and production support practices.
• Strong stakeholder communication, problem-solving, documentation, and cross-functional collaboration skills.