Software Development Engineer 5

Sr. Staff Data Platform Engineer

Position Summary

As a Sr. Staff Data Platform Engineer , you will be the primary architect and visionary for the core data infrastructure that powers the entire enterprise. This is a "platform-as-a-product" role; your mission is to build the internal foundation—creating the high-performance engines, abstraction layers, and self-service frameworks that enable hundreds of other engineers to move faster.

You will operate at the intersection of Systems Programming and Data Engineering , solving "hard-tech" problems like automated schema evolution, multi-engine compute optimization, and global data discovery. As a top-level technical individual contributor, you will bridge the gap between long-term business strategy and deep-kernel technical execution, ensuring WEX’s data platform remains a competitive advantage.

Is this role for you?

• YES, if: You are a Backend/Software Engineer who loves solving "Big Data" problems, building APIs for data discovery, and optimizing distributed systems.

• NO, if: Your primary expertise is writing SQL queries, building Tableau dashboards, or managing ETL workflows without deep experience in Java or Python system architecture.

Responsibilities

• Architectural Sovereignty: Define the 3-5 year technical roadmap for the Data Lakehouse. You aren't just using tools; you are deciding how storage, compute, and metadata layers (e.g., Apache Polaris, Unity Catalog or Datahub Catalog ) interact at an elemental level.

• Platform-as-a-Product: Build internal SDKs, CLI tools, and automated orchestration frameworks. Your goal is to abstract away cloud complexity via Control Planes and Custom Operators , allowing Data Engineers to focus on business logic rather than infrastructure boilerplate.

• Internal R&D: Prototype and benchmark emerging technologies (e.g., specialized Spark extensions ) to keep the platform at the bleeding edge of performance and cost-efficiency.

• Global Governance & Security: Architect "compliance-by-design" systems. Automate data lineage, PII masking, and fine-grained access control across petabyte-scale environments without sacrificing developer velocity.

• Engineering Excellence & Influence: Set the gold standard for code quality and system design across the company. You will lead Cross-Functional Architecture Reviews and serve as the final escalation point for the most complex system outages or performance bottlenecks.

• Organizational Mentorship: Beyond individual mentoring, you will foster an "Engineering Community," influencing the hiring bar and professional development paths for the entire data engineering organization.

Qualifications & Experience

• Experience: 15+ years in software engineering and distributed systems, with at least 4 years in a principal or staff-level capacity leading platform-scale initiatives.

Core Technical Competencies (Software Engineering Focus):

• Strong Software Foundations: Strong fundamentals on software engineering, system architecture, and scalable production applications (Algorithms, Data Structures, and System Design).

• Experience in the Java/J2EE ecosystem (Spring Boot, Microservices) and python. We are looking for a developer who writes clean, testable, and high-performance code, not just scripts.

• Data as a Product: Experience building the platforms / framework engines and APIs that power data movement, rather than just building the ETL/ELT pipelines themselves.

Data Lakehouse Mastery:

• Deep internal knowledge of Apache Iceberg, Hudi, or Delta Lake (metadata management, manifest files, and compaction strategies).

• Experience contributing to or deeply customizing open-source data projects (e.g., Spark, dbt ).

Cloud & Infrastructure:

• Extensive experience with cloud architecture and services, including AWS (S3, EMR, Kubernetes, Lambda) and Azure .

• Deep understanding of CI/CD automation, modern development tools, Git Actions, Terraform and frameworks .

AI-Driven Development & Productivity:

• AI Native Development: Experience leveraging AI Code Gen platforms into software development lifecycle (SDLC) to automate code generation, reviews, generate unit tests, and perform root-cause analysis of system failures.

• LLM-Ops for Platform: Ability to architect the infrastructure required to support AI Agent development by enabling vector database integration.

Leadership & Vision:

• Proven track record of "leading by influence" —driving adoption of new technologies across multiple autonomous teams.

• Ability to communicate complex architectural trade-offs (e.g., "Latency vs. Consistency" or "Build vs. Buy") to C-suite executives and junior engineers alike.

Education:  

• Bachelor’s or Master’s degree in Computer Science (Distributed Systems focus) preferred, or equivalent deep industry experience.