Lead Data Engineer

A Global Insurance firm is looking for a talented Lead Data Engineer in the Data Platform & Strategy Department.
As a Lead Data Engineer, you will be a key technical executor within the data engineering team, focusing on hands-on development of critical platform components using AWS and Databricks.

Responsibilities:
- Develop and maintain core data platform components using AWS and Databricks
- Design and implement scalable data pipelines, ingestion patterns, and transformation workflows for multiple data sources
- Optimize performance of data processing workflows and platform components
- Implement monitoring, alerting, and operational best practices for platform reliability
- Develop high-quality, maintainable code following engineering best practices
- Manage infrastructure using Terraform, including infrastructure provisioning, testing, validation, and optimization
- Implement security, compliance, and governance requirements across platform components
- Create and maintain technical documentation for platform components and solution architecture
- Collaborate with data engineers, developers, and cross-functional stakeholders to design and implement effective solutions
- Participate in code reviews, technical discussions, project planning, and provide progress updates while escalating risks when necessary


Why should you apply: 
- This is a long-term opportunity 
- You will be working with international team members 
- Learn new technologies and on large scale projects


Company Details: A leading international insurance organization operates in over 130 countries and jurisdictions, offering services to commercial, institutional, and individual clients through one of the most extensive global property and casualty insurance networks. In addition to general insurance, the organization is also a prominent provider of life insurance and retirement services in the United States. Its common stock is publicly traded on both the New York and Tokyo stock exchange.The group's Japan-based affiliate provides IT-related services to support the organization’s operations within the country.

Working Hours:  9:00 - 17:30 (Mon-Fri) 
Working Style: Hybrid (4 days in office, 1 day work from home) 
Holidays:  Saturday, Sunday, National Holidays, Year-end and New Year Holidays, 27 Paid Holidays every year (20 paid leaves, and 7 especial leaves)
Services/Benefits:  Social insurance, Employee Pension Scheme, Transportation Fee
Interview Process: 3

- 6+ years of experience in data engineering and software development
- Strong expertise in Cloud Platforms (AWS preferred) and data processing frameworks
- Advanced knowledge of Python (preferred), Scala, or Java, SQL (preferred)
- Knowledge in data processing frameworks (Spark preferred)
- Advanced experience with modern SQL Warehouse (Databricks preferred, Snowflake, Big Query), columnar format handling (Parquet / Delta format, ORC preferred), and modern data architecture (Lakehouse preferred)
- Experience with DBT (Data Build Tool) and data modeling methodologies
- Experience with CI/CD/CO practices and DevOps tools (JIRA, Jenkins, Terraform/AWS CDK for IaC)
- Solid understanding of data modeling and data architecture principles
- Experience with real-time data processing and streaming technologies