Backend Engineer Data Platform

Position Overview

You will be responsible for the development and operation of the data platform that supports the collection, search, and analysis of large-scale data generated by autonomous driving systems.

Background

To realize a safer and more secure autonomous driving society, we are building a data platform to collect, search, and analyze a wide variety of large-scale driving data. This data platform must address several key requirements: handling the massive increase in data volume as autonomous driving becomes more widespread, aggregating and searching real-time driving metrics and logs, automatically classifying driving scenes, and visualizing unstructured data (such as images and point clouds).

To support globally utilized autonomous driving services and democratize this technology, we are looking for members who can build this data infrastructure and deliver value to our users together.

Job Responsibilities

You will develop and operate the data platform supporting the collection, search, and analysis of autonomous driving data:

Develop and operate data pipelines to collect and process driving data from vehicles.

Develop and operate APIs providing driving history and statistical information.

Develop and operate driving data visualization and search services.

Build and operate internal data analysis infrastructure.

Challenges & Rewards

Developing data pipelines that efficiently collect and process massive amounts of driving metrics and logs, enabling near-real-time analysis.

Developing systems that process unstructured data (such as images and point clouds) to enable searching and visualization.

Achieving data management cost optimization through the automated analysis of large-scale driving data.

Important Information for Software Development Applicants

A degree in Computer Science is not mandatory; equivalent practical experience is highly welcome.

Foundational knowledge of computer science, including data structures, algorithms, and computational complexity.

A proven track record of software design and implementation with careful consideration for maintainability, testability, and performance.

A basic understanding of concurrent/parallel processing as well as network and communication protocols.

Experience with debugging and code reviews in a team development environment.

X+ years of practical experience in developing software running in production environments.

Consistent experience across the entire software development lifecycle-from planning and design to release and subsequent operations (at least once).

Scope of Duties

Initial Assignment: The duties described in this job description.

Potential Changes: Duties designated by the company in addition to the above.

Required Skills

3+ years of experience in web application development and operations.

Development experience using programming languages such as Python or Go.

Development and operational experience using cloud services such as AWS.

Experience designing system architectures and APIs.

Experience developing and operating data pipelines.

Experience building and operating CI/CD pipelines.

Japanese language proficiency is not required.

Preferred Skills

Experience in data modeling and architecture design/construction using DWH or Data Lakehouse systems.

Knowledge or experience regarding automotive data attributes.

Experience leading a development team.