Data Engineer　AI Division

Job Description

As a Data Engineer in our AI Division, you will play a key role in designing, developing, and maintaining scalable, reliable, and high-performance production systems for financial data and AI-driven investment products. You will work closely with data scientists and other engineers to manage end-to-end data pipelines and contribute to product innovation.

Key Responsibilities:

• Design, develop, and maintain production-grade data pipelines and systems for financial data.
• Collect requirements and manage end-to-end data workflows from ingestion to shutdown.
• Collaborate with data scientists to implement and optimize data pipelines.
• Evaluate and integrate new data sources, negotiate onboarding, and ensure compliance with data usage licenses.
• Assess new technologies and drive innovation to enhance products.
• Improve and maintain existing products and infrastructure.

Required Qualifications

• Japanese (reading and writing, JLPT N3 or above).
• Residing in Japan.
• Bachelor’s degree in Computer Science or a related field with at least 5 years of practical experience.
• Solid understanding of computer science fundamentals, from low-level to high-level system structures.
• Proficiency in Python, including Pandas and NumPy.

Preferred / Advantageous Skills:

• Experience with big data and out-of-core processing.
• Knowledge of infrastructure and DevOps practices.
• Hands-on experience with cloud platforms (AWS, GCP, Azure).
• Experience with data pipelines and workflow engines (Airflow, ArgoWorkflow, Prefect).
• Software engineering experience.
• Database management and monitoring knowledge.
• Business-level Japanese and English skills.
• Familiarity with Golang, Rust, or Java.
• Financial data handling experience (FX, stocks, ETFs).
• Streaming data processing and protocol experience.
• Experience with monitoring tools (e.g., DataDog) and ML frameworks (scikit-learn, PyTorch, TensorFlow, Chainer).
• Project and team management experience.
• Terraform experience for managing AWS resources (RDS, S3, Athena, EC2, EKS, Lambda, VPC, DynamoDB, Redshift).