Distributed Training & Inference Optimization Engineer (LLM)

Job description

Distributed Training & Inference Optimization Engineer (LLM)
Join to apply for the
Distributed Training & Inference Optimization Engineer (LLM)
role at
Rakuten Asia Pte Ltd
Distributed Training & Inference Optimization Engineer (LLM)
4 days ago Be among the first 25 applicants
Join to apply for the
Distributed Training & Inference Optimization Engineer (LLM)
role at
Rakuten Asia Pte Ltd
Get AI-powered advice on this job and more exclusive features.


Situated in the heart of Singapore's Central Business District, Rakuten Asia Pte.

Ltd.

is Rakuten's Asia Regional headquarters.

Established in August 2012 as part of Rakuten's global expansion strategy, Rakuten Asia comprises various businesses that provide essential value-added services to Rakuten's global ecosystem.

Through advertisement product development, product strategy, and data management, among others, Rakuten Asia is strengthening Rakuten Group's core competencies to take the lead in an increasingly digitalized world.


AI & Data Division (AIDD) spearheads data science & AI initiatives by leveraging data from Rakuten Group.

We build a platform for large-scale field experimentations using cutting-edge technologies to provide critical insights that enable faster and better and faster contribution for our business.

Our division boasts an international culture created by talented employees from around the world.

Following the strategic vision “Rakuten as a data-driven membership company”, AIDD is expanding its data & AI related activities across multiple Rakuten Group companies.


As a
GPU Training & Inference Optimization Engineer , you will focus on maximizing the performance, efficiency, and scalability of LLM training and inference workloads on Rakuten’s GPU clusters.

You will deeply optimize training frameworks (e.g., PyTorch, DeepSpeed, FSDP) and inference engines (e.g., vLLM, TensorRT-LLM, Triton, SGLang), ensuring Rakuten’s AI models run at peak efficiency.

This role requires
strong expertise in GPU-accelerated ML frameworks, distributed training, and inference optimization , with a focus on reducing training time, improving GPU utilization, and minimizing inference latency.


Key Responsibilities
Optimize LLM training frameworks (e.g., PyTorch, DeepSpeed, Megatron-LM, FSDP) to maximize GPU utilization and reduce training time.


Profile and optimize distributed training bottlenecks (e.g., NCCL issues, CUDA kernel efficiency, communication overhead).


Implement and tune inference optimizations (e.g., quantization, dynamic batching, KV caching) for low-latency, high-throughput LLM serving (vLLM, TensorRT-LLM, Triton, SGLang).


Collaborate with infrastructure teams to improve GPU cluster scheduling, resource allocation, and fault tolerance for large-scale training jobs.


Develop benchmarking tools to measure and improve training throughput, memory efficiency, and inference latency.


Research and apply cutting-edge techniques (e.g., mixture-of-experts, speculative decoding) to optimize LLM performance.


Mandatory Qualifications
3+ years of hands-on experience in GPU-accelerated ML training & inference optimization, preferably for LLMs or large-scale deep learning models.


Deep expertise in PyTorch, DeepSpeed, FSDP, or Megatron-LM, with experience in distributed training optimizations.


Strong knowledge of LLM inference optimizations (e.g., quantization, pruning, KV caching, continuous batching).


Bachelor’s or higher degree in Computer Science, Engineering, or related field.


Nice-to-Have Skills
Proficiency in CUDA, Triton kernel, NVIDIA tools (Nsight, NCCL), and performance profiling (e.g., PyTorch Profiler, TensorBoard).


Experience with LLM-specific optimizations (e.g., FlashAttention, PagedAttention, LoRA, speculative decoding).


Familiarity with Kubernetes (K8s) for GPU workloads (e.g., KubeFlow, Volcano).


Contributions to open-source ML frameworks (e.g., PyTorch, DeepSpeed, vLLM).


Experience with inference serving frameworks (e.g., vLLM, TensorRT-LLM, Triton, Hugging Face TGI).


Why Join Us?
Work on cutting-edge LLM training & inference optimization at scale.


Directly impact Rakuten’s AI infrastructure by improving efficiency and reducing costs.


Collaborate with global AI/ML teams on high-impact challenges.


Opportunity to research and implement state-of-the-art GPU optimizations.


Rakuten is an equal opportunities employer and welcomes applications regardless of sex, marital status, ethnic origin, sexual orientation, religious belief, or age.


Seniority level
Seniority level Mid-Senior level
Employment type
Employment type Full-time
Job function
Job function Information Technology and Engineering
Industries Software Development
Referrals increase your chances of interviewing at Rakuten Asia Pte Ltd by 2x
Sign in to set job alerts for “Optimization Engineer” roles.


Process Engineer (Open to hiring fresh graduates)
Manufacturing Production Control Engineer
Etch Process Engineer - Advanced Packaging
Assistant/ Engineer, Mechanical (Production)
Process Engineer (PCP) 15 months training programme
Research Engineer (Machine Learning in Chemical Engineering), ChBE
Queenstown, Central Singapore Community Development Council, Singapore 1 month ago
Process Engineer (PCP) 15 months training programme
NPU System Optimization Engineer (Software - Python)
Process Engineer (Manufacturing Engineer)
NTI PCVD Process Development Engineer/Snr Engineer
NTI PCVD Process Development Engineer/Snr Engineer
We’re unlocking community knowledge in a new way.

Experts add insights directly into each article, started with the help of AI.


#J-18808-Ljbffr

Required Skill Profession

Other General