Tech Expert/Backend Engineer - Global Live (LLM Model Serving)

About Our TeamMission of Global Live Service Architecture team is Build Real-time Interactive Architecture, Safeguard Global LIVE.

We are seeking highly skilled and experienced Expert/Senior Engineers to join our TikTok Live Architecture team.

TikTok Live is a world-wide leader in live streaming, which occupies more than 50% of the market share.

In the LLM team, you have the chance to understand the most advanced LLM models, and design architecture to apply LLM in the world 's largest businesses.

We're people at the forefront of the world.

Responsibilities:
- Model Service Deployment: Responsible for converting large-scale deep learning models into scalable services that meet the diverse needs of TikTok's live streaming business.
- Performance Optimization: Optimize the performance of model inference, including but not limited to efficient utilization of computing resources, minimizing response time, and maximizing throughput.
- Cross-Team Collaboration: Work closely with algorithm and business teams to facilitate the deployment of models into production and resolve issues that arise in the production environment.
- Technical Innovation: Continuously monitor and explore new technologies and methods in the AI field to drive technological advancement in model services.



Minimum Qualifications:
- Bachelor's degree or higher in Computer Science, Software Engineering, Artificial Intelligence, or related fields.
- 3+ years of relevant work experience, with experience in deploying and servicing large-scale machine learning models.
- Proficiency in mainstream deep learning frameworks (such as TensorFlow, PyTorch, DeepSpeed) and their deployment in production environments.
- Familiarity with model inference optimization techniques, such as quantization, distillation, distributed inference, ONNX, ZeRO, etc.
- Familiarity with online service tech stacks, such as RPC, Redis, Kafka, etc.
- Strong programming skills, proficient in Python, C++ or Golang, with a deep understanding of system performance optimization.

Preferred Qualification:
- Have LLMs deployment and optimization experience