Member of Technical Staff, Inference
Apply on company site View on the company's hiring page (jobs.ashbyhq.com)
Posted by Inferact on the hiring system it uses for its careers page; applications go straight to the company.
Can I apply?
- Full-time
- Remote from anywhere
- Visa sponsorship available
Remote (fully remote, worldwide) Member of Technical Staff role building and optimizing the vLLM inference engine for LLM and diffusion model serving.
Requires Python and PyTorch internals experience, plus hands-on work with LLM inference systems such as vLLM, TensorRT-LLM, SGLang or TGI.
Bachelor's degree or equivalent experience is required; regular overlap with Pacific Time is expected for critical syncs.
Visa sponsorship is offered case-by-case, so non-resident applicants may be considered.
From the posting
- workplace
ATS workplace type: remote
- locations
ATS location: Remote
- remote eligible countries
Fully remote, worldwide.
- remote timezone requirement
expect regular overlap with Pacific Time for critical syncs
- work auth required
We sponsor visas on a case-by-case basis.
- employment types
<ats_employment_type>full_time</ats_employment_type>
- education required
Bachelor's degree or equivalent experience in computer science, engineering, or similar.
- skills
Experience with LLM inference systems (vLLM, TensorRT-LLM, SGLang, TGI).
- skills
Strong programming skills in Python with experience in PyTorch internals.
- job function
We're looking for an inference runtime engineer
- role family
optimizing how models execute across diverse hardware and architectures
- actionable
ABOUT THE ROLE
Job description
Inferact's mission is to grow vLLM as the world's AI inference engine and accelerate AI progress by making inference cheaper and faster. Founded by the creators and core maintainers of vLLM, we sit at the intersection of models and hardware—a position that took years to build.
ABOUT THE ROLE
We're looking for an inference runtime engineer to push the boundaries of what's possible in LLM and diffusion model serving. Models grow larger. Architectures shift: mixture-of-experts, multimodal, agentic. Every breakthrough demands innovations on the inference engine itself. You'll work at the core of vLLM, optimizing how models execute across diverse hardware and architectures. Your work will directly impact how the world runs AI inference.
SKILLS AND QUALIFICATIONS
Minimum qualifications:
- Bachelor's degree or equivalent experience in computer science, engineering, or similar.
- Deep understanding of transformer architectures and their variants.
- Strong programming skills in Python with experience in PyTorch internals.
- Experience with LLM inference systems (vLLM, TensorRT-LLM, SGLang, TGI).
- Ability to read and implement model architectures and inference techniques from research papers.
- Demonstrate the ability to contribute performant and maintainable code and debug in complex ML codebases.
Preferred qualifications:
- Deep understanding of KV-cache memory management, prefix caching, and hybrid model serving.
- Familiarity with RL frameworks and algorithms for LLMs.
- Experience with multimodal inference (audio/image/video/text).
- Contributions to open-source ML or system infrastructure projects.
Bonus points if you have:
- Implemented core features in vLLM or other inference engine projects.
- Contributed to vLLM integrations (verl, OpenRLHF, Unsloth, LlamaFactory, etc).
- Written widely-shared technical blogs or side projects on vLLM or LLM inference.
LOGISTICS
- Location: Fully remote, worldwide. We're timezone-flexible but expect regular overlap with Pacific Time for critical syncs.
- Compensation: We offer competitive compensations (salary + equity) compared to the local market conditions.
- Visa sponsorship: We sponsor visas on a case-by-case basis.
- Benefits: Inferact offers competitive benefits appropriate to your location, including health coverage where applicable.
Seen on
- ashby — jobs.ashbyhq.com