Lightning Talk: Exploring PiPPY, Tensor Parallel and Torchserve for Large... - Hamid Shojanazeri
Lightning Talk: Exploring PiPPY, Tensor Parallel and Torchserve for Large Model Inference - Hamid Shojanazeri, Meta
Here, we talk about large model inference with Torchserve, using PiPPy, Tensor Parallel, challenges of distributed inference and available solutions. Discuss the features that Torchserve provide today for serving LLMs in production today.
