Inferencing has become ubiquitous across cloud, regional, edge, and device environments, powering a wide spectrum of AI use cases spanning vision, language, and traditional machine learning applications. In recent years, Large Language Models (LLMs), initially developed for natural language tasks, have expanded to multimodal applications including vision speech, reasoning and planning each demanding distinct service-level objectives (SLOs). Achieving high-performance inferencing for such diverse workloads requires both model-level and system-level optimizations. This talk focuses on system-level optimization techniques that maximize token throughput , achieve user experience metrics and inference service-provider efficiency. We review several recent innovations including KV caching, Paged/Flash/Radix Attention, Speculative Decoding, P/D Disaggregation, KV Routing and Parallelism, and explain how these mechanisms enhance performance by reducing latency, memory footprint, and compute overhead. These techniques are implemented in leading open-source inference frameworks such as vLLM, SGLang, Hugging Face TGI, and NVIDIA’s TensorRT-llm, which form the backbone of large-scale public and private LLM serving platforms. Attendees will gain a practical understanding of the challenges in delivering scalable, low-latency LLM inference, and of the architectural and algorithmic innovations driving next-generation high-performance inference systems. Virtual: https://events.vtools.ieee.org/m/516797

Come celebrate the holiday season with cornhole, spikeball, cookie decorating, and catered food from Side Street Inn. It’s a great chance to relax, hang out, and connect with members from the UH Mānoa Student Branch, HKN, and the Hawaii Young Professionals community. Bldg: Holmes Hall, 2540 Dole Street, Honolulu, Hawaii, United States, 96813