If you want to deploy an LLM endpoint, it is critical to think about how different requests are going to be handled.