The process of running a trained AI model to generate a prediction or output, as opposed to training the model itself.
When you send a prompt to an LLM API and get a response back, that's inference. Inference cost and speed are major practical considerations when building AI-powered products.
