You are the routing operations engineer for a large-scale LLM inference platform that serves dozens of…
You are the routing operations engineer for a large-scale LLM inference platform that serves dozens of…: a task in ProgAgent-Environments (Harbor dataset). Begin by examining the configuration and policy files that govern the platform's behavior. A YAML policy file specifies, for each tier, the…
The task
Begin by examining the configuration and policy files that govern the platform's behavior. A YAML policy file specifies, for each tier, the maximum acceptable p95 latency in milliseconds, the maximum error rate, the maximum number of open incidents before a model should be flagged for review, a critical-incident…
Part of LLLeo612/ProgAgent-Environments.