How to Give AI Agents Live GPU Metrics on OKE, No Prometheus

You deploy a vLLM inference server on an Oracle Kubernetes Engine GPU node pool, then wire up Claude or Goose as your coding assistant. The agent writes code and reads Kubernetes logs without complaint — but ask it how the GPU on that node is doing, and it goes quiet.

That blind spot is common across GPU fleets on Oracle Cloud Infrastructure. The agent can query pod status and check CPU metrics through the Kubernetes API, but GPU utilization, memory pressure, temperature, and power draw stay invisible — on the single most expensive resource in the cluster.

This guide walks through closing that gap with an MCP server that reads metrics straight off the GPU hardware and exposes them as tools any MCP-compatible agent can call, without standing up a Prometheus and Grafana pipeline just to answer one question.

What MCP Adds That kubectl Can't Show You

Model Context Protocol (MCP) is an open standard created by Anthropic and adopted across the broader AI tooling ecosystem. It works like a universal connector for agents — a standardized way to plug data sources and tools into whatever model is doing the reasoning.

To close the GPU visibility gap specifically, Pavan Madduri — a senior cloud platform engineer at W.W. Grainger and a CNCF Golden Kubestronaut — built gpu-mcp-server. In his write-up, Madduri says he built it to read NVIDIA GPU metrics directly from NVML and expose them as tools any MCP-compatible agent can call — no Prometheus pipeline, no PromQL, no dashboard, the agent just asks.

Step 1: Build and Push the Image to OCIR

The DaemonSet needs an image Kubernetes can actually pull, so start with the container. Build the Go binary with cgo enabled — the NVML bindings that read GPU state directly require it — tag the image, and push it to Oracle Container Registry (OCIR) before you touch a single Kubernetes manifest.

# Illustrative example — adapt the registry path and tag to your tenancy
docker build -t <region>.ocir.io/<tenancy>/gpu-mcp-server:latest .
docker push <region>.ocir.io/<tenancy>/gpu-mcp-server:latest

Step 2: Deploy the DaemonSet Across Every GPU Node

Running one pod per node — not a central collector — keeps this out of Prometheus territory. The DaemonSet targets nodes with NVIDIA GPUs and, according to the project, tolerates the standard GPU taint OKE applies. Each pod calls NVML directly through Go's cgo bindings, with no sidecar collectors and no network hop to a central server.

The project also reports each pod consumes under 64MB of memory, since it only reads from the GPU driver rather than running a heavyweight collector or storing metrics itself.

# Illustrative DaemonSet sketch — adapt labels, image, and namespace to your cluster
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: gpu-mcp-server
spec:
  template:
    spec:
      nodeSelector:
        node.kubernetes.io/instance-type: gpu
      tolerations:
        - key: "nvidia.com/gpu"
          operator: "Exists"
      containers:
        - name: gpu-mcp-server
          image: <region>.ocir.io/<tenancy>/gpu-mcp-server:latest
          resources:
            limits: { memory: "128Mi" }
          securityContext:
            privileged: true

Step 3: Query the Tools the Agent Actually Calls

Before wiring anything into Claude, confirm what's queryable. gpu-mcp-server exposes four tools, and knowing what each one returns tells you — and the agent — exactly what questions are answerable.

A call to get_gpu_metrics against a live A10 node returns something like this:

{
  "index": 0,
  "name": "NVIDIA A10",
  "gpu_utilization_percent": 87,
  "memory_used_mib": 18432,
  "memory_total_mib": 24576,
  "temperature_celsius": 68,
  "power_draw_watts": 135,
  "power_limit_watts": 150,
  "pcie_tx_kbps": 524288,
  "pcie_rx_kbps": 262144
}

Step 4: Connect Claude Desktop and Put the Metrics to Work

The payoff isn't a JSON blob — it's an agent that can reason about placement and cost, not just print numbers. Port-forward the DaemonSet's endpoint locally, then point your MCP client's configuration at it.

kubectl port-forward daemonset/gpu-mcp-server 8080:8080 -n gpu-mcp-server

Then add the local endpoint to your MCP client's configuration — Claude Desktop, Cursor, and Goose all support a local MCP server entry — and restart the client.

OCI's shape lineup keeps expanding — VM.GPU.A10.1, BM.GPU.A10.4, BM.GPU.H100.8, and now the newer GB200 shapes — and Madduri argues that an agent able to query metrics across that mixed fleet, A10s for inference and H100s for training, and make informed placement calls is "genuinely useful, not just a demo."

That visibility pays off in a concrete way. Madduri writes that when an agent can see GPU 0 on a BM.GPU.A10.4 sitting idle, it can recommend moving the workload to a cheaper VM.GPU.A10.1 or scaling it down entirely — what he calls "the first step to GPU FinOps."

Madduri also shared his own rough hourly-rate estimates for context. Treat these as approximate and author-reported, not Oracle's official list pricing:

Before You Wire This Into Anything Critical

Read access to GPU metrics is low-risk. Give an agent write access to anything cluster-critical and the calculus changes fast, as other developers building MCP tooling elsewhere have already found.

"The first tool I gave an LLM was a way to look at a job queue I run. The queue lives in Redis. The second thing I wanted was a guarantee that looking was all the model could do: no enqueue, no delete, no retry."

Scope gaps cut the other way too — a team wiring Claude into Jira ran into a tool that was too narrow rather than too permissive, unable to post internal notes because, as they put it:

"the current Jira MCP server that exists doesn't support Jira Service Management, meaning we couldn't use Claude to post internal notes on tickets."

Every tool gpu-mcp-server exposes — list_gpus, get_gpu_metrics, get_gpu_processes, gpu_summary — reads state; none can act on the cluster. That's a reasonable default. Add a tool that can scale or kill a process later, and scope it separately from the read-only tools deployed here.

Start With One Node Pool

The mechanism here is simple: a DaemonSet reads NVML on every GPU node directly, an MCP server exposes that as callable tools, and your agent gets GPU state without a metrics pipeline standing between the hardware and the model.

Start smaller than a full rollout. Deploy the DaemonSet on one GPU node pool, port-forward it locally, and run list_gpus and get_gpu_metrics by hand before pointing Claude Desktop at it. Once the output looks right on one node, extending to the rest of the fleet is a label change, not a rewrite.