Measuring per-inference energy on a Jetson without an external meter

2026-06-28

To compare what two DVFS operating points cost per inference, you’d normally clamp a probe on a rail and integrate. On a deployed Jetson Orin there’s usually no bench meter available. The onboard INA3221 power monitor is enough — if you read the right rail at a fine enough interval and calibrate to the workload’s duty cycle. Here’s the method, and the two places the naive version goes wrong.

Why `tegrastats` is insufficient for a single inference

tegrastats is the usual source of Jetson power numbers, and for a few-millisecond inference on a periodic loop it has two problems:

Resolution. It updates about every 100 ms. A ~5 ms inference is a twentieth of one sample, so a single reading spans many inferences plus the idle between them — you measure a window that is mostly not an inference.
Idle averaging. A periodic workload at, say, ~33 Hz with a ~5 ms compute has the GPU busy roughly a sixth of each period and idle the rest. A 100 ms sample is dominated by that idle power, so the active-power difference between two clocks is buried under a large, nearly constant idle floor. The number looks stable while mostly reporting idle.

For resolving a single inference you need a finer instrument and a duty-aware formula.

Read VDD_IN directly through `hwmon`

The INA3221 exposes three rails through hwmon, readable as fast as the chip updates (~1 ms). For module-level cost, use VDD_IN — the whole module’s input. Power per sample is voltage * current:

# locate the ina3221 hwmon, then per sample:  power_mW = voltage_mV * current_mA / 1000
p_vdd_in = rd('in1_input') * rd('curr1_input') / 1000.0   # VDD_IN  (module input)
p_cpugpu = rd('in2_input') * rd('curr2_input') / 1000.0   # VDD_CPU_GPU_CV
p_soc    = rd('in3_input') * rd('curr3_input') / 1000.0   # VDD_SOC

One thing to state honestly: the hwmon update interval is 1 ms, but under load you do not get 1000 distinct samples a second — closer to ~100–800 Hz, because the read path competes with the workload. Describe it as a ~1 ms-interval onboard sample, not a “1 kHz” instrument. It is still one to two orders of magnitude finer than tegrastats, which is all you need to resolve a 5 ms inference.

Calibrate to the duty cycle

What “energy per inference” means depends on whether the GPU idles between inferences, so there are two formulas:

Periodic, with idle gaps (a deployed vision loop). The clock is pinned for the run, so the average power already folds in both the active spikes and the idle floor at that clock. Per-inference energy spreads that average over the release rate:

E_per_inference = mean VDD_IN power / release_rate     # mW / (1/s) -> mJ

Saturated, continuously busy (LLM token decode). No idle to account for; each inference runs back-to-back, so energy is power times the time it takes:

E_per_inference = mean active power * compute_latency

Trim warmup and edges, average a few repeats, and you have calibrated per-inference joules from a sensor already on the board.

The duty cycle decides the sign of “faster vs cheaper”

A common assumption is race-to-idle: a higher clock finishes sooner, idles sooner, saves energy. That holds only when the clock actually drops after the work finishes.

When a governor pins one clock for the whole run, there is no race-to-idle — the higher clock draws more power the entire time, idle gaps included. So for a periodic vision workload, the higher-clock operating point costs more per inference (in one deployment, +1.9 to +7.2 mJ/inf), not less. For a saturated decode the same arithmetic runs the other way: no idle to lose, so the faster clock genuinely finishes each token cheaper. The duty cycle sets the sign, which is why you must pick the matching formula rather than “just integrate the power.”

A fine measurement also surfaces non-monotonic cases a coarse one can’t: on one L2-resident kernel the lower memory clock was both faster and lower-energy — 368 vs 410 mJ/inf at 2133 vs 3199 MHz — a strict win from turning a clock down, visible only because the measurement resolved it.

Know the scope of the rail

VDD_IN is the module input: CPU, GPU, SoC, memory, and conversion losses — everything. That’s the right rail for “what does this policy cost the deployed system per inference,” a question of ordering and magnitude that a robot’s battery actually pays. It is not an isolation of “GPU energy” or “memory energy”; for that you’d use the component rails (VDD_CPU_GPU_CV, VDD_SOC), and even those are not a substitute for an instrumented bench supply. Quote VDD_IN for policy cost, cross-check ordering on the compute rail, and don’t claim absolute component energy from an onboard sensor.

Takeaways

tegrastats is too coarse for one inference (~100 ms vs ~5 ms) and averages idle into the active number — on a low-duty periodic loop that can dominate the reading.
Read the INA3221 through hwmon (VDD_IN = in1_input * curr1_input / 1000), ~1 ms interval. Call it ~1 ms-interval, not 1 kHz: under load you get ~100–800 effective samples/s.
The per-inference formula depends on duty cycle. Periodic with idle gaps: mean power / release rate. Saturated: mean active power * compute latency. The wrong one flips the sign.
Race-to-idle requires the clock to drop after the work. When the governor pins the clock for the whole run, the higher clock costs more for periodic duty and less for saturated duty.
VDD_IN is the module input — total policy cost, not isolated component energy. It answers what the deployed system pays per inference; don’t oversell it as a bench meter.