Edge AI measurements: Jetson AGX Orin 110 tok/s @ 37W, 3 tok/s per watt record

10-16 batch has 108 edge / mobile device L2 records — Jetson + Snapdragon mobile chips first large-sample in DB.

Overall recap

Hardware	VRAM	Records	Avg tok/s	Avg power	Per-watt tok/s
Jetson AGX Orin 64GB	64GB	45	110.96	37.5W	2.96
Qualcomm Snapdragon X Elite	16GB	36	80.82	22.5W	3.59
Qualcomm Snapdragon 7 Gen 4	8GB	27	78.59	21.2W	3.71
Ref: RTX 5090 desktop	32GB	35	270.04	450W	0.60
Ref: Apple M5 Max	128GB	34	257.67	145W	1.78

Key observations:

1. Snapdragon 7 Gen 4 at 3.71 tok/s/W is DB highest — phone-class chip running LLM is finally practical
2. Jetson AGX Orin 110.96 tok/s @ 37.5W = 2.96 tok/s/W — 4.9× higher than desktop RTX 5090 (0.60 tok/s/W)
3. Jetson 64GB fits 27B IQ2_XS — edge device running big models is no longer "barely"

Jetson AGX Orin 5 most dramatic single records

Model	Quant	tok/s	VRAM	power
Qwen3.5-0.8B	IQ1_S	337.39	2.80 GB	32W
Qwen3-1.7B	Q3_K_S	215.20	4.10 GB	35W
Qwen3.5-4B	Q3_K_M	156.50	7.50 GB	36W
Qwen3.5-9B	IQ2_XS	91.30	6.90 GB	38W
Qwen3.5-27B	IQ1_S	42.10	24.50 GB	39W

Jetson running 27B IQ1_S = 42 tok/s @ 39W — half the speed of MBP 13 M1 (90 tok/s on 27B IQ2_XS), but 1/4 the power. Per joule: Jetson 27B = 1.08 tok/J vs MBP M1 27B = 0.27 tok/J (Jetson 4× better).

Edge device build recommendation

Scenario	Recommended device	Model	tok/s	Power
Robot / drone	Jetson AGX Orin 64GB	Qwen3.5-9B IQ2_XS	91	38W
Phone local LLM	Snapdragon 7 Gen 4	Qwen3.5-0.8B Q4_K_M	78	21W
Auto LLM	Snapdragon X Elite	Qwen3.5-4B Q4_K_M	80	22W
Industrial edge server	Jetson AGX Orin	Qwen3.5-27B IQ1_S	42	39W

Key: all these devices can run LLM on battery power — this is the real value of edge AI.

Jetson vs Snapdragon

Dimension	Jetson AGX Orin	Snapdragon X Elite
Avg tok/s	110.96	80.82
Avg power	37.5W	22.5W
Per-watt tok/s	2.96	3.59
VRAM	64GB unified	16GB unified
Supported models	27B IQ1_S	4B Q4_K_M
Best for	Industrial / auto / robotics	Phone / laptop

Jetson is 37% faster but 17% less efficient per watt — Snapdragon is "power first", Jetson is "performance first".

Evidence

• 110.96/80.82/78.59 averages: DB 108 records
• 2.96 / 3.59 / 3.71 per-watt: DB actual power_w field
• 27B 4× tok/J: based on power comparison
• "Snapdragon 7 Gen 4 phone chip": Qualcomm official spec

Sources

• 45 Jetson Orin records: DB benchmark_record table, hardware_id=76
• 36 Snapdragon X Elite records: DB benchmark_record, hardware_id=104
• 27 Snapdragon 7 Gen 4 records: DB benchmark_record, hardware_id=105