Edge-Cloud Ledger: Local, Edge, or Cloud?

Where should an inference task run — on a phone/edge box, or on cloud GPUs? The answer is not "the cloud is stronger"; it is a joint comparison of four ledgers: latency, quality, cost, bandwidth. This article turns "where to place" into a reproducible decision flow. Method framework references the open-source book Understanding AI Infra (Apache-2.0); formulas and examples are derived by this site and are reproducible.

1. Framework: Four Ledgers Together, Not One Optimum

Ledger	Local (edge small model)	Cloud (large model)
Latency	local inference latency (ms)	transport RTT + queue + cloud inference
Quality	small-model quality (parameter-discounted)	large-model quality (baseline)
Cost	one-time hardware + power	GPU rental × usage
Bandwidth	no egress	upload raw data (audio/video/long text)

Core takeaway: the trade-off point shifts with workload — latency-sensitive and data-heavy → local; quality-sensitive and high-utilization → cloud.

2. Latency Ledger: Find the Crossover

Whether edge inference is slower than "network round-trip + cloud inference" is the first gate:

$$T_{local} = T_{infer}(small), \quad T_{cloud} = RTT + T_{queue} + T_{infer}(large) + T_{upload}$$

Scenario	Local	Cloud path	Verdict
Voice wake/translation	10-50 ms	RTT 50-200 ms + queue	local (interaction needs ms)
Long-text summarization	seconds (small)	1-2 s (incl. transport)	depends on quality gap (next)
Video understanding	per-frame local	upload takes several × inference	pre-process locally first (see bandwidth)

Takeaway: interactive tasks (wait per step) usually have a latency budget <100ms — local is the only choice; offline batch workloads can consider cloud.

3. Quality Ledger: Trading Quality for Latency & Cost

Edge devices are power/memory-limited, so models are typically 1-2 orders smaller. Quality differences can't be exactly quantified, but parameter count as a quality proxy supports the decision:

$$Quality\_proxy = f(N_{params}), \quad 7B \ vs \ 70B: 10× scale, quality gap grows with task difficulty$$

• Simple tasks (classification, wake, template QA): small-model loss acceptable → local;
• Complex tasks (long-context reasoning, multi-turn, code): gap unacceptable → cloud, or use Distributed Inference Ledger's PD separation to leave heavy work to a cloud large model.

Takeaway: set a "quality floor" by task difficulty first; anything below the floor goes to cloud — don't save cost on quality.

4. Cost Ledger: Buy Hardware vs Rent GPU Hours

Edge deployment is fixed cost; cloud is metered. Breakeven:

$$BreakEven = \frac{Hardware\_cost}{Cloud\_cost\_per\_hour \times utilization}, \quad keep\ renting\ past\ the\ point$$

Variable	Example
Edge inference box (incl. development)	¥5,000-20,000
Cloud GPU inference rental	¥20-60/hour
Breakeven	100-1000 hours ≈ 4-40 days at full load

The key variable is utilization: low daily activity and sparse requests favor renting (see GPU Utilization Ledger); only 7×24 high load justifies owning.

5. Bandwidth Ledger: Process First, Then Upload

For data-heavy workloads (video, audio, image streams), uploading raw data often costs more than the inference itself:

$$Upload\_time = \frac{Data\_size}{Uplink\_BW}, \quad local\ pre-processing\ compresses\ 10-100×\ (frame\ sampling/downscale/transcribe)$$

Example: 1 hour of 1080p video ≈ 5-10 GB; at 10 Mbps uplink that takes 1-2 hours. Sampling key frames or transcribing locally first drops upload to MB scale — this is the quantitative rationale for the "edge pre-processing + cloud inference" hybrid.

6. Reproducible Python

def placement_decision(local_ms, cloud_rtt_ms, cloud_queue_ms, cloud_ms,
                       data_gb, uplink_mbps, hw_cost, cloud_cost_h, daily_hours):
    """Four-ledger joint decision: latency compare, upload time, breakeven (days)"""
    t_local = local_ms
    t_cloud = cloud_rtt_ms + cloud_queue_ms + cloud_ms + data_gb*8e3/(uplink_mbps/1000*60)
    upload_min = data_gb * 8 * 1000 / uplink_mbps / 60
    breakeven_days = hw_cost / (cloud_cost_h * daily_hours)
    return {"cloud_total_ms": round(t_cloud,1), "upload_min": round(upload_min,1),
            "breakeven_days": round(breakeven_days,1),
            "verdict": "local" if t_local < t_cloud else "cloud"}

placement_decision(local_ms=40, cloud_rtt_ms=80, cloud_queue_ms=50, cloud_ms=300,
                   data_gb=0.001, uplink_mbps=10, hw_cost=8000, cloud_cost_h=40, daily_hours=4)
# cloud_total_ms=430.1 → local 40ms wins → verdict=local
# breakeven_days=50 days of full load before owning pays off

7. Checklist

1. Set the latency budget first: interactive <100ms → local; batch → check the other three ledgers;
2. Set a quality floor by task difficulty; don't save cost on quality below the floor;
3. Use utilization for breakeven: low activity rents, 7×24 owns;
4. For data-heavy workloads, pre-process locally (sampling/transcribing) to cut uploads 10-100×;
5. The hybrid (edge pre-processing + cloud large model + PD separation) is usually the joint optimum.

---

Derived and organized by this site; method framework references the open-source book Understanding AI Infra (Apache-2.0). Version numbers and measurements are subject to on-site conditions.