Grass

The problem

AI companies need residential bandwidth to collect training data without hitting anti-bot systems that saturate datacenter IPs. Individual users have surplus bandwidth they don't monetize. The market to connect these two sides needs infrastructure — routing, payment, verification, provider quality — that neither AI companies nor bandwidth suppliers want to build themselves. Every prior attempt at bandwidth markets has stalled at the same set of engineering problems. Solving those problems requires making bandwidth attributable (which provider delivered which request), verifiable (proof the bandwidth was actually used), and payable (settlement at a granularity that reflects the actual delivery).

Key challenges

Bandwidth marketplaces have hard problems in every direction. Routing has to balance latency, geography, and cost across a heterogeneous provider base that changes constantly. Verification has to prove the bandwidth was actually delivered without trusting the provider. Payment has to settle at per-request granularity or providers can't calibrate their contributions economically. And the whole thing has to scale to thousands of concurrent providers without becoming an ops nightmare.

What we built

AR Data contributed engineering on Grass's infrastructure — the routing, verification, and payment plumbing that turns residential bandwidth into a usable AI training data pipeline. The engineering focus was on scale (thousands of concurrent providers), reliability (requests either succeed cleanly or fail loudly), and attribution (every request tracked to the provider that fulfilled it). Routing balances geographic constraints, provider capacity, and per-request cost. Verification uses request-response attestation with sampling for high-volume flows. Payment clears per verified request with provider-side dashboards showing earnings in near-real-time. The whole system runs on Kubernetes with per-provider isolation.

Our approach

1. 1

   Geography-aware routing

   Bandwidth quality is regional. Routing decisions incorporate geography, latency, and provider capacity to match requests to the right supply.

2. 2

   Attestation with sampling for high-volume flows

   Verifying every request individually doesn't scale. Sampling with statistical guarantees keeps verification cost manageable.

3. 3

   Per-request payment settlement

   Providers need to see earnings at request granularity to calibrate their contributions. Aggregating settlement loses too much signal.

4. 4

   Per-provider isolation at the infra layer

   Providers are heterogeneous and occasionally hostile. Isolation prevents one bad provider from degrading service for the rest.

Key architectural decisions

Results

• High-scale bandwidth routing across thousands of providers
• Provider verification with sampling-based attestation
• Per-request payment settlement with near-real-time dashboards
• AI training data pipelines with attribution and quality controls
• Per-provider infrastructure isolation
• Kubernetes-based operational surface at network scale

Impact

Grass is the class of infrastructure that decides whether alternative bandwidth markets can serve serious workloads. The engineering patterns we contributed — routing, verification, per-request settlement — are the primitives every future permissionless-supply marketplace will need. It's also part of the depth we bring to any distributed-supply engineering engagement.

Tech stack