Compare commits

...
Author SHA1 Message Date
Brooklyn Nicholson 52d07f5415 feat(skills): add food-delivery skill for Uber Eats + Instacart
Browser-driven food and grocery ordering on the real consumer sites —
neither service exposes a self-serve consumer ordering API, so the
Hermes managed browser (headed once for login, then the persistent
profile) is the path for both. One skill, one cookie store, a hard
confirm-before-pay gate.

Instacart's official Developer Platform API is the only non-browser
shortcut: an optional, key-gated helper that builds a "Shop with
Instacart" checkout link (search/cart only — never places/pays).

References capture the headless reality (PerimeterX/DataDome block
headless + cloud browsers), the login-then-operate pattern, and prior
art, including why we drive our own browser instead of the existing
MCP servers (which are Playwright under the hood anyway).
2026-06-15 13:05:36 -05:00
7 changed files with 582 additions and 0 deletions
+126
View File
@@ -0,0 +1,126 @@
---
name: food-delivery
description: "Order food and groceries via Uber Eats and Instacart."
version: 1.0.0
author: Brooklyn + Hermes Agent
license: MIT
platforms: [linux, macos, windows]
metadata:
hermes:
tags: [food, delivery, groceries, ubereats, instacart, ordering, browser]
category: productivity
requires_toolsets: [browser]
---
# Food Delivery Skill
Order restaurant food (Uber Eats) and groceries (Instacart) by driving the
real consumer sites in the Hermes browser with your already-logged-in
session. There is no self-serve consumer ordering API for either service, so
the browser is the path: it sees the same menus, prices, fees, and saved
payment methods you do.
This skill builds and reviews carts and places orders **only after you
confirm**. It does not enter new payment cards or addresses — it uses what's
already saved on the account. It is not a partner/merchant integration.
## When to Use
- "order me dinner from <restaurant>", "get my usual", "reorder last night's"
- "order groceries", "add milk, eggs, and bananas to Instacart", "restock coffee"
- "what's the delivery ETA / where's my order" → check an in-progress order
- Service routing: restaurant/prepared food → **Uber Eats**; grocery/store
items → **Instacart**. If ambiguous, ask.
## Prerequisites
- The `browser` toolset is configured and `browser_navigate` works.
- A **persistent browser profile** (Camofox managed-persistence, the default
managed browser) so a login survives across turns and sessions. Without it
you'll re-login every task.
- You are logged in to the service once in the Hermes browser. First run walks
you through it; OTP/passkey/CAPTCHA steps are handed to you (see Pitfalls).
- A saved delivery address and payment method on the account. This skill will
not type in a new card.
- Optional power-up (no key needed for the browser flow): an Instacart
Developer Platform key in `INSTACART_IDP_API_KEY` enables a structured
"Shop with Instacart" cart link via `scripts/instacart_link.py` — see
`references/instacart-api.md`.
## How to Run
Drive the page with the `browser` tools. The loop is always: `browser_snapshot`
to get fresh element refs → act (`browser_click` / `browser_type` /
`browser_press`) → re-snapshot. Refs go stale after any navigation, so never
reuse a ref across steps. When the accessibility tree is ambiguous (item cards,
image-heavy menus, modifier modals), use `browser_vision` to read the rendered
page.
Per-service walkthroughs with URLs and on-page landmarks:
- Uber Eats → `references/ubereats-flow.md`
- Instacart → `references/instacart-flow.md`
- Headless reality, session reuse, prior art → `references/headless-and-sessions.md`
Login is the only step that truly needs a visible browser + the user (CAPTCHA /
2FA / passkey). Once logged in, the persistent profile keeps you authenticated,
so the rest runs unattended.
## Quick Reference
| Intent | Service | Entry URL |
|---|---|---|
| Order restaurant food | Uber Eats | `https://www.ubereats.com` |
| Track a food order | Uber Eats | `https://www.ubereats.com/orders` |
| Order / add groceries | Instacart | `https://www.instacart.com/store` |
| Track a grocery order | Instacart | `https://www.instacart.com/store/account/orders` |
| Structured cart link (opt-in API) | Instacart | `terminal`: `python scripts/instacart_link.py …` |
## Procedure
1. **Route** the request to a service (table above). Confirm the store/restaurant
when the user named one loosely ("the taco place") by searching on-site.
2. **Open** the entry URL with `browser_navigate`, then `browser_snapshot`.
3. **Check auth.** If the snapshot shows a logged-out state (Sign in / Log in
landmarks), go to the login page and **hand control to the user** for
credentials, OTP, or passkey — do not guess or loop. Once logged in, the
persistent profile keeps you in for later turns.
4. **Confirm the address.** Delivery address drives availability, prices, and
fees. If it's not the user's intended one, set it before building the cart.
5. **Build the cart.** Search items / open the restaurant, add each item, and
apply required modifiers (size, options, substitutions). For "my usual" or
"reorder", use the order history / Reorder control.
6. **Review out loud.** Summarize cart contents, subtotal, **delivery fee,
service fee, taxes, and tip**, plus the ETA. Surface Instacart item
substitutions and any surge/busy fees explicitly.
7. **Confirm before paying.** State the final total and ask the user to confirm.
Place the order (Place order / Checkout) **only on an explicit yes**.
8. **Verify** (below) and report the confirmation number + ETA.
## Pitfalls
- **Never place an order, change the tip, or switch payment without explicit
user confirmation.** This spends real money.
- **Stale refs.** Any click that navigates invalidates prior refs —
`browser_snapshot` again before the next action.
- **Auth walls.** Logged-out mid-flow, OTP, passkey, or CAPTCHA → stop and ask
the user to complete it. Don't retry the same failing action; report the
blocker and the next step.
- **No fully-headless end-to-end.** Both sites run PerimeterX/DataDome; headless
and cloud browsers get blocked. Use the real managed browser, headed for
login. See `references/headless-and-sessions.md`.
- **Address/store changes everything.** Switching the delivery address can
re-price the cart or make items unavailable. Set it first, re-verify after.
- **Instacart substitutions.** Items get substituted or go out of stock; set or
confirm the substitution preference and surface it in the review.
- **Fees are not the subtotal.** Always report delivery + service fees + tax +
tip, not just item prices.
## Verification
- After placing, `browser_navigate` to the order-tracking URL and
`browser_snapshot` (or `browser_vision`) to capture the **confirmation
number and ETA**; report both back to the user.
- For the optional API path, a successful `scripts/instacart_link.py` run prints
a `products_link_url` — give that link to the user to complete checkout on
Instacart.
@@ -0,0 +1,63 @@
# Headless, sessions & prior art
## Verdict
Fully headless end-to-end **does not work**. Uber Eats and Instacart sit behind
PerimeterX (HUMAN) / DataDome, which fingerprint headless at the TLS, WASM
CPU-timing, and behavioral layers — JS stealth patches don't cover it. Cloud
browsers (Browserbase, Browser Use) get CAPTCHA'd; a real local browser passes.
## The pattern everyone converges on
Two phases. Only the first needs a visible browser + a human:
1. **Login (headed, once).** Human solves CAPTCHA / 2FA / passkey. The
`browser` toolset's persistent profile (Camofox managed-persistence) keeps
the session alive across turns — same idea as the prior-art repos' profile
dirs.
2. **Operate (effectively headless).** With the authenticated session, the bulk
of work — search, cart, checkout prep — needs no visible interaction.
So: drive login headed, then run cart/search unattended. Final place-order keeps
a browser fallback.
## Fast lane: replay the site's own JSON/GraphQL (optional, fragile)
The fastest projects skip the DOM entirely once logged in and replay the web
client's internal calls with the session cookie:
- Uber Eats: `addItemsToDraftOrderV2` / `removeItemsFromDraftOrderV2`, then the
checkout/order endpoints (`matiasconcha11/uber_eats_mcp`).
- Instacart: Apollo persisted-query hashes, e.g. `UpdateCartItemsMutation`,
driven over GET (queries) / POST (mutations) — sub-second vs 2040s of DOM
automation (`mdwoicke/cli-printing-press-library`).
Trade-off: these endpoints and persisted-query hashes are undocumented and
change without notice. Prefer the robust DOM flow; reach for replay only when
speed matters and you can absorb the breakage.
## Prior art
| Repo | Service | Approach |
|---|---|---|
| `amrezo/mcp-ubereats` | Uber Eats | Playwright, persisted cookies, `confirm` gate |
| `matiasconcha11/uber_eats_mcp` | Uber Eats | Web JSON API + Playwright for login/fallback |
| `markswendsen-code/mcp-instacart` | Instacart | Playwright, stealth, `confirm=true` to place |
| `@striderlabs/mcp-instacart` | Instacart | Playwright, persistent profile, MFA |
| `mdwoicke/cli-printing-press-library` | Instacart | GraphQL replay via existing Chrome session |
| `Keeeeeeeks/trenchcoat-mpp` | Multi | Local real Chrome over CDP — "only thing that works" |
Shared takeaways: persist the session, never auto-place an order, expect UI/API
drift. All of which this skill already does.
## Why this skill, not those MCP servers
The MCP servers above *are* Playwright under the hood — browser-vs-browser, not
API-vs-browser. Driving Hermes' own managed browser instead wins on every axis:
one login + one cookie store (theirs is a separate profile you'd log into
twice), no extra Node process or supply-chain surface, Camofox stealth ≥ their
bundled Chromium, and selectors we control rather than a stranger's. The one
real asymmetry isn't MCP — it's that **Instacart has an official self-serve API**
(`references/instacart-api.md`) and Uber Eats has none. So: managed browser for
both, Instacart's official API as the only non-browser shortcut. Reach for an
external MCP only to outsource selector upkeep — and then for both, never split.
@@ -0,0 +1,47 @@
# Instacart Developer Platform API — optional power-up
The browser flow needs **no key**. This path is opt-in: it turns a known item
list into a one-tap "Shop with Instacart" link. It never places or pays for an
order — the user opens the link and checks out on Instacart.
## When to prefer it
- The user gave an explicit list ("tortillas, ground beef, limes, cilantro")
and you want a clean handoff link instead of hand-driving the cart.
- You're composing a recipe/meal into a shoppable list.
For "order my usual", live price checks, or tracking an order, use the browser
flow (`references/instacart-flow.md`).
## Setup
1. Get a self-serve key at the Instacart Developer Platform
(`https://docs.instacart.com/developer_platform_api`).
2. Store it as a secret (it's a credential, so it belongs in `.env`):
```bash
hermes setup # add INSTACART_IDP_API_KEY when prompted
# or append to ~/.hermes/.env: INSTACART_IDP_API_KEY=ic_...
```
## Use
```bash
python scripts/instacart_link.py \
--title "Taco night" \
--item "tortillas:8:count" \
--item "ground beef:1:pound" \
--item lime:6:count \
--instruction "Pick ripe avocados"
```
Each `--item` is `name[:quantity[:unit]]`. The script POSTs to
`/idp/v1/products/products_link` and prints a `products_link_url`. Pass `--dev`
to use the development host while testing.
## Capabilities & limits
- **Can:** product search, build a cart/shopping-list, generate a checkout
link, list nearby retailers, recipe pages.
- **Cannot:** place an order, pay, or read someone's order status — those need
the partner-gated Connect API. The link hands checkout to the user.
@@ -0,0 +1,51 @@
# Instacart — browser flow
The self-serve Developer Platform API does product search and **checkout
links** but cannot place or pay for an order programmatically (full
cart→pay→fulfillment is the partner-gated Connect API). So for actual
ordering, drive `https://www.instacart.com` with the `browser` tools. For a
structured "Shop with Instacart" link from a known item list, the optional API
path in `references/instacart-api.md` is faster — but it still hands final
checkout back to the user.
## URLs
| Purpose | URL |
|---|---|
| Store picker / browse | `https://www.instacart.com/store` |
| Order history & tracking | `https://www.instacart.com/store/account/orders` |
| Login | reached via the "Log in" button on the home page |
## On-page landmarks
- **Logged-out:** "Log in" in the header. Hand login to the user (email + OTP,
Google/Apple SSO, or passkey).
- **Store/retailer:** Instacart is multi-retailer — pick the store first
(Costco, Safeway, etc.); availability and price are per-store.
- **Address/ZIP:** drives which stores and delivery windows are available; set
it before building the cart.
- **Search:** per-store search field; add items via the "Add" / "+" control on
each product card (`browser_vision` helps read dense product grids).
- **Cart:** cart button (top-right) → "Go to checkout".
- **Checkout:** delivery window, **service fee + delivery fee + tax + tip**,
and the substitution preference, then "Place order".
- **Reorder:** "Buy it again" / order history surfaces past items.
## Flow
1. `browser_navigate``/store`; `browser_snapshot`.
2. Verify login; set address/ZIP; pick the retailer.
3. Search and add each item; set quantities and substitution preference.
4. Open the cart → "Go to checkout".
5. Read back items + subtotal + delivery + service fee + tax + tip + delivery
window, and flag any out-of-stock / substituted items.
6. On explicit user confirmation, click "Place order".
7. `browser_navigate` → order history, capture confirmation + delivery window.
## Notes
- Substitutions are core to Instacart: confirm "best match", "specific
replacement", or "refund" before placing.
- Instacart+ membership, minimum-basket, and busy-pricing fees show at
checkout — surface them.
- Some items are alcohol/age-restricted and need ID at delivery; flag to user.
@@ -0,0 +1,50 @@
# Uber Eats — browser flow
No consumer ordering API exists (the Consumer Delivery API is early-access,
NDA + written Uber approval only). Drive the consumer site at
`https://www.ubereats.com` with the `browser` tools.
## URLs
| Purpose | URL |
|---|---|
| Home / search | `https://www.ubereats.com` |
| Order history & tracking | `https://www.ubereats.com/orders` |
| Login | `https://auth.uber.com/login` (reached via the Sign in button) |
## On-page landmarks
Snapshot and match on visible text/roles rather than memorized refs — Uber
ships frequent layout and A/B changes.
- **Logged-out:** a "Sign in" / "Log in" link in the header. Hand login to the
user (phone + OTP, or passkey).
- **Address:** a delivery-address button in the header (shows the current
address). Click it to search and select the intended address before browsing.
- **Restaurant search:** the top search field ("Food, groceries, drinks, etc.").
- **Item add:** clicking a menu item opens a modal — set required options, then
"Add to cart" (or "Add 1 to order").
- **Cart:** the cart button (top-right). "Go to checkout" advances to the
checkout page.
- **Checkout:** review address, delivery time, **fees + tip**, then the final
"Place order" button.
- **Reorder:** order history (`/orders`) exposes a "Reorder" control per past
order — fastest path for "my usual".
## Flow
1. `browser_navigate` → home; `browser_snapshot`.
2. Verify login; set the delivery address.
3. Search the restaurant → open it → add items, handling modifier modals
(`browser_vision` if the modal is image-heavy or ambiguous).
4. Open the cart → "Go to checkout".
5. Read back items + subtotal + delivery fee + service fee + tax + tip + ETA.
6. On explicit user confirmation, click "Place order".
7. `browser_navigate``/orders`, capture confirmation number + ETA.
## Notes
- Surge/"busy area" fees appear at checkout — surface them before confirming.
- Scheduled vs. ASAP delivery is chosen at checkout; default to ASAP unless the
user asked to schedule.
- Tip defaults to a preselected percentage; confirm or adjust per the user.
@@ -0,0 +1,126 @@
#!/usr/bin/env python3
"""Optional power-up: build an Instacart shoppable cart link via the IDP API.
The browser flow needs no keys. This is for users who set
``INSTACART_IDP_API_KEY`` (a self-serve Instacart Developer Platform key) and
want a structured, one-tap "Shop with Instacart" link instead of driving the
cart by hand. The API never places or pays for an order — it returns a
``products_link_url`` the user opens to check out on Instacart.
Usage:
python instacart_link.py --title "Taco night" --item "tortillas:8:count" \\
--item "ground beef:1:pound" --item lime:6:count
Each ``--item`` is ``name[:quantity[:unit]]``. Reads the API key from
INSTACART_IDP_API_KEY (env or ~/.hermes/.env). Pass --dev to hit the
development host.
"""
from __future__ import annotations
import argparse
import json
import os
import sys
import urllib.error
import urllib.request
from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple
PROD_BASE = "https://connect.instacart.com"
DEV_BASE = "https://connect.dev.instacart.tools"
PRODUCTS_LINK_PATH = "/idp/v1/products/products_link"
ENV_KEY = "INSTACART_IDP_API_KEY"
def api_base(dev: bool = False) -> str:
return DEV_BASE if dev else PROD_BASE
def parse_item(spec: str) -> Dict[str, Any]:
"""``name[:quantity[:unit]]`` -> a line_items entry. Name may contain spaces."""
parts = [p.strip() for p in spec.split(":")]
name = parts[0]
if not name:
raise ValueError(f"item spec missing a name: {spec!r}")
item: Dict[str, Any] = {"name": name}
if len(parts) > 1 and parts[1]:
item["quantity"] = float(parts[1]) if "." in parts[1] else int(parts[1])
if len(parts) > 2 and parts[2]:
item["unit"] = parts[2]
return item
def build_products_link_request(
title: str,
items: List[str],
*,
api_key: str,
dev: bool = False,
instructions: Optional[List[str]] = None,
image_url: Optional[str] = None,
) -> Tuple[str, Dict[str, str], Dict[str, Any]]:
"""Return ``(url, headers, body)`` for the products_link call. No network."""
if not title:
raise ValueError("title is required")
line_items = [parse_item(s) for s in items]
if not line_items:
raise ValueError("at least one --item is required")
body: Dict[str, Any] = {"title": title, "link_type": "shopping_list", "line_items": line_items}
if instructions:
body["instructions"] = instructions
if image_url:
body["image_url"] = image_url
headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
return f"{api_base(dev)}{PRODUCTS_LINK_PATH}", headers, body
def resolve_api_key() -> str:
key = os.environ.get(ENV_KEY)
if key:
return key
env_path = Path(os.environ.get("HERMES_HOME", Path.home() / ".hermes")) / ".env"
if env_path.exists():
for line in env_path.read_text(encoding="utf-8").splitlines():
if line.startswith(f"{ENV_KEY}="):
return line.split("=", 1)[1].strip().strip('"').strip("'")
raise SystemExit(
f"{ENV_KEY} not set. Use the browser flow (no key needed), or add the key "
"to ~/.hermes/.env — see references/instacart-api.md."
)
def create_products_link(title: str, items: List[str], *, dev: bool = False, **kw: Any) -> Dict[str, Any]:
url, headers, body = build_products_link_request(
title, items, api_key=resolve_api_key(), dev=dev, **kw
)
req = urllib.request.Request(url, data=json.dumps(body).encode(), headers=headers, method="POST")
try:
with urllib.request.urlopen(req, timeout=30) as resp: # noqa: S310 - fixed Instacart host
return json.loads(resp.read().decode())
except urllib.error.HTTPError as e:
raise SystemExit(f"Instacart API {e.code}: {e.read().decode(errors='replace')}") from e
def main(argv: Optional[List[str]] = None) -> int:
ap = argparse.ArgumentParser(description="Build an Instacart shoppable cart link (IDP API).")
ap.add_argument("--title", required=True)
ap.add_argument("--item", dest="items", action="append", default=[], help="name[:quantity[:unit]]")
ap.add_argument("--instruction", dest="instructions", action="append", default=[])
ap.add_argument("--image-url")
ap.add_argument("--dev", action="store_true", help="use the development host")
args = ap.parse_args(argv)
out = create_products_link(
args.title,
args.items,
dev=args.dev,
instructions=args.instructions or None,
image_url=args.image_url,
)
print(json.dumps({"products_link_url": out.get("products_link_url"), "raw": out}, indent=2))
return 0
if __name__ == "__main__":
sys.exit(main())
+119
View File
@@ -0,0 +1,119 @@
from __future__ import annotations
import importlib.util
import re
import sys
from pathlib import Path
import pytest
SKILL_DIR = (
Path(__file__).resolve().parents[2]
/ "skills"
/ "productivity"
/ "food-delivery"
)
SKILL_MD = SKILL_DIR / "SKILL.md"
SCRIPT_PATH = SKILL_DIR / "scripts" / "instacart_link.py"
def load_module():
spec = importlib.util.spec_from_file_location("food_delivery_instacart_link", SCRIPT_PATH)
module = importlib.util.module_from_spec(spec)
assert spec.loader is not None
sys.modules[spec.name] = module
spec.loader.exec_module(module)
return module
def frontmatter() -> dict[str, str]:
text = SKILL_MD.read_text(encoding="utf-8")
block = text.split("---", 2)[1]
return {
m.group(1): m.group(2).strip()
for m in re.finditer(r"^([a-zA-Z_]+):\s*(.*)$", block, re.MULTILINE)
}
def test_description_within_limit_and_well_formed():
desc = frontmatter()["description"].strip('"')
assert len(desc) <= 60, f"{len(desc)} chars: {desc!r}"
assert desc.endswith(".")
assert "food-delivery" not in desc.lower()
def test_required_sections_present():
body = SKILL_MD.read_text(encoding="utf-8")
for heading in (
"## When to Use",
"## Prerequisites",
"## How to Run",
"## Quick Reference",
"## Procedure",
"## Pitfalls",
"## Verification",
):
assert heading in body, f"missing section: {heading}"
def test_referenced_files_exist():
for rel in (
"references/ubereats-flow.md",
"references/instacart-flow.md",
"references/instacart-api.md",
"references/headless-and-sessions.md",
"scripts/instacart_link.py",
):
assert (SKILL_DIR / rel).exists(), f"missing referenced file: {rel}"
def test_confirmation_gate_is_documented():
"""Spending money must require explicit user confirmation."""
body = SKILL_MD.read_text(encoding="utf-8").lower()
assert "confirm" in body
assert "never place an order" in body
def test_parse_item_variants():
mod = load_module()
assert mod.parse_item("lime") == {"name": "lime"}
assert mod.parse_item("ground beef:1:pound") == {"name": "ground beef", "quantity": 1, "unit": "pound"}
assert mod.parse_item("oil:1.5:liter")["quantity"] == 1.5
with pytest.raises(ValueError):
mod.parse_item(":2:count")
def test_build_products_link_request_shape():
mod = load_module()
url, headers, body = mod.build_products_link_request(
"Taco night",
["tortillas:8:count", "lime"],
api_key="ic_test",
instructions=["ripe avocados"],
)
assert url == "https://connect.instacart.com/idp/v1/products/products_link"
assert headers["Authorization"] == "Bearer ic_test"
assert body["title"] == "Taco night"
assert body["link_type"] == "shopping_list"
assert body["line_items"][0] == {"name": "tortillas", "quantity": 8, "unit": "count"}
assert body["instructions"] == ["ripe avocados"]
def test_build_products_link_request_dev_host_and_validation():
mod = load_module()
url, _, _ = mod.build_products_link_request("x", ["a"], api_key="k", dev=True)
assert url.startswith("https://connect.dev.instacart.tools")
with pytest.raises(ValueError):
mod.build_products_link_request("", ["a"], api_key="k")
with pytest.raises(ValueError):
mod.build_products_link_request("t", [], api_key="k")
def test_resolve_api_key_reads_env_file(tmp_path, monkeypatch):
mod = load_module()
monkeypatch.delenv("INSTACART_IDP_API_KEY", raising=False)
home = tmp_path / ".hermes"
home.mkdir()
(home / ".env").write_text('INSTACART_IDP_API_KEY="ic_from_file"\n', encoding="utf-8")
monkeypatch.setenv("HERMES_HOME", str(home))
assert mod.resolve_api_key() == "ic_from_file"