diff --git a/README.md b/README.md index f6fae93e..5b2933ad 100644 --- a/README.md +++ b/README.md @@ -136,6 +136,29 @@ Preview URL: https://skill-deploy-abc123.vercel.app Claim URL: https://vercel.com/claim-deployment?code=... ``` +### openweb + +Typed JSON access to 90+ real websites including Google, Amazon, Reddit, YouTube, GitHub, Instagram, Bloomberg, and Zillow. Backed by the openweb CLI which calls the same APIs the websites call internally — no browser automation, no vision API. + +**Use when:** +- "Look up the price of X on Amazon" +- "Search Reddit for posts about Y" +- "Get my Bloomberg watchlist" +- "Post a comment on Hacker News" +- "Find flights from SFO to NRT" +- Reading user-private data behind a login (Reddit feed, GitHub issues, watchlists) +- Adding agent web access without browser-automation overhead + +**Categories covered:** +- Search (Google, DuckDuckGo, Bing, Brave, Kagi) +- Shopping (Amazon, Walmart, eBay, Target, Costco, Home Depot) +- Travel (Booking, Google Flights, Airbnb, Expedia, Tripadvisor) +- Finance (Bloomberg, Yahoo Finance, Robinhood, Coinbase, Xueqiu) +- Social (Reddit, X, Instagram, Hacker News, Threads) +- News & content (YouTube, Wikipedia, Medium, Substack) +- Dev tools (GitHub, GitLab, Stack Overflow, npm) +- Real estate (Zillow, Redfin, Realtor) + ## Installation ```bash diff --git a/skills/openweb.zip b/skills/openweb.zip new file mode 100644 index 00000000..fbede664 Binary files /dev/null and b/skills/openweb.zip differ diff --git a/skills/openweb/SKILL.md b/skills/openweb/SKILL.md new file mode 100644 index 00000000..670dd2eb --- /dev/null +++ b/skills/openweb/SKILL.md @@ -0,0 +1,74 @@ +--- +name: openweb +description: Typed JSON access to 90+ real websites — Google, Amazon, Reddit, YouTube, GitHub, Instagram, Bloomberg, Zillow, and more — across search, shopping, travel, finance, social, news, and dev tools. Use to read, search, post, comment, message, or otherwise interact with these sites (prices, products, articles, stock quotes, flights, posts, comments, DMs, etc.). Backed by the openweb CLI which calls the same APIs the websites call internally. +metadata: + author: openweb-org + version: "0.1.5" + homepage: "https://getopenweb.com" + repository: "https://github.com/openweb-org/openweb" +--- + +# OpenWeb + +Agent-native way to access any website. Calls the same APIs the website calls — typed JSON in, typed JSON out, no browser automation, no vision API. + +## Setup + +```bash +npm install -g @openweb-org/openweb +``` + +For sites that need authenticated access, OpenWeb starts a managed Chrome instance and reuses the user's existing browser session (auth cookies, JWTs, CSRF tokens auto-resolved). The source profile is never modified. + +## Use existing site + +```bash +openweb sites # list 90+ available sites +openweb # show transport, auth, operations +openweb # show params + permission tier +openweb --example # real example params +openweb '{"key":"value"}' # execute (stdout = JSON, stderr = JSON error) +``` + +Responses over 4096 bytes auto-spill to a temp file (`{status, output: , size, truncated}`). + +## Permission tiers + +Check tier before executing: + +| Tier | Default | Behavior | +|---|---|---| +| `read` | allow | GET-like operations — execute freely | +| `write` | prompt | Creates/updates — ask user before executing | +| `delete` | prompt | Destructive — ask user before executing | +| `transact` | deny | Financial/irreversible — always skip | + +Override in `$OPENWEB_HOME/config.json` (`permissions.defaults.*` or `permissions.sites..*`). + +## On failure + +Errors on stderr include `failureClass`. Common ones: + +| failureClass | Action | +|---|---| +| `needs_browser` | Browser auto-starts; if it fails, check Chrome installation | +| `needs_login` | `openweb login ` then `openweb browser restart` | +| `bot_blocked` | `openweb browser restart --no-headless`, solve CAPTCHA, retry | +| `permission_required` | Ask user for confirmation, then retry | +| `retriable` | Wait a few seconds, retry (max 2) | + +## Missing site or coverage + +If a site is not in `openweb sites`, do NOT say "unsupported." OpenWeb is extensible — install the full skill bundle (which includes an `add-site` workflow for adding new sites through the agent) from the canonical repo: + +```bash +curl -fsSL https://raw.githubusercontent.com/openweb-org/openweb/main/install-skill.sh | bash +``` + +## Full agent skill bundle + +This SKILL.md is a thin entry point. The full skill (workflows, knowledge files for auth/extraction/bot-detection patterns, add-site authoring guide) lives at the canonical repo: + +- Repo: https://github.com/openweb-org/openweb +- Skill source: https://github.com/openweb-org/openweb/tree/main/skills/openweb +- Full installer (above) sets it up in `~/.claude/skills/openweb/`, `~/.agents/skills/openweb/`, etc.