From a21da000d236e41ca11fba7b769a3ba0ff8c428d Mon Sep 17 00:00:00 2001 From: Qi Guo Date: Tue, 21 Apr 2026 22:54:29 -0400 Subject: [PATCH] Add openweb skill for typed JSON access to 90+ websites MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds skills/openweb — a thin entry-point skill backed by the openweb CLI (npm: @openweb-org/openweb). Returns typed JSON from 90+ real websites across search, shopping, travel, finance, social, news, and dev tools. For sites that need authenticated access, the CLI starts a managed Chrome instance and reuses the user's existing browser session; auth (cookies, JWT, CSRF, request signing, exchange chains) is auto-resolved. Operations are gated by a four-tier permission system (read / write / delete / transact) configurable via $OPENWEB_HOME/config.json. The full skill bundle (workflows, knowledge files, add-site authoring guide) lives at the canonical repo (https://github.com/openweb-org/openweb) and can be installed via the documented one-liner. --- README.md | 23 +++++++++++++ skills/openweb.zip | Bin 0 -> 1921 bytes skills/openweb/SKILL.md | 74 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 97 insertions(+) create mode 100644 skills/openweb.zip create mode 100644 skills/openweb/SKILL.md diff --git a/README.md b/README.md index f6fae93e..5b2933ad 100644 --- a/README.md +++ b/README.md @@ -136,6 +136,29 @@ Preview URL: https://skill-deploy-abc123.vercel.app Claim URL: https://vercel.com/claim-deployment?code=... ``` +### openweb + +Typed JSON access to 90+ real websites including Google, Amazon, Reddit, YouTube, GitHub, Instagram, Bloomberg, and Zillow. Backed by the openweb CLI which calls the same APIs the websites call internally — no browser automation, no vision API. + +**Use when:** +- "Look up the price of X on Amazon" +- "Search Reddit for posts about Y" +- "Get my Bloomberg watchlist" +- "Post a comment on Hacker News" +- "Find flights from SFO to NRT" +- Reading user-private data behind a login (Reddit feed, GitHub issues, watchlists) +- Adding agent web access without browser-automation overhead + +**Categories covered:** +- Search (Google, DuckDuckGo, Bing, Brave, Kagi) +- Shopping (Amazon, Walmart, eBay, Target, Costco, Home Depot) +- Travel (Booking, Google Flights, Airbnb, Expedia, Tripadvisor) +- Finance (Bloomberg, Yahoo Finance, Robinhood, Coinbase, Xueqiu) +- Social (Reddit, X, Instagram, Hacker News, Threads) +- News & content (YouTube, Wikipedia, Medium, Substack) +- Dev tools (GitHub, GitLab, Stack Overflow, npm) +- Real estate (Zillow, Redfin, Realtor) + ## Installation ```bash diff --git a/skills/openweb.zip b/skills/openweb.zip new file mode 100644 index 0000000000000000000000000000000000000000..fbede6649dc4c7dcf2e653ee279deb9044e8b07f GIT binary patch literal 1921 zcmZ{lcQhM_AHZX!R1m}tN~%^-YL6o5AzUe?QX^DsVg{u~Rr_V;((VkN7 zw25(MdOBhjEhSO0s_psxw7vJcd-q%GmmWZTqwitf(}d}S`^CXl{OfIzabYM2{n zNUW}zK#jmWy-2hot7^mom&W|=JQj0 zJ$HLssX=AdgdIf*?+qL4s;$hn($h5RnD!(5bf&40obvq1d7(3xNcPhJ&X~;oPKFHb z%~U*1#Iwx6Bg+YIfbH!S0PFdV1dSlf<*c>q#my>Rl6}$wCl8?~o*)xCts#D+s_$%1 z?R4j<-`#r4wVCGI{XwQn?M&o4UDvivU$--0r3P*2phQ~oz1AQJ)Gu53NodvcV_j~Q zuH(Jm#0r|#CAmX6B2Dl4u=w)!&rb>u(oYRY{FagPetZz-MBp3uQRw&0<0NrTpFlO< z1imxE@n?fNSp6FoXmuH7Mx}PEknE89im95fD0te6*JMKwbe8|=v!q1rUpGH|9u=PO zvSq@T+X!QLL`eo#l=_FRcys@9iP|D#@l7T0-5K|@O55U)<|>>|E*n@`em{Jp;?Mmw zdlIj-A`wpTLt>RW+t_&X)I)joi5p@gY1qy-TTPmo|20PbbY7F0-mbDEb~=Q^IeOowVwM%9yv%GZT% z9RPRIin_Owb~?l#e~`+3W-=tgSnYo)v=)I#YiG1RRFRetF{a7u63)6Y_t>aGZ-^KD zrhD^R$(^ja9ZA_i_-V1mW#jH!a$$#8F~jUTet8S$)B47Rl%pENqtzSk4kslCxD7`; zE^Z8SUOopBbk|Fg9)N-+_2M)?iYv`1LMDDF4#zE1@bOKiMS_O@mQ|1Y;R~==Jd8ml z>kV=){5BjTts063dj=$*af9zp%!)Tsw^kw9ZY!Rj@F{WEupd6Ye!swey7u#cbHI+n zXpwo@q^eH*z=h&vjV{dtSP!0U0sY1bZFX;< z%DR{mWMLTGld3P(_d?#U^nvU#OnzSnJG?q9G_mgsFsSj!;84k#53Y`JBkuK1X=jxh)hdo(m9URR zZ}?Gp#v)PVmS!Vrb2yUz`v4-FyH5z~{HE^se96T5Y8-L#pe>$$okFeB8lO4f+rPlULuQ^M&LjCMK(;?jJmcHrR!pEdpvFbA_WQfJ;=jTgBMK!}x z8iQdR^=+au=5IPS3{pGwj_D+gSubx)^9UJU8U0RM+ruSjDU>6Pe^+ zN&^F!mS5JtwZ`*JYyT3_H`@24_KkL=KR``7{2#Qx756 # show transport, auth, operations +openweb # show params + permission tier +openweb --example # real example params +openweb '{"key":"value"}' # execute (stdout = JSON, stderr = JSON error) +``` + +Responses over 4096 bytes auto-spill to a temp file (`{status, output: , size, truncated}`). + +## Permission tiers + +Check tier before executing: + +| Tier | Default | Behavior | +|---|---|---| +| `read` | allow | GET-like operations — execute freely | +| `write` | prompt | Creates/updates — ask user before executing | +| `delete` | prompt | Destructive — ask user before executing | +| `transact` | deny | Financial/irreversible — always skip | + +Override in `$OPENWEB_HOME/config.json` (`permissions.defaults.*` or `permissions.sites..*`). + +## On failure + +Errors on stderr include `failureClass`. Common ones: + +| failureClass | Action | +|---|---| +| `needs_browser` | Browser auto-starts; if it fails, check Chrome installation | +| `needs_login` | `openweb login ` then `openweb browser restart` | +| `bot_blocked` | `openweb browser restart --no-headless`, solve CAPTCHA, retry | +| `permission_required` | Ask user for confirmation, then retry | +| `retriable` | Wait a few seconds, retry (max 2) | + +## Missing site or coverage + +If a site is not in `openweb sites`, do NOT say "unsupported." OpenWeb is extensible — install the full skill bundle (which includes an `add-site` workflow for adding new sites through the agent) from the canonical repo: + +```bash +curl -fsSL https://raw.githubusercontent.com/openweb-org/openweb/main/install-skill.sh | bash +``` + +## Full agent skill bundle + +This SKILL.md is a thin entry point. The full skill (workflows, knowledge files for auth/extraction/bot-detection patterns, add-site authoring guide) lives at the canonical repo: + +- Repo: https://github.com/openweb-org/openweb +- Skill source: https://github.com/openweb-org/openweb/tree/main/skills/openweb +- Full installer (above) sets it up in `~/.claude/skills/openweb/`, `~/.agents/skills/openweb/`, etc.