-
Notifications
You must be signed in to change notification settings - Fork 17
Add EnsRainbowBeam app with endpoint for submitting labels #2015
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
djstrong
wants to merge
32
commits into
main
Choose a base branch
from
2003-add-new-app-with-endpoint-for-submitting-labels
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 14 commits
Commits
Show all changes
32 commits
Select commit
Hold shift + click to select a range
19b42ed
feat: add ens-labels-collector app for label submission and classific…
djstrong c57108e
feat: increase label submission limits and update related tests
djstrong c0c013e
chore: update pnpm-lock.yaml with new dependency hashes and version a…
djstrong 3537486
feat(yoga): add masked error handling for production environment
djstrong 440023c
feat(ens-labels-collector): enhance error handling and add timeout fo…
djstrong 4b49138
chore(pnpm-lock): add ensnode-sdk dependency for ens-labels-collector
djstrong 912c16c
Merge branch 'main' into 2003-add-new-app-with-endpoint-for-submittin…
djstrong 3021d5c
feat(ens-labels-collector): enhance timeout handling in label lookups
djstrong 8c41088
refactor: improve configuration parsing and logging
djstrong 4634cc4
refactor(ens-labels-collector): streamline configuration handling and…
djstrong 0b5365b
docs(ens-labels-collector): clarify submission limits and normalizati…
djstrong ff194f9
feat(ens-labels-collector): improve error handling for omnigraph lookups
djstrong 2adbc92
Merge branch 'main' into 2003-add-new-app-with-endpoint-for-submittin…
djstrong 74ff599
Merge branch 'main' into 2003-add-new-app-with-endpoint-for-submittin…
djstrong 053913b
update zod
djstrong 2005200
Merge branch 'main' into 2003-add-new-app-with-endpoint-for-submittin…
djstrong caf277d
rename app to ensrainbowbeam
djstrong a723be4
Add `EnsRainbowBeam` app with `/api/discover` endpoint for label subm…
djstrong d643370
Refactor label handling in submissions: update `SubmissionsRequestSch…
djstrong 7597168
Enhance `Query.labels` functionality: Introduce stricter validation f…
djstrong 6e1f410
Enhance `LabelHash` parsing: Update `parseValue` to include error han…
djstrong 07c9d1f
lint
djstrong 33e3ed1
Add `EnsRainbowBeamClient` for HTTP interactions: Implement `health()…
djstrong 23b4eaa
Add tests for `EnsRainbowBeamClient` and `validateDiscoverParams`: Im…
djstrong b79863c
Add `EnsRainbowBeam` support: Integrate CORS handling, update deploym…
djstrong d01acae
lint
djstrong 2ae9acc
Refactor label submission handling: Update submissions to skip unnorm…
djstrong 5cfdb4a
Remove CORS handling from configuration and application logic: Update…
djstrong e8e4767
enables CORS for all origins
djstrong ba036a6
Merge branch 'main' into 2003-add-new-app-with-endpoint-for-submittin…
djstrong e7f9df5
refactor(ensrainbowbeam): update label submission processing to norma…
djstrong 5b92ad7
refactor(ensrainbowbeam): update LabelHit structure to use 'labelhash…
djstrong File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -11,6 +11,7 @@ | |
| "ensrainbow", | ||
| "ensapi", | ||
| "fallback-ensapi", | ||
| "ens-labels-collector", | ||
| "enssdk", | ||
| "enscli", | ||
| "enskit", | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| --- | ||
| "ens-labels-collector": minor | ||
|
djstrong marked this conversation as resolved.
Outdated
|
||
| "ensapi": minor | ||
| "enssdk": minor | ||
| --- | ||
|
|
||
|
|
||
| Add `apps/ens-labels-collector` and a new `Query.labels` Omnigraph field to support label submission collection (issue [#2003](https://github.com/namehash/ensnode/issues/2003)). | ||
|
|
||
| - **New app `apps/ens-labels-collector`**: Hono server exposing `POST /api/submissions` that accepts `{ labels: string[], callerAddress: Address }`, classifies each label against ENSNode's index via the typed `enssdk/omnigraph` client, and emits a structured JSON line per submission to stdout. For each submitted raw label the collector computes both the literal labelhash and (when normalizable to a different value) the normalized labelhash, then assigns one of three statuses per label: `unknown_in_index` (referenced in the index but unhealed), `healed_in_index`, or `absent_from_index`. Persistent storage, batched on-chain emission, and a caller-leaderboard are explicitly deferred to follow-up work; the JSON log shape is the future row shape so adding a sink later is mechanical. | ||
| - **New ENSApi `Query.labels(by: { hashes: [Hex!]! }): [Label!]!`**: batch lookup of `Label` rows by `LabelHash`. Hashes that are not present in the index are simply omitted from the result. Capped at 200 hashes per request. | ||
| - **`enssdk/omnigraph`**: regenerated GraphQL introspection so the new `Query.labels` field is available to the typed `graphql(...)` client. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| # Port the ens-labels-collector HTTP server listens on. | ||
| PORT=4444 | ||
|
|
||
| # Base URL of an ENSNode (ENSApi) instance that exposes the Omnigraph GraphQL endpoint. | ||
| # The collector calls `${ENSNODE_URL}/api/omnigraph` to classify submitted labels. | ||
| ENSNODE_URL=http://localhost:4334 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| FROM node:24-slim AS base | ||
| ENV PNPM_HOME="/pnpm" | ||
| ENV PATH="$PNPM_HOME:$PATH" | ||
| RUN corepack enable | ||
| WORKDIR /app | ||
|
|
||
| FROM base AS deps | ||
| COPY . . | ||
| RUN --mount=type=cache,id=pnpm,target=/pnpm/store pnpm install --frozen-lockfile | ||
|
|
||
| FROM deps AS runner | ||
| WORKDIR /app/apps/ens-labels-collector | ||
| ENV NODE_ENV=production | ||
| EXPOSE 4444 | ||
|
|
||
| CMD ["pnpm", "start"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| # ens-labels-collector | ||
|
|
||
| Receives ENS Label submissions from external callers, classifies each label against ENSNode's | ||
| indexed Label table, and (for now) emits a structured JSON line per submission to stdout. | ||
|
|
||
| The app is intentionally minimal; persistent storage, batched on-chain emission, and a | ||
| caller-leaderboard are explicitly deferred to follow-up work (see GitHub issue | ||
| [#2003](https://github.com/namehash/ensnode/issues/2003)). The submission JSONL shape is the | ||
| future row shape so adding a sink later is mechanical. | ||
|
|
||
| ## Endpoints | ||
|
|
||
| - `GET /health` — liveness probe; always returns `{ message: "ok" }`. | ||
| - `POST /api/submissions` — accepts `{ labels: string[], callerAddress: Address }` and | ||
| responds with per-label classification (`unknown_in_index` / `healed_in_index` / | ||
| `absent_from_index`). | ||
|
|
||
| ## How label classification works | ||
|
|
||
| For each submitted raw label the collector: | ||
|
|
||
| 1. Computes `labelhashLiteralLabel(rawLabel)`. | ||
| 2. If the label is normalizable AND the normalized form differs from the raw label, also | ||
| computes `labelhashLiteralLabel(normalizedLabel)`. | ||
| 3. Sends every distinct labelhash to ENSNode via the typed `enssdk/omnigraph` client using | ||
| the `labels(by: { hashes })` query. | ||
| 4. Classifies each submitted label: | ||
| - `unknown_in_index` — at least one of its hashes is present in the index but not yet | ||
| healed (i.e. `interpreted` is the encoded labelhash form). These are the interesting | ||
| submissions for future on-chain emission. | ||
| - `healed_in_index` — at least one of its hashes is present in the index and all | ||
| returned hits are already healed. | ||
| - `absent_from_index` — none of its hashes are present in the index. | ||
|
|
||
| ## Configuration | ||
|
|
||
| | Env var | Required | Description | | ||
| |---------|----------|-------------| | ||
| | `PORT` | no (default `4444`) | HTTP listen port. | | ||
| | `ENSNODE_URL` | yes | Base URL of an ENSNode (ENSApi) instance with Omnigraph at `/api/omnigraph`. | | ||
|
|
||
| See `.env.local.example` for a local-development template. | ||
|
|
||
| ## Development | ||
|
|
||
| ```bash | ||
| pnpm -F ens-labels-collector dev | ||
| pnpm -F ens-labels-collector typecheck | ||
| pnpm -F ens-labels-collector test | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| { | ||
| "private": true, | ||
| "name": "ens-labels-collector", | ||
| "version": "1.10.1", | ||
| "type": "module", | ||
| "description": "Collects ENS Label submissions and classifies them against ENSNode's Omnigraph index", | ||
| "license": "MIT", | ||
| "repository": { | ||
| "type": "git", | ||
| "url": "git+https://github.com/namehash/ensnode.git", | ||
| "directory": "apps/ens-labels-collector" | ||
| }, | ||
| "homepage": "https://github.com/namehash/ensnode/tree/main/apps/ens-labels-collector", | ||
| "scripts": { | ||
| "start": "tsx src/index.ts", | ||
| "dev": "tsx watch --env-file ./.env.local src/index.ts", | ||
| "test": "vitest", | ||
| "lint": "biome check --write .", | ||
| "lint:ci": "biome ci", | ||
| "typecheck": "tsgo --noEmit" | ||
| }, | ||
| "dependencies": { | ||
| "@ensnode/ensnode-sdk": "workspace:*", | ||
| "@hono/node-server": "catalog:", | ||
| "enssdk": "workspace:*", | ||
| "graphql": "^16.11.0", | ||
| "hono": "catalog:", | ||
| "viem": "catalog:", | ||
| "zod": "catalog:" | ||
| }, | ||
| "devDependencies": { | ||
| "@ensnode/shared-configs": "workspace:*", | ||
| "@types/node": "catalog:", | ||
| "tsx": "^4.19.3", | ||
| "typescript": "catalog:", | ||
| "vitest": "catalog:" | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| import { Hono } from "hono"; | ||
|
|
||
| import { healthHandler } from "@/handlers/health"; | ||
| import { submissionsHandler } from "@/handlers/submissions"; | ||
| import { errorResponse } from "@/lib/error-response"; | ||
|
|
||
| const app = new Hono(); | ||
|
|
||
| app.get("/health", healthHandler); | ||
|
|
||
| app.post("/api/submissions", submissionsHandler); | ||
|
|
||
| app.notFound((c) => errorResponse(c, { message: "Not Found", status: 404 })); | ||
|
|
||
| app.onError((error, c) => { | ||
| console.error("[ens-labels-collector] unhandled error", error); | ||
| // Do not leak the underlying error message to clients; respond with a generic 500. | ||
| return errorResponse(c, { message: "Internal Server Error", status: 500 }); | ||
| }); | ||
|
|
||
| export default app; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| import { z } from "zod"; | ||
|
|
||
| import { OptionalPortNumberSchema } from "@ensnode/ensnode-sdk/internal"; | ||
|
|
||
| /** | ||
| * Default port for the ens-labels-collector HTTP server. Used when `PORT` env var is unset. | ||
| */ | ||
| export const ENS_LABELS_COLLECTOR_DEFAULT_PORT = 4444; | ||
|
|
||
| const ConfigSchema = z.object({ | ||
| PORT: OptionalPortNumberSchema.default(ENS_LABELS_COLLECTOR_DEFAULT_PORT), | ||
| ENSNODE_URL: z.string().url(), | ||
|
djstrong marked this conversation as resolved.
Outdated
|
||
| }); | ||
|
|
||
| const parsed = ConfigSchema.parse(process.env); | ||
|
|
||
| /** | ||
| * Process configuration parsed from `process.env` at module load. | ||
| * | ||
| * Throws (via Zod) if any required env var is missing or invalid. | ||
| */ | ||
| export const config = { | ||
| port: parsed.PORT, | ||
| ensNodeUrl: parsed.ENSNODE_URL, | ||
| }; | ||
|
|
||
| export type Config = typeof config; | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| import type { Context } from "hono"; | ||
|
|
||
| export function healthHandler(c: Context) { | ||
| return c.json({ message: "ok" }); | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe you may also need to make some updates to our release scripts / CI scripts in
.githubwhen adding a new app.