From 3086a205bef0f1e5966cc58984c7e86f375523c9 Mon Sep 17 00:00:00 2001 From: dskvr Date: Mon, 15 Jul 2024 14:57:34 +0200 Subject: [PATCH 1/7] NIP-119 --- 119.md | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) create mode 100644 119.md diff --git a/119.md b/119.md new file mode 100644 index 0000000000..1481533eb0 --- /dev/null +++ b/119.md @@ -0,0 +1,38 @@ +# NIP-119: AND Operator in Filters + +Enable `AND` within a single tag filter by using an `&` modifier in filters for indexable tags. + +``` +filters: { + "kinds": [1], + "&t": ["meme", "cat"], + "#t": ["black", "white"] +} +// returns kind `1` events with `t` tags that have both "meme" and "cat" that have the tag "black" or "white" +``` + +## Rationale +- Reduce bandwidth for all, with an emphasis on mobile users: `meme AND cat` objectively consumes less bandwidth than `meme OR cat ` +- Reduce clock-time for relays, indexing with `AND` is fast for all common index formats, and faster compared to `OR` for some index formats. (See section below) +- Reduce client-side caching requirements +- Reduce centralization vectors by reducing or even eliminating the need for centralized REST, GraphQL APIs or specialized relay "feed" endpoints. +- Give relays the option to be more useful at the protocol level while improving efficiency for all parties. + +## Rules + +- `AND` **MUST** take precedence over `OR` +- Tags used in `AND` **SHOULD NOT** be used in standard `OR` tags [`#`] +- Any tag used in `AND` **SHOULD** be ignored in `OR` + +## Considerations + +- New field for `NIP-11.limitations`: `max_tags_per_and` and `max_tags_and` +- Benchmarking should be conducted to validate that bandwidth and protocol usability as benefits supercede implementation and clock-time cost. + +## Index Efficiency +| Index Type | AND Operation Efficiency | OR Operation Efficiency | Notes | +|----------------|--------------------------|-------------------------|-------| +| B-Tree | High | Moderate | B-Tree indexes are very efficient for AND operations, especially with compound indexes. For OR operations, they are less efficient than for AND, as the database engine might need to traverse multiple paths. | +| Bitmap | High | High | Bitmap indexes excel in both AND and OR operations, particularly for columns with low cardinality. They utilize fast bitwise operations, making them ideal for read-heavy environments. | +| Hash | Not Applicable | Not Applicable | Hash indexes are designed for equality checks and do not directly support range-based queries or optimize for AND/OR operations efficiently. | +| Full-Text | High | High | Optimized for text search, full-text indexes efficiently handle both AND and OR conditions, making them suitable for complex text queries. | \ No newline at end of file From b0491854f89dbd841c122ac139673c9a9b93dc64 Mon Sep 17 00:00:00 2001 From: dskvr Date: Thu, 29 Aug 2024 16:55:30 +0200 Subject: [PATCH 2/7] remove 'considerations' and 'index efficiency' --- 119.md | 15 +-------------- 1 file changed, 1 insertion(+), 14 deletions(-) diff --git a/119.md b/119.md index 1481533eb0..d5fe598339 100644 --- a/119.md +++ b/119.md @@ -22,17 +22,4 @@ filters: { - `AND` **MUST** take precedence over `OR` - Tags used in `AND` **SHOULD NOT** be used in standard `OR` tags [`#`] -- Any tag used in `AND` **SHOULD** be ignored in `OR` - -## Considerations - -- New field for `NIP-11.limitations`: `max_tags_per_and` and `max_tags_and` -- Benchmarking should be conducted to validate that bandwidth and protocol usability as benefits supercede implementation and clock-time cost. - -## Index Efficiency -| Index Type | AND Operation Efficiency | OR Operation Efficiency | Notes | -|----------------|--------------------------|-------------------------|-------| -| B-Tree | High | Moderate | B-Tree indexes are very efficient for AND operations, especially with compound indexes. For OR operations, they are less efficient than for AND, as the database engine might need to traverse multiple paths. | -| Bitmap | High | High | Bitmap indexes excel in both AND and OR operations, particularly for columns with low cardinality. They utilize fast bitwise operations, making them ideal for read-heavy environments. | -| Hash | Not Applicable | Not Applicable | Hash indexes are designed for equality checks and do not directly support range-based queries or optimize for AND/OR operations efficiently. | -| Full-Text | High | High | Optimized for text search, full-text indexes efficiently handle both AND and OR conditions, making them suitable for complex text queries. | \ No newline at end of file +- Any tag used in `AND` **SHOULD** be ignored in `OR` \ No newline at end of file From c7da9d7409f25cdb34be1ea7d7aded259cb65e7f Mon Sep 17 00:00:00 2001 From: Sandwich <299465+dskvr@users.noreply.github.com> Date: Mon, 23 Sep 2024 23:43:48 +0200 Subject: [PATCH 3/7] remove rationale --- 119.md | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/119.md b/119.md index d5fe598339..3ab120a251 100644 --- a/119.md +++ b/119.md @@ -11,15 +11,8 @@ filters: { // returns kind `1` events with `t` tags that have both "meme" and "cat" that have the tag "black" or "white" ``` -## Rationale -- Reduce bandwidth for all, with an emphasis on mobile users: `meme AND cat` objectively consumes less bandwidth than `meme OR cat ` -- Reduce clock-time for relays, indexing with `AND` is fast for all common index formats, and faster compared to `OR` for some index formats. (See section below) -- Reduce client-side caching requirements -- Reduce centralization vectors by reducing or even eliminating the need for centralized REST, GraphQL APIs or specialized relay "feed" endpoints. -- Give relays the option to be more useful at the protocol level while improving efficiency for all parties. - ## Rules - `AND` **MUST** take precedence over `OR` - Tags used in `AND` **SHOULD NOT** be used in standard `OR` tags [`#`] -- Any tag used in `AND` **SHOULD** be ignored in `OR` \ No newline at end of file +- Any tag used in `AND` **SHOULD** be ignored in `OR` From b792e36eebc4df0464d5290c3dda20f38a3d7242 Mon Sep 17 00:00:00 2001 From: Sandwich <299465+dskvr@users.noreply.github.com> Date: Fri, 18 Oct 2024 10:47:57 +0200 Subject: [PATCH 4/7] Update 119.md --- 119.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/119.md b/119.md index 3ab120a251..df54085d11 100644 --- a/119.md +++ b/119.md @@ -14,5 +14,5 @@ filters: { ## Rules - `AND` **MUST** take precedence over `OR` -- Tags used in `AND` **SHOULD NOT** be used in standard `OR` tags [`#`] -- Any tag used in `AND` **SHOULD** be ignored in `OR` +- Tag values used in `AND` **SHOULD NOT** be used in standard `OR` tags [`#`] +- Any tag value used in `AND` **SHOULD** be ignored in `OR` From 6280c81f24904719c82fa8bbdb5c5527ed1968ad Mon Sep 17 00:00:00 2001 From: dskvr Date: Thu, 20 Nov 2025 09:37:03 -0600 Subject: [PATCH 5/7] Add 91.md, add Filter table to README --- 119.md => 91.md | 0 README.md | 12 ++++++++++++ 2 files changed, 12 insertions(+) rename 119.md => 91.md (100%) diff --git a/119.md b/91.md similarity index 100% rename from 119.md rename to 91.md diff --git a/README.md b/README.md index dd2e812ac1..5f8a2c3b72 100644 --- a/README.md +++ b/README.md @@ -82,6 +82,7 @@ They exist to document what may be implemented by [Nostr](https://github.com/nos - [NIP-84: Highlights](84.md) - [NIP-89: Recommended Application Handlers](89.md) - [NIP-90: Data Vending Machines](90.md) +- [NIP-91: AND filters](91.md) - [NIP-92: Media Attachments](92.md) - [NIP-94: File Metadata](94.md) - [NIP-96: HTTP File Storage Integration](96.md) @@ -201,6 +202,17 @@ They exist to document what may be implemented by [Nostr](https://github.com/nos [lnpub]: https://github.com/shocknet/Lightning.Pub/blob/master/proto/autogenerated/client.md [joinstr]: https://gitlab.com/1440000bytes/joinstr/-/blob/main/NIP.md +## Filters + +| kind | description | NIP | +| --------------- | -------------------------- | ------------------------ | +| `authors` | by pubkey | [01](01.md) | +| `ids` | by ids | [01](01.md) | +| `kinds` | by kinds | [01](01.md) | +| `#{}` | by indexable tag (OR) | [01](01.md) | +| `&{}` | by indexable tag (AND) | [91](91.md) | +| `search` | by search times | [50](50.md) | + ## Message types ### Client to Relay From c5c075059ea915f71bd3e77519fadd30ea82f465 Mon Sep 17 00:00:00 2001 From: dskvr Date: Thu, 20 Nov 2025 10:15:44 -0600 Subject: [PATCH 6/7] hotfix: heading NIP-119 -> NIP-91 --- 91.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/91.md b/91.md index df54085d11..af90e15d70 100644 --- a/91.md +++ b/91.md @@ -1,4 +1,4 @@ -# NIP-119: AND Operator in Filters +# NIP-19: AND Operator in Filters Enable `AND` within a single tag filter by using an `&` modifier in filters for indexable tags. From 09846e09c608f44d7d6651c8de8e71e15f3a8bc7 Mon Sep 17 00:00:00 2001 From: sandwich Date: Mon, 2 Mar 2026 20:56:54 +0100 Subject: [PATCH 7/7] fix: heading --- 91.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/91.md b/91.md index af90e15d70..7ac15ae281 100644 --- a/91.md +++ b/91.md @@ -1,4 +1,4 @@ -# NIP-19: AND Operator in Filters +# NIP-91: AND Operator in Filters Enable `AND` within a single tag filter by using an `&` modifier in filters for indexable tags.