Vector db project - #7961
Conversation
The diagram also includes a new node to represent Vector Database project types.
Also update the trial (Sign up) page to include Vector Database
Elastic Docs AI PR menuCheck the box to run an AI review for this pull request.
Powered by GitHub Agentic Workflows and docs-actions. For more information, reach out to the docs team. |
Elastic Docs Style Checker (Vale)Summary: 2 warnings, 3 suggestions found
|
| File | Line | Rule | Message |
|---|---|---|---|
| solutions/vector-database/get-started.md | 35 | Elastic.Spelling | 'ecloud' is a possible misspelling. |
| solutions/vector-database/get-started.md | 55 | Elastic.DirectionalLanguage | Don't use directional language. Use 'the following element' instead of 'the steps below'. |
💡 Suggestions (3): Optional style improvements. Apply when helpful.
| File | Line | Rule | Message |
|---|---|---|---|
| deploy-manage/cloud-organization/billing/vector-database-billing-dimensions.md | 60 | Elastic.WordChoice | Consider using 'can, might' instead of 'may', unless the term is in the UI. |
| solutions/vector-database.md | 54 | Elastic.Semicolons | Use semicolons sparingly. Consider splitting the sentence or using a comma or conjunction. |
| solutions/vector-database.md | 69 | Elastic.Semicolons | Use semicolons sparingly. Consider splitting the sentence or using a comma or conjunction. |
The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.
There was a problem hiding this comment.
Thank you @yetanothertw, this already looks really good!I
I've left a few suggestions for your consideration. Many of them are related to the new search tutorial we're adding and how it might change the role of the Get started page, as well as the semantic field type and the multimodal search page/tutorial.
|
|
||
| * **[{{es-serverless}}](/solutions/search/get-started.md)**: Build powerful applications and search experiences using a rich ecosystem of vector search capabilities, APIs, and libraries. | ||
| * **[{{es-serverless}}](/solutions/search/get-started.md)**: Build powerful, scalable search and analytics applications across structured data, logs, metrics, documents, and vectors as part of a broader {{stack}}. | ||
| * **[{{es}} Vector Database](/solutions/vector-database/get-started.md)**: Build embedding-driven workloads such as semantic search, RAG, and AI-powered retrieval. Built-in models, `semantic_text`, and vector-optimized defaults mean less configuration and faster time to production. |
There was a problem hiding this comment.
I'm wondering whether we should mention semantic_text here. On one hand, it's a powerful tool we might want to promote (in this case, we could also mention the recently added semantic field type), but on the other hand, mentioning it only for the Vector DB project type could imply that it's exclusive to that one, which is not the case. So, for technical accuracy and to avoid misleading wording, we should either mention it for others too, or omit it entirely.
I'm leaning towards the latter, since we don't highlight other specific field types or configuration options either.
| | Your use case | What to use | Description | | ||
| | --- | --- | --- | | ||
| | Building search-powered applications | 1. [Core search capabilities](/solutions/search.md)<br><br> 2. [Elasticsearch solution](/solutions/elasticsearch-solution-project.md) | 1. Core {{es}} search features available across all deployment types, solutions, and project types<br><br>2. Additional UI tools that complement the core search features | | ||
| | Building search-powered applications | 1. [Core search capabilities](/solutions/search.md)<br><br> 2. [Elasticsearch solution](/solutions/elasticsearch-solution-project.md)<br><br> 3. [{{es}} Vector Database](/solutions/vector-database.md) {applies_to}`serverless: preview` | 1. Core {{es}} search features available across all deployment types, solutions, and project types<br><br>2. Additional UI tools that complement the core search features<br><br>3. Dedicated {{serverless-full}} project type for AI-powered retrieval (RAG, recommendations, semantic and hybrid search) with vector-tuned defaults | |
There was a problem hiding this comment.
This is not part of (and maybe scope for) this change, but I feel like we shouldn't present Core search capabilities in the "Choose your path" section. Since they're available for all deployment types, they're not really a choice in the same sense as choosing between Elasticsearch, Vector DB, etc. project types.
IMO, we should move the Core search capabilities part outside of this table, keep a similar explanation to what we have now in the Description column, and note that these features are available across all project and deployment types.
| | Your use case | What to use | Description | | ||
| | --- | --- | --- | | ||
| | Building search-powered applications | 1. [Core search capabilities](/solutions/search.md)<br><br> 2. [Elasticsearch solution](/solutions/elasticsearch-solution-project.md) | 1. Core {{es}} search features available across all deployment types, solutions, and project types<br><br>2. Additional UI tools that complement the core search features | | ||
| | Building search-powered applications | 1. [Core search capabilities](/solutions/search.md)<br><br> 2. [Elasticsearch solution](/solutions/elasticsearch-solution-project.md)<br><br> 3. [{{es}} Vector Database](/solutions/vector-database.md) {applies_to}`serverless: preview` | 1. Core {{es}} search features available across all deployment types, solutions, and project types<br><br>2. Additional UI tools that complement the core search features<br><br>3. Dedicated {{serverless-full}} project type for AI-powered retrieval (RAG, recommendations, semantic and hybrid search) with vector-tuned defaults | |
There was a problem hiding this comment.
Also (and sorry, this is not your change at all): I'v just realized we refer to "Core search capabilities" here, while nav presents the same concept as "Search use case". This might be a larger scope-change, but in the meantime, what about renaming "Core search capabilities" to "Search use case" to match the nav?
|
|
||
| [Vector search](/solutions/search/vector.md) uses the same query APIs in both project types. | ||
|
|
||
| Use it when embeddings and similarity search are central to your application, for example RAG, recommendations, [semantic search](/solutions/search/semantic-search.md), [hybrid search](/solutions/search/hybrid-search.md), or multimodal search. |
There was a problem hiding this comment.
We've just added a landing page for Multimodal search, we could link to that here as well.
|
|
||
| ### Access to {{infer}} | ||
|
|
||
| Vector Database projects are set up for embedding workflows: generate vectors in {{es}} with managed models (for example through `semantic_text`), or store vectors you create yourself and attach the same model at query time. In-product setup guides walk through both paths. |
There was a problem hiding this comment.
We could say semantic_text and semantic here, while linking to the related pages
| :::{note} | ||
| Not sure whether this project type is right for you? Refer to [When to use this project type](/solutions/vector-database.md#when-to-use-this-project-type). | ||
|
|
||
| If you're looking for an introduction to the {{stack}} or the {{es}} product, go to [](/get-started/index.md) or [](/manage-data/data-store.md). |
There was a problem hiding this comment.
| If you're looking for an introduction to the {{stack}} or the {{es}} product, go to [](/get-started/index.md) or [](/manage-data/data-store.md). | |
| If you're looking for an introduction to the {{stack}} or the {{es}} product, refer to [](/get-started/index.md) or [](/manage-data/data-store.md). |
| | Generate embeddings from your content | You want {{es}} to create embeddings for you | Ingest content into a `semantic_text` field, then run a semantic or hybrid query | | ||
| | Store your existing embeddings | You already have vectors from your own model | Index pre-generated embeddings into a `dense_vector` field, then run a semantic or hybrid query | | ||
|
|
||
| You can also skip the setup guide and continue with the steps below. |
There was a problem hiding this comment.
Just above this, we can point to the new quickstart, something like:
Alternatively, you can also use the Elasticsearch vector and full-text search in 10 minutes quickstart to connect a client, index sample data, and run semantic, hybrid, and ES|QL searches.
| You can generate embeddings as part of the ingestion workflow instead of creating them in advance. | ||
|
|
||
| For text content, the recommended approach is to map the target field as [`semantic_text`](elasticsearch://reference/elasticsearch/mapping-reference/semantic-text.md). When you ingest documents, {{es}} uses the configured {{infer}} endpoint to generate and store embeddings automatically. You can check which models are used by default and learn how to change them in [Configure {{infer}} endpoints documentation](elasticsearch://reference/elasticsearch/mapping-reference/semantic-text-setup-configuration.md#configure-inference-endpoints). | ||
|
|
||
| For more control over the {{infer}} and ingestion workflow, you can also generate embeddings with an {{infer}} processor in an ingest pipeline and store them in a vector field. For more information, see [](/solutions/search/semantic-search/semantic-search-elser-ingest-pipelines.md). | ||
|
|
||
| To walk through mapping a `semantic_text` field, ingesting sample content, and running a hybrid query, follow the [semantic search quickstart](/solutions/search/get-started/semantic-search.md). | ||
| :::: | ||
|
|
||
| ::::{dropdown} Store your existing embeddings | ||
| Create an index with a [`dense_vector`](elasticsearch://reference/elasticsearch/mapping-reference/dense-vector.md) field, and any other text or metadata fields you might need, then index your pre-generated vectors. | ||
|
|
||
| To walk through indexing sample embeddings and running a kNN search, follow [Bring your own dense vectors to {{es}}](/solutions/search/vector/bring-own-vectors.md). |
There was a problem hiding this comment.
I'm thinking about which embedding generation methods we should highlight here. Technically, there are several options: using the semantic_text field type, the semantic field type for multimodal, and inference processors and ingest pipelines with dense and sparse vectors. But looking at this page now, I feel like for a Get started guide we should present only the most recommended approaches, which are, AFAIK the semantic_text and semantic field types.
I'd suggest removing all other content from this section and reorganizing "Generating embeddings for your content" into two categories:
- Text-only content: semantic_text
- Multimodal content: semantic
Each linking to the related tutorial: for semantic_text, to the new quickstart: #7923, and for multimodal, to the multimodal quickstart: https://www.elastic.co/docs/solutions/search/multimodal-search/multimodal-search-tutorial
That said, this is more of a product question, so @dustincoates or @alexmarhaba could probably weigh in as well.
There was a problem hiding this comment.
I would do only semantic_text, since semantic is still in tech preview.
| To walk through indexing sample embeddings and running a kNN search, follow [Bring your own dense vectors to {{es}}](/solutions/search/vector/bring-own-vectors.md). | ||
| :::: | ||
|
|
||
| Use a language [client](/reference/elasticsearch-clients/index.md) or the [Bulk]({{es-serverless-apis}}operation/operation-bulk) API to load data. If you're not ready to use your own content, follow the [semantic search quickstart](/solutions/search/get-started/semantic-search.md). |
There was a problem hiding this comment.
I think we could omit this whole note.
Use a language client or the Bulk API to load data.
IMO this information isn't necessary at this level, since the guides we direct users to above already cover these methods in tutorials. (Note to @elastic/developer-docs : it would be nice to have a section somewhere in the docs that summarizes this kind of information, so we can link to it from places like this.)
I think we can omit this part too:
If you're not ready to use your own content, follow the semantic search quickstart.
since we already link to a better tutorial above that serves the same purpose. (IA changes around this area are coming soon 😄 )
| :::::{step} Search your data | ||
| Match your query to the vector field type in your mapping. For details on which queries each field type supports, see [Field types and queries](/solutions/search/vector.md#vector-queries-and-field-types). | ||
|
|
||
| To walk through common patterns, refer to the following pages: | ||
|
|
||
| * [kNN search in {{es}}](/solutions/search/vector/knn.md) | ||
| * [Hybrid search](/solutions/search/hybrid-search.md) | ||
|
|
||
| For more query options, see [Querying for search](/solutions/search/querying-for-search.md) and the [{{es-serverless}} API documentation]({{es-serverless-apis}}). | ||
| ::::: | ||
| :::::: |
There was a problem hiding this comment.
I'd reframe this slightly to something like:
The search query type you can use depends on the vector field type you want to search. For an overview of field types and the queries you can use with each, refer to Field types and queries.
I think this is all the information needed here since the linked page is focused on vector search queries, which is exactly the use case we want to cover.
So, I'd remove the following:
To walk through common patterns, refer to the following pages:
For more query options, see Querying for search and the {{es-serverless}} API documentation.
Because we already provided tutorials above, and from the Field types and queries section users can navigate to whichever query type they're interested in.
|
|
||
| Search charges cover the search capacity reserved for your stored data. All stored data is searchable by default, which means it actively contributes to Search charges. | ||
|
|
||
| [Search Power](/deploy-manage/deploy/elastic-cloud/project-settings.md#elasticsearch-manage-project-search-power-settings) determines how much capacity is reserved. At Search Power **100**, enough capacity is reserved so that 100% of your project data remains available for low-latency search. Higher Search Power values reserve more capacity in proportion to that baseline: for example, **200** reserves about twice as much as **100**. |
There was a problem hiding this comment.
We've added a floor where anything less than 16 GB is billed at 16 GB. It's a little more complex than that, but just barely and not in a way that is negative to customers. Do we want to mention that here? (We also plan on reducing that floor over time.)
|
|
||
| ## Managing Vector Database costs [vector-database-billing-managing-costs] | ||
|
|
||
| Vector Database costs follow your storage footprint, search resource allocation, ingest volume, and the hours your project is available. To balance performance with spend, adjust the controls described in this section. |
There was a problem hiding this comment.
While technically true about the hours that your project is available, we don't have a way to make a project unavailable short of deleting it. The hourly billing is really just a reflection on how we meter: to a customer it should look a lot like a monthly fee that is pro-rated if they start/end in the middle of the month.
| | Setting | Description | Project Type | | ||
| | :--- | :--- | :--- | | ||
| | **Search Power** | Search Power controls the amount of baseline resources kept ready for your data and the maximum amount of resources the system can allocate to your project under high search load.<br><br>Increase **Search Power** to add resources for improved query throughput and performance as load increases. Decrease it to reduce the amount of provisioned resources, which lowers both active search costs and the reduced rate billed during idle periods, at the cost of more variable query latency. The On-demand setting provides the lowest baseline and idle cost.<br><br>Choose from three Search Power settings:<br><br>**On-demand:** Low cost option. Query latency is variable and maximum throughput is limited, particularly during demand spikes.<br><br>**Performant:** Delivers low _average_ query latency with no practical query throughput limits to meet varying demand.<br><br>**High availability:** Maintains consistent low query latency across most search workloads, from steady baseline to peak demand.<br> | {{es}} | | ||
| | **Search Power** | Search Power reserves search capacity for your project. Unlike {{es}} projects, capacity is not scaled automatically based on query load.<br><br>The default value is **100**, which reserves enough search resources so that 100% of your project data remains available for low-latency search. You can increase Search Power in increments of 100. Higher values reserve proportionally more capacity: for example, **200** reserves about twice the capacity of **100**. You can increase Search Power up to **1900**.<br> | {{es}} Vector Database | |
There was a problem hiding this comment.
Does this have Search Power twice?
There was a problem hiding this comment.
Oh, I see, it will only be for VDB.
| |-----------|-------------------|----------| | ||
| | Build powerful, scalable searches to quickly search, analyze, and visualize large amounts of data for real-time insights| [{{es}}](/solutions/elasticsearch-solution-project.md)<br>• [Get started](/solutions/elasticsearch-solution-project/get-started.md)| Developers, architects, data engineers | | ||
| | Build powerful, scalable search and analytics applications across structured data, logs, metrics, documents, and vectors as part of a broader {{stack}} | [{{es}}](/solutions/elasticsearch-solution-project.md)<br>• [Get started](/solutions/elasticsearch-solution-project/get-started.md)| Developers, architects, data engineers | | ||
| | Build embedding-driven workloads such as semantic search, RAG, and AI-powered retrieval | [{{es}} Vector Database](/solutions/vector-database.md) {applies_to}`serverless:`<br>• [Get started](/solutions/vector-database/get-started.md) {applies_to}`serverless:` | Developers building AI retrieval and similarity search applications | |
There was a problem hiding this comment.
For Best For, I would leave that as the same as ES--I think Your Need communicates better where VDB is targeting.
| You can generate embeddings as part of the ingestion workflow instead of creating them in advance. | ||
|
|
||
| For text content, the recommended approach is to map the target field as [`semantic_text`](elasticsearch://reference/elasticsearch/mapping-reference/semantic-text.md). When you ingest documents, {{es}} uses the configured {{infer}} endpoint to generate and store embeddings automatically. You can check which models are used by default and learn how to change them in [Configure {{infer}} endpoints documentation](elasticsearch://reference/elasticsearch/mapping-reference/semantic-text-setup-configuration.md#configure-inference-endpoints). | ||
|
|
||
| For more control over the {{infer}} and ingestion workflow, you can also generate embeddings with an {{infer}} processor in an ingest pipeline and store them in a vector field. For more information, see [](/solutions/search/semantic-search/semantic-search-elser-ingest-pipelines.md). | ||
|
|
||
| To walk through mapping a `semantic_text` field, ingesting sample content, and running a hybrid query, follow the [semantic search quickstart](/solutions/search/get-started/semantic-search.md). | ||
| :::: | ||
|
|
||
| ::::{dropdown} Store your existing embeddings | ||
| Create an index with a [`dense_vector`](elasticsearch://reference/elasticsearch/mapping-reference/dense-vector.md) field, and any other text or metadata fields you might need, then index your pre-generated vectors. | ||
|
|
||
| To walk through indexing sample embeddings and running a kNN search, follow [Bring your own dense vectors to {{es}}](/solutions/search/vector/bring-own-vectors.md). |
There was a problem hiding this comment.
I would do only semantic_text, since semantic is still in tech preview.
3 participants
Summary
Closes #1413
What's new
What's changed
Related to #7923
Generative AI disclosure
Tool(s) and model(s) used: Used Cursor's Auto agent to surface pages that needed updates.