For completeness, we should support indexers, data sources, and skillsets.
It would be nice if these could be wired in via Aspire to pull data from other resources automatically.
i.e.: (note: this example is likely incomplete or incorrect; it's just a sketch)
var ai = // some AI resource for OCR
var search = builder.AddAzureSearchEmulator("search")
.WithIndexesVolume();
var blobDataSource = search.WithBlobDataSource("blobs", myBlobContainerResource);
var blobSkillset = search.WithSkillset("pdfimport", ai)
.WithOcrSkill("ocr", config =>
config.WithInput("image", "/document/normalized_images/*")
.WithOutput("text", "ocrText"))
.WithMergeSkill("merge", config =>
config.WithInput("text", "/document/content")
.WithInput("itemsToInsert", "/document/normalized_images/*/ocrText")
.WithInput("offsets", "/document/normalized_images/*/contentOffset")
.WithOutput("mergedText"))
.WithLanguageDetectionSkill("detect-language", config =>
config.WithInput("text", "/document/mergedText")
.WithOutput("languageCode"))
.WithTranslationSkill("translate", "en", config =>
config.WithInput("text", "/document/mergedText")
.WithInput("fromLanguageCode", "/document/languageCode")
.WithOutput("translatedText"));
var index = search.WithIndex("files")
.WithFieldsFromType<DocumentFile>()
.WithIndexer("blobindexer", blobDataSource, blobSkillset, config =>
config.WithOutputFieldMapping("/document/mergedText", "originalText")
.WithOutputFieldMapping("/document/translatedText", "englishText")
.WithOutputFieldMapping("/document/languageCode", "detectedLanguage"));Note that we can do some of the built-in Azure skills ourselves without Foundry/AI integration via my NOpenNLP port of OpenNLP:
- Entity Recognition
- Language Detection
- Sentiment
At a minimum this could avoid the need for AI for local use if you only need these skills. But that's probably rare; most users would need to plug into some kind of AI model (and it would be nice if we could support local ollama as well).
Note that Indexers/Data sources don't implicitly require a skillset, so you should be able to use those to do text ingestion without AI.
For completeness, we should support indexers, data sources, and skillsets.
It would be nice if these could be wired in via Aspire to pull data from other resources automatically.
i.e.: (note: this example is likely incomplete or incorrect; it's just a sketch)
Note that we can do some of the built-in Azure skills ourselves without Foundry/AI integration via my NOpenNLP port of OpenNLP:
At a minimum this could avoid the need for AI for local use if you only need these skills. But that's probably rare; most users would need to plug into some kind of AI model (and it would be nice if we could support local ollama as well).
Note that Indexers/Data sources don't implicitly require a skillset, so you should be able to use those to do text ingestion without AI.