diff --git a/docs/client-api/smuggler/content/_what-is-smuggler-csharp.mdx b/docs/client-api/smuggler/content/_what-is-smuggler-csharp.mdx index 4bdf46a8fc..464d78c647 100644 --- a/docs/client-api/smuggler/content/_what-is-smuggler-csharp.mdx +++ b/docs/client-api/smuggler/content/_what-is-smuggler-csharp.mdx @@ -1,179 +1,397 @@ import Admonition from '@theme/Admonition'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -import CodeBlock from '@theme/CodeBlock'; +import Panel from "@site/src/components/Panel"; +import ContentFrame from "@site/src/components/ContentFrame"; -Smuggler gives you the ability to export or import data from or to a database using JSON format. -It is exposed via the `DocumentStore.Smuggler` property. + -## ForDatabase +* **Smuggler** is a tool for moving data between databases. + Use it to export documents, indexes, and other items from a database as JSON, + and to import that data into another database. -By default, the `DocumentStore.Smuggler` works on the default document store database from the `DocumentStore.Database` property. +* Smuggler is exposed through the `DocumentStore.Smuggler` property. -In order to switch it to a different database use the `.ForDatabase` method. +* An export can write to a `.ravendbdump` file or stream directly into another database. + An import reads that data back into a database. - - -{`var northwindSmuggler = store - .Smuggler - .ForDatabase("Northwind"); -`} - - +* You can narrow what is transferred by collection, by item type, + and by applying a transform script to each document. +* In this article: + * [ForDatabase](../../../client-api/smuggler/what-is-smuggler.mdx#fordatabase) + * [Export](../../../client-api/smuggler/what-is-smuggler.mdx#export) + * [Export specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#export-specific-collections) + * [Import](../../../client-api/smuggler/what-is-smuggler.mdx#import) + * [Import specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#import-specific-collections) + * [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript) + * [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) + * [Methods](../../../client-api/smuggler/what-is-smuggler.mdx#methods) + * [Classes](../../../client-api/smuggler/what-is-smuggler.mdx#classes) + -## Export + -### Syntax +By default, `DocumentStore.Smuggler` works on the database set in the `DocumentStore.Database` property. +To run Smuggler against a different database, switch to it with the `ForDatabase` method. - - -{`Task ExportAsync( - DatabaseSmugglerExportOptions options, - DatabaseSmuggler toDatabase, - CancellationToken token = default(CancellationToken)); - -Task ExportAsync( - DatabaseSmugglerExportOptions options, - string toFile, - CancellationToken token = default(CancellationToken)); -`} - - +```csharp +var northwindSmuggler = store + .Smuggler + .ForDatabase("Northwind"); +``` -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerExportOptions` | Options that will be used during the export. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | -| **toDatabase** | `DatabaseSmuggler` | `DatabaseSmuggler` instance used as a destination | -| **toFile** | `string` | Path to a file where exported data will be written | -| **token** | `CancellationToken` | Token used to cancel the operation | + -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | + -### DatabaseSmugglerExportOptions +Export writes the selected data to a `.ravendbdump` file, or streams it directly into another database. +Use `DatabaseSmugglerExportOptions` to choose what to include. -| Parameters | | | -| ------------- | ------------- | ----- | -| **Collections** | `List` | List of specific collections to export. If empty, then all collections will be exported.
Default: `empty` | -| **OperateOnTypes** | `DatabaseItemType` | Indicates what should be exported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `ReplicationHubCertificates`, `Identities`, `CompareExchange`, `Attachments`, `CounterGroups`, `Subscriptions`, `TimeSeries` | -| **OperateOnDatabaseRecordTypes** | `DatabaseRecordItemType` | Indicates what should be exported from database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `Settings`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications`, `TimeSeries`, `DocumentsCompression`, `Analyzers`, `LockMode`, `OlapConnectionStrings`, `OlapEtls`, `ElasticSearchConnectionStrings`, `ElasticSearchEtls`, `PostgreSQLIntegration`, `QueueConnectionStrings`, `QueueEtls`, `IndexesHistory`, `Refresh`, `DataArchival` | -| **IncludeExpired** | `bool` | Should expired documents be exported.
Default: `true` | -| **IncludeArtificial** | `bool` | Should artificial documents be exported.
Default: `false` | -| **IncludeArchived** | `bool` | Should archived documents be exported.
Default: `true` | -| **RemoveAnalyzers** | `bool` | Should analyzers be removed from Indexes.
Default: `false` | -| **TransformScript** | `string` | JavaScript-based script applied to every exported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **MaxStepsForTransformScript** | `int` | Maximum number of steps that the transform script can process before failing.
Default: **10000** | + -### Example +#### Example: export indexes and documents to a file - - -{`// export only Indexes and Documents to a given file +```csharp +// export only Indexes and Documents to a given file var exportOperation = await store .Smuggler .ExportAsync( new DatabaseSmugglerExportOptions - \{ + { OperateOnTypes = DatabaseItemType.Indexes | DatabaseItemType.Documents - \}, - @"C:\\ravendb-exports\\Northwind.ravendbdump", + }, + @"C:\ravendb-exports\Northwind.ravendbdump", token); await exportOperation.WaitForCompletionAsync(); -`} - - +``` +
+ -## Import +### Export specific collections -### Syntax +By default, all collections are exported. +To export only specific collections, list their names in the `Collections` option. - - -{`Task ImportAsync( - DatabaseSmugglerImportOptions options, - Stream stream, - CancellationToken token = default(CancellationToken)); +```csharp +// Export only the "Orders" and "Categories" collections to a file +var exportOperation = await store + .Smuggler + .ExportAsync( + new DatabaseSmugglerExportOptions + { + Collections = new List { "Orders", "Categories" } + }, + @"C:\ravendb-exports\Northwind.ravendbdump", + token); -Task ImportAsync( - DatabaseSmugglerImportOptions options, - string fromFile, - CancellationToken token = default(CancellationToken)); -`} - - +await exportOperation.WaitForCompletionAsync(); +``` -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerImportOptions` | Options that will be used during the import. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | -| **stream** | `Stream` | Stream with data to import | -| **fromFile** | `string` | Path to a file from which data will be imported | -| **token** | `CancellationToken` | Token used to cancel the operation | + -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation-class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | +
-### DatabaseSmugglerImportOptions +See [ExportAsync](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. -| Parameters | | | -| - | - | - | -| **Collections** | `List` | List of specific collections to import. If empty, then all collections will be imported.
Default: `empty` | -| **OperateOnTypes** | `DatabaseItemType` | Indicates what should be imported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `ReplicationHubCertificates`, `Identities`, `CompareExchange`, `Attachments`, `CounterGroups`, `Subscriptions`, `TimeSeries` | -| **OperateOnDatabaseRecordTypes** | `DatabaseRecordItemType` | Indicates what should be imported from database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `Settings`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications`, `TimeSeries`, `DocumentsCompression`, `Analyzers`, `LockMode`, `OlapConnectionStrings`, `OlapEtls`, `ElasticSearchConnectionStrings`, `ElasticSearchEtls`, `PostgreSQLIntegration`, `QueueConnectionStrings`, `QueueEtls`, `IndexesHistory`, `Refresh`, `DataArchival` | -| **IncludeExpired** | `bool` | Should expired documents be imported.
Default: `true` | -| **IncludeArtificial** | `bool` | Should artificial documents be imported.
Default: `false` | -| **IncludeArchived** | `bool` | Should archived documents be imported.
Default: `true` | -| **RemoveAnalyzers** | `bool` | Should analyzers be removed from Indexes.
Default: `false` | -| **TransformScript** | `string` | JavaScript-based script applied to every imported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **MaxStepsForTransformScript** | `int` | Maximum number of steps that the transform script can process before failing.
Default: **10000** | +
+ + -### Example +Import reads data from a `.ravendbdump` file or a stream and writes it into the database. +Use `DatabaseSmugglerImportOptions` to choose what to include. - - -{`// import only Documents from a given file + + +#### Example: import documents from a file + +```csharp +// import only Documents from a given file var importOperation = await store .Smuggler .ImportAsync( new DatabaseSmugglerImportOptions - \{ + { OperateOnTypes = DatabaseItemType.Documents - \}, + }, // import the .ravendbdump file that you exported (i.e. in the export example above) - @"C:\\ravendb-exports\\Northwind.ravendbdump", + @"C:\ravendb-exports\Northwind.ravendbdump", token); await importOperation.WaitForCompletionAsync(); -`} - - +``` +
+ -## TransformScript +### Import specific collections -`TransformScript` exposes the ability to modify or even filter-out the document during the import and export process using the provided JavaScript. +By default, all collections are imported. +To import only specific collections, list their names in the `Collections` option. -Underneath the JavaScript engine is exactly the same as used for [patching operations](../../../client-api/operations/patching/single-document.mdx) giving you identical syntax and capabilities with additional **ability to filter out documents by throwing a 'skip' exception**. +```csharp +// Import only the "Orders" and "Categories" collections from the file +var importOperation = await store + .Smuggler + .ImportAsync( + new DatabaseSmugglerImportOptions + { + Collections = new List { "Orders", "Categories" } + }, + @"C:\ravendb-exports\Northwind.ravendbdump", + token); + +await importOperation.WaitForCompletionAsync(); +``` - - -{`var id = this['@metadata']['@id']; + + +
+ +See [ImportAsync](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. + + + + + +`TransformScript` lets you modify or filter out each document during an export or import, using JavaScript that you provide. + +The JavaScript engine is the same one used for [patching operations](../../../client-api/operations/patching/single-document.mdx), so you have identical syntax and capabilities, plus the ability to filter out a document by throwing a `skip` exception. + +```javascript +var id = this['@metadata']['@id']; if (id === 'orders/999-A') throw 'skip'; // filter-out this.Freight = 15.3; -`} - +``` + + + + + +## Methods + + + +### Smuggler methods + + + + +Switches Smuggler to operate on a database other than the document store's default database. + +```csharp +public DatabaseSmuggler ForDatabase(string databaseName); +``` + +
+ +Usage: + +```csharp +var northwindSmuggler = store.Smuggler.ForDatabase("Northwind"); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **databaseName** | `string` | The database that Smuggler will operate on. | + +| Return value | | +|-----------|-------------| +| `DatabaseSmuggler` | A Smuggler instance scoped to the specified database. | + +
+ + +Exports the selected data to a file, to a stream, or to another database. + +```csharp +Task ExportAsync( + DatabaseSmugglerExportOptions options, + string toFile, + CancellationToken token = default); + +Task ExportAsync( + DatabaseSmugglerExportOptions options, + Stream toStream, + CancellationToken token = default); + +Task ExportAsync( + DatabaseSmugglerExportOptions options, + DatabaseSmuggler toDatabase, + CancellationToken token = default); +``` + +
+ +Usage: + +```csharp +var exportOperation = await store.Smuggler.ExportAsync(options, toFile, token); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerExportOptions` | Options used during the export. See [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | +| **toFile** | `string` | Path to the file where exported data is written. | +| **toStream** | `Stream` | Stream that exported data is written to. | +| **toDatabase** | `DatabaseSmuggler` | A `DatabaseSmuggler` instance used as the destination. | +| **token** | `CancellationToken` | Token used to cancel the operation. | + +| Return value | | +|-----------|-------------| +| `Operation` | An `Operation` instance you can use to wait for completion and subscribe to progress events. | +
+ + +Imports data from a file or a stream into the database. + +```csharp +Task ImportAsync( + DatabaseSmugglerImportOptions options, + string fromFile, + CancellationToken token = default); + +Task ImportAsync( + DatabaseSmugglerImportOptions options, + Stream stream, + CancellationToken token = default); +``` + +
+ +Usage: + +```csharp +var importOperation = await store.Smuggler.ImportAsync(options, fromFile, token); +``` +
+| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerImportOptions` | Options used during the import. See [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | +| **fromFile** | `string` | Path to the file the data is imported from. | +| **stream** | `Stream` | Stream with the data to import. | +| **token** | `CancellationToken` | Token used to cancel the operation. | + +| Return value | | +|-----------|-------------| +| `Operation` | An `Operation` instance you can use to wait for completion and subscribe to progress events. | + +
+
+ +
+ +## Classes + + + +### Smuggler options classes + +
+ + + + + +**Options that control what an export includes.** + +```csharp +class DatabaseSmugglerExportOptions +{ + DatabaseItemType OperateOnTypes + DatabaseRecordItemType OperateOnDatabaseRecordTypes + bool IncludeExpired + bool IncludeArtificial + bool IncludeArchived + bool RemoveAnalyzers + string TransformScript + int MaxStepsForTransformScript + string EncryptionKey + List Collections + int? MaxReadOpsPerSecond + bool SkipCorruptedData + ExportCompressionAlgorithm? CompressionAlgorithm +} +``` + +
+ +| Property | Type | Description | +|-----------|------|-------------| +| **Collections** | `List` | List of specific collections to export. If empty, all collections are exported.
Default: `empty` | +| **OperateOnTypes** | `DatabaseItemType` | Indicates what should be exported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `ReplicationHubCertificates`, `Identities`, `CompareExchange`, `Attachments`, `CounterGroups`, `Subscriptions`, `TimeSeries`, `TimeSeriesDeletedRanges` | +| **OperateOnDatabaseRecordTypes** | `DatabaseRecordItemType` | Indicates what should be exported from the database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `Settings`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications`, `TimeSeries`, `DocumentsCompression`, `Analyzers`, `LockMode`, `OlapConnectionStrings`, `OlapEtls`, `ElasticSearchConnectionStrings`, `ElasticSearchEtls`, `PostgreSQLIntegration`, `QueueConnectionStrings`, `QueueEtls`, `IndexesHistory`, `Refresh`, `DataArchival`, `QueueSinks`, `SnowflakeEtls`, `SnowflakeConnectionStrings`, `EmbeddingsGenerations`, `AiConnectionStrings`, `GenAiEtls`, `AiAgents`, `RemoteAttachments`, `SchemaValidation` | +| **IncludeExpired** | `bool` | Should expired documents be exported.
Default: `true` | +| **IncludeArtificial** | `bool` | Should artificial documents be exported.
Default: `false` | +| **IncludeArchived** | `bool` | Should archived documents be exported.
Default: `true` | +| **RemoveAnalyzers** | `bool` | Should analyzers be removed from indexes.
Default: `false` | +| **TransformScript** | `string` | JavaScript applied to every exported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **MaxStepsForTransformScript** | `int` | Maximum number of steps the transform script can run before failing.
Default: `10000` | +| **EncryptionKey** | `string` | Encryption key used to encrypt the exported file.
Default: `null` | +| **MaxReadOpsPerSecond** | `int?` | Limits the read-operation rate during the export, to throttle the load placed on the database.
Default: `null` (no limit) | +| **SkipCorruptedData** | `bool` | When `true`, the export continues if corrupted data is encountered (for example, lost compression dictionaries), recording an error entry in the result report instead of stopping.
Default: `false` | +| **CompressionAlgorithm** | `ExportCompressionAlgorithm?` | Compression algorithm used for the exported file: `Zstd` or `Gzip`.
Default: `null` | + +
+ + +**Options that control what an import includes.** + +```csharp +class DatabaseSmugglerImportOptions +{ + DatabaseItemType OperateOnTypes + DatabaseRecordItemType OperateOnDatabaseRecordTypes + bool IncludeExpired + bool IncludeArtificial + bool IncludeArchived + bool RemoveAnalyzers + string TransformScript + int MaxStepsForTransformScript + string EncryptionKey + List Collections + int? MaxReadOpsPerSecond + bool SkipCorruptedData + bool SkipRevisionCreation +} +``` + +
+ +| Property | Type | Description | +|-----------|------|-------------| +| **Collections** | `List` | List of specific collections to import. If empty, all collections are imported.
Default: `empty` | +| **OperateOnTypes** | `DatabaseItemType` | Indicates what should be imported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `ReplicationHubCertificates`, `Identities`, `CompareExchange`, `Attachments`, `CounterGroups`, `Subscriptions`, `TimeSeries`, `TimeSeriesDeletedRanges` | +| **OperateOnDatabaseRecordTypes** | `DatabaseRecordItemType` | Indicates what should be imported into the database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `Settings`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications`, `TimeSeries`, `DocumentsCompression`, `Analyzers`, `LockMode`, `OlapConnectionStrings`, `OlapEtls`, `ElasticSearchConnectionStrings`, `ElasticSearchEtls`, `PostgreSQLIntegration`, `QueueConnectionStrings`, `QueueEtls`, `IndexesHistory`, `Refresh`, `DataArchival`, `QueueSinks`, `SnowflakeEtls`, `SnowflakeConnectionStrings`, `EmbeddingsGenerations`, `AiConnectionStrings`, `GenAiEtls`, `AiAgents`, `RemoteAttachments`, `SchemaValidation` | +| **IncludeExpired** | `bool` | Should expired documents be imported.
Default: `true` | +| **IncludeArtificial** | `bool` | Should artificial documents be imported.
Default: `false` | +| **IncludeArchived** | `bool` | Should archived documents be imported.
Default: `true` | +| **RemoveAnalyzers** | `bool` | Should analyzers be removed from indexes.
Default: `false` | +| **TransformScript** | `string` | JavaScript applied to every imported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **MaxStepsForTransformScript** | `int` | Maximum number of steps the transform script can run before failing.
Default: `10000` | +| **EncryptionKey** | `string` | Encryption key used to read an encrypted `.ravendbdump` file.
Default: `null` | +| **MaxReadOpsPerSecond** | `int?` | Limits the read-operation rate during the import, to throttle the load placed on the database.
Default: `null` (no limit) | +| **SkipCorruptedData** | `bool` | When `true`, the import continues if corrupted data is encountered, recording an error entry in the result report instead of stopping.
Default: `false` | +| **SkipRevisionCreation** | `bool` | When `true`, revisions are not created for the imported documents.
Default: `false` | + +
+
+ + diff --git a/docs/client-api/smuggler/content/_what-is-smuggler-java.mdx b/docs/client-api/smuggler/content/_what-is-smuggler-java.mdx index dd4f20fd0c..4d89fe491a 100644 --- a/docs/client-api/smuggler/content/_what-is-smuggler-java.mdx +++ b/docs/client-api/smuggler/content/_what-is-smuggler-java.mdx @@ -1,137 +1,364 @@ import Admonition from '@theme/Admonition'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -import CodeBlock from '@theme/CodeBlock'; +import Panel from "@site/src/components/Panel"; +import ContentFrame from "@site/src/components/ContentFrame"; -Smuggler gives you the ability to export or import data from or to a database using JSON format. -It is exposed via the `DocumentStore.smuggler()`. + -## ForDatabase +* **Smuggler** is a tool for moving data between databases. + Use it to export documents, indexes, and other items from a database as JSON, + and to import that data into another database. -By default, the `IDocumentStore.smuggler` works on the default document store database from the `IDocumentStore.database` property. +* Smuggler is exposed through the `DocumentStore.smuggler()` method. -In order to switch it to a different database use the `.forDatabase` method. +* An export can write to a `.ravendbdump` file or stream directly into another database. + An import reads that data back into a database. - - -{`DatabaseSmuggler northwindSmuggler = store.smuggler().forDatabase("Northwind"); -`} - - +* You can narrow what is transferred by collection, by item type, + and by applying a transform script to each document. +* In this article: + * [ForDatabase](../../../client-api/smuggler/what-is-smuggler.mdx#fordatabase) + * [Export](../../../client-api/smuggler/what-is-smuggler.mdx#export) + * [Export specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#export-specific-collections) + * [Import](../../../client-api/smuggler/what-is-smuggler.mdx#import) + * [Import specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#import-specific-collections) + * [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript) + * [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) + * [Methods](../../../client-api/smuggler/what-is-smuggler.mdx#methods) + * [Classes](../../../client-api/smuggler/what-is-smuggler.mdx#classes) + -## Export + -### Syntax +By default, the smuggler works on the database set in the `DocumentStore.getDatabase()` property. +To run the smuggler against a different database, switch to it with the `forDatabase` method. - - -{`//export -public Operation exportAsync(DatabaseSmugglerExportOptions options, String toFile); +```java +DatabaseSmuggler northwindSmuggler = store + .smuggler() + .forDatabase("Northwind"); +``` -public Operation exportAsync(DatabaseSmugglerExportOptions options, DatabaseSmuggler toDatabase); -`} - - + -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerExportOptions` | Options that will be used during the export. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | -| **toDatabase** | `DatabaseSmuggler` | `DatabaseSmuggler` instance used as a destination | -| **toFile** | `String` | Path to a file where exported data will be written | - -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | - -### DatabaseSmugglerExportOptions - -| Parameters | | | -| ------------- | ------------- | ----- | -| **Collections** | `List` | List of specific collections to export. If empty, then all collections will be exported. Default: `empty` | -| **operateOnTypes** | `DatabaseItemType` | Indicates what should be exported. Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `Identities`, `CompareExchange`, `Subscriptions` | -| **operateOnDatabaseRecordType** | `DatabaseRecordItemType` | Indicates what should be exported from database record. Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications` | -| **includeExpired** | `boolean` | Should expired documents be included in the export. Default: `true` | -| **removeAnalyzers** | `boolean` | Should analyzers be removed from Indexes. Default: `false` | -| **transformScript** | `String` | JavaScript-based script applied to every exported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **maxStepsForTransformScript** | `int` | Maximum number of steps that transform script can process before failing. Default: 10000 | - -### Example - - - -{`// export only Indexes and Documents to a given file -Operation exportOperation = store.smuggler().exportAsync(exportOptions, "C:\\\\ravendb-exports\\\\Northwind.ravendbdump"); -`} - - + +Export writes the selected data to a `.ravendbdump` file, or streams it directly into another database. +Use `DatabaseSmugglerExportOptions` to choose what to include. +
-## Import +#### Example: export indexes and documents to a file -### Syntax +```java +// export only Indexes and Documents to a given file +DatabaseSmugglerExportOptions options = new DatabaseSmugglerExportOptions(); +options.setOperateOnTypes(EnumSet.of( + DatabaseItemType.INDEXES, + DatabaseItemType.DOCUMENTS)); - - -{`public Operation importAsync(DatabaseSmugglerImportOptions options, String fromFile); -public Operation importAsync(DatabaseSmugglerImportOptions options, InputStream stream); -`} - - +Operation exportOperation = store + .smuggler() + .exportAsync(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerImportOptions` | Options that will be used during the import. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | -| **stream** | `InputStream` | Stream with data to import | -| **fromFile** | `String` | Path to a file from which data will be imported | +exportOperation.waitForCompletion(); +``` -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation-class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | +
-### DatabaseSmugglerImportOptions + -| Parameters | | | -| - | - | - | -| **Collections** | `List` | List of specific collections to import. If empty, then all collections will be exported. Default: `empty` | -| **operateOnTypes** | `DatabaseItemType` | Indicates what should be imported. Default: `INDEXES`, `DOCUMENTS`, `REVISION_DOCUMENTS`, `CONFLICTS`, `DATABASE_RECORD`, `IDENTITIES`, `COMPARE_EXCHANGE`, `SUBSCRIPTIONS` | -| **operateOnDatabaseRecordType** | `DatabaseRecordItemType` | Indicates what should be imported. Default: `CLIENT`, `CONFLICT_SOLVER_CONFIG`, `EXPIRATION`, `EXTERNAL_REPLICATIONS`, `PERIODIC_BACKUPS`, `RAVEN_CONNECTION_STRINGS`, `RAVEN_ETLS`, `REVISIONS`, `SQL_CONNECTION_STRINGS`, `SORTERS`, `SQL_ETLS`, `HUB_PULL_REPLICATIONS`, `SINK_PULL_REPLICATIONS` | -| **includeExpired** | `boolean` | Should expired documents be included in the import. Default: `true` | -| **removeAnalyzers** | `boolean` | Should analyzers be removed from Indexes. Default: `false` | -| **transformScript** | `String` | JavaScript-based script applied to every exported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **maxStepsForTransformScript** | `int` | Maximum number of steps that transform script can process before failing. Default: 10000 | +### Export specific collections +By default, all collections are exported. +To export only specific collections, list their names in the `collections` option. -### Example +```java +// Export only the "Orders" and "Categories" collections to a file +DatabaseSmugglerExportOptions options = new DatabaseSmugglerExportOptions(); +options.setCollections(Arrays.asList("Orders", "Categories")); - - -{`Operation importOperation = store.smuggler().importAsync(importOptions, "C:\\\\ravendb-exports\\\\Northwind.ravendbdump"); -`} - - +Operation exportOperation = store + .smuggler() + .exportAsync(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + +exportOperation.waitForCompletion(); +``` + + + +
+ +See [exportAsync](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. + + + + + +Import reads data from a `.ravendbdump` file or a stream and writes it into the database. +Use `DatabaseSmugglerImportOptions` to choose what to include. + +
+ +#### Example: import documents from a file + +```java +// import only Documents from a given file +DatabaseSmugglerImportOptions options = new DatabaseSmugglerImportOptions(); +options.setOperateOnTypes(EnumSet.of(DatabaseItemType.DOCUMENTS)); +// import the .ravendbdump file that you exported (i.e. in the export example above) +Operation importOperation = store + .smuggler() + .importAsync(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); +importOperation.waitForCompletion(); +``` -## TransformScript +
-`TransformScript` exposes the ability to modify or even filter-out the document during the import and export process using the provided JavaScript. + -Underneath the JavaScript engine is exactly the same as used for [patching operations](../../../client-api/operations/patching/single-document.mdx) giving you identical syntax and capabilities with additional **ability to filter out documents by throwing a 'skip' exception**. +### Import specific collections - - -{`var id = this['@metadata']['@id']; +By default, all collections are imported. +To import only specific collections, list their names in the `collections` option. + +```java +// Import only the "Orders" and "Categories" collections from the file +DatabaseSmugglerImportOptions options = new DatabaseSmugglerImportOptions(); +options.setCollections(Arrays.asList("Orders", "Categories")); + +Operation importOperation = store + .smuggler() + .importAsync(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + +importOperation.waitForCompletion(); +``` + + + +
+ +See [importAsync](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. + + + + + +`TransformScript` lets you modify or filter out each document during an export or import, using JavaScript that you provide. + +The JavaScript engine is the same one used for [patching operations](../../../client-api/operations/patching/single-document.mdx), so you have identical syntax and capabilities, plus the ability to filter out a document by throwing a `skip` exception. + +```javascript +var id = this['@metadata']['@id']; if (id === 'orders/999-A') throw 'skip'; // filter-out this.Freight = 15.3; -`} - +``` + + + + + +## Methods + + + +### Smuggler methods + + + + +Switches the smuggler to operate on a database other than the document store's default database. + +```java +public DatabaseSmuggler forDatabase(String databaseName); +``` + +
+ +Usage: + +```java +DatabaseSmuggler northwindSmuggler = store.smuggler().forDatabase("Northwind"); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **databaseName** | `String` | The database that the smuggler will operate on. | + +| Return value | | +|-----------|-------------| +| `DatabaseSmuggler` | A smuggler instance scoped to the specified database. | +
+ + +Exports the selected data to a file or to another database. + +```java +public Operation exportAsync(DatabaseSmugglerExportOptions options, String toFile); + +public Operation exportAsync(DatabaseSmugglerExportOptions options, DatabaseSmuggler toDatabase); +``` + +
+ +Usage: + +```java +Operation exportOperation = store.smuggler().exportAsync(options, toFile); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerExportOptions` | Options used during the export. See [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | +| **toFile** | `String` | Path to the file where exported data is written. | +| **toDatabase** | `DatabaseSmuggler` | A `DatabaseSmuggler` instance used as the destination. | +| Return value | | +|-----------|-------------| +| `Operation` | An `Operation` instance you can use to wait for completion and subscribe to progress events. | +
+ + +Imports data from a file or a stream into the database. + +```java +public Operation importAsync(DatabaseSmugglerImportOptions options, String fromFile); + +public Operation importAsync(DatabaseSmugglerImportOptions options, InputStream stream); +``` + +
+ +Usage: + +```java +Operation importOperation = store.smuggler().importAsync(options, fromFile); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerImportOptions` | Options used during the import. See [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | +| **fromFile** | `String` | Path to the file the data is imported from. | +| **stream** | `InputStream` | Stream with the data to import. | + +| Return value | | +|-----------|-------------| +| `Operation` | An `Operation` instance you can use to wait for completion and subscribe to progress events. | + +
+
+ +
+ +## Classes + + + +### Smuggler options classes + +
+ + + + + +**Options that control what an export includes.** + +```java +class DatabaseSmugglerExportOptions +{ + EnumSet operateOnTypes + EnumSet operateOnDatabaseRecordType + boolean includeExpired + boolean includeArtificial + boolean includeArchived + boolean removeAnalyzers + String transformScript + int maxStepsForTransformScript + boolean skipRevisionCreation + String encryptionKey + List collections + boolean skipCorruptedData + ExportCompressionAlgorithm compressionAlgorithm +} +``` + +
+ +| Property | Type | Description | +|-----------|------|-------------| +| **collections** | `List` | List of specific collections to export. If empty, all collections are exported.
Default: `empty` | +| **operateOnTypes** | `EnumSet` | Indicates what should be exported.
Default: `INDEXES`, `DOCUMENTS`, `REVISION_DOCUMENTS`, `CONFLICTS`, `DATABASE_RECORD`, `REPLICATION_HUB_CERTIFICATES`, `IDENTITIES`, `COMPARE_EXCHANGE`, `ATTACHMENTS`, `COUNTER_GROUPS`, `SUBSCRIPTIONS`, `TIME_SERIES`, `TIME_SERIES_DELETED_RANGES` | +| **operateOnDatabaseRecordType** | `EnumSet` | Indicates what should be exported from the database record.
Default: `CLIENT`, `CONFLICT_SOLVER_CONFIG`, `EXPIRATION`, `EXTERNAL_REPLICATIONS`, `PERIODIC_BACKUPS`, `RAVEN_CONNECTION_STRINGS`, `RAVEN_ETLS`, `REVISIONS`, `SETTINGS`, `SQL_CONNECTION_STRINGS`, `SORTERS`, `SQL_ETLS`, `HUB_PULL_REPLICATIONS`, `SINK_PULL_REPLICATIONS`, `TIME_SERIES`, `DOCUMENTS_COMPRESSION`, `ANALYZERS`, `LOCK_MODE`, `OLAP_CONNECTION_STRINGS`, `OLAP_ETLS`, `ELASTIC_SEARCH_CONNECTION_STRINGS`, `ELASTIC_SEARCH_ETLS`, `POSTGRE_SQL_INTEGRATION`, `QUEUE_CONNECTION_STRINGS`, `QUEUE_ETLS`, `INDEXES_HISTORY`, `REFRESH`, `QUEUE_SINKS`, `DATA_ARCHIVAL`, `SNOWFLAKE_CONNECTION_STRINGS`, `SNOWFLAKE_ETLS`, `EMBEDDINGS_GENERATIONS`, `AI_CONNECTION_STRINGS`, `GEN_AI_ETLS`, `AI_AGENTS`, `REMOTE_ATTACHMENTS`, `SCHEMA_VALIDATION` | +| **includeExpired** | `boolean` | Should expired documents be exported.
Default: `true` | +| **includeArtificial** | `boolean` | Should artificial documents be exported.
Default: `false` | +| **includeArchived** | `boolean` | Should archived documents be exported.
Default: `true` | +| **removeAnalyzers** | `boolean` | Should analyzers be removed from indexes.
Default: `false` | +| **transformScript** | `String` | JavaScript applied to every exported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **maxStepsForTransformScript** | `int` | Maximum number of steps the transform script can run before failing.
Default: `10000` | +| **skipRevisionCreation** | `boolean` | When `true`, revisions are not created for the processed documents.
Default: `false` | +| **encryptionKey** | `String` | Encryption key used to encrypt the exported file.
Default: `null` | +| **skipCorruptedData** | `boolean` | When `true`, the export continues if corrupted data is encountered (for example, lost compression dictionaries), recording an error entry in the result report instead of stopping.
Default: `false` | +| **compressionAlgorithm** | `ExportCompressionAlgorithm` | Compression algorithm used for the exported file: `Zstd` or `Gzip`.
Default: `null` | + +
+ + +**Options that control what an import includes.** + +```java +class DatabaseSmugglerImportOptions +{ + EnumSet operateOnTypes + EnumSet operateOnDatabaseRecordType + boolean includeExpired + boolean includeArtificial + boolean includeArchived + boolean removeAnalyzers + String transformScript + int maxStepsForTransformScript + boolean skipRevisionCreation + String encryptionKey + List collections + boolean skipCorruptedData +} +``` + +
+ +| Property | Type | Description | +|-----------|------|-------------| +| **collections** | `List` | List of specific collections to import. If empty, all collections are imported.
Default: `empty` | +| **operateOnTypes** | `EnumSet` | Indicates what should be imported.
Default: `INDEXES`, `DOCUMENTS`, `REVISION_DOCUMENTS`, `CONFLICTS`, `DATABASE_RECORD`, `REPLICATION_HUB_CERTIFICATES`, `IDENTITIES`, `COMPARE_EXCHANGE`, `ATTACHMENTS`, `COUNTER_GROUPS`, `SUBSCRIPTIONS`, `TIME_SERIES`, `TIME_SERIES_DELETED_RANGES` | +| **operateOnDatabaseRecordType** | `EnumSet` | Indicates what should be imported into the database record.
Default: `CLIENT`, `CONFLICT_SOLVER_CONFIG`, `EXPIRATION`, `EXTERNAL_REPLICATIONS`, `PERIODIC_BACKUPS`, `RAVEN_CONNECTION_STRINGS`, `RAVEN_ETLS`, `REVISIONS`, `SETTINGS`, `SQL_CONNECTION_STRINGS`, `SORTERS`, `SQL_ETLS`, `HUB_PULL_REPLICATIONS`, `SINK_PULL_REPLICATIONS`, `TIME_SERIES`, `DOCUMENTS_COMPRESSION`, `ANALYZERS`, `LOCK_MODE`, `OLAP_CONNECTION_STRINGS`, `OLAP_ETLS`, `ELASTIC_SEARCH_CONNECTION_STRINGS`, `ELASTIC_SEARCH_ETLS`, `POSTGRE_SQL_INTEGRATION`, `QUEUE_CONNECTION_STRINGS`, `QUEUE_ETLS`, `INDEXES_HISTORY`, `REFRESH`, `QUEUE_SINKS`, `DATA_ARCHIVAL`, `SNOWFLAKE_CONNECTION_STRINGS`, `SNOWFLAKE_ETLS`, `EMBEDDINGS_GENERATIONS`, `AI_CONNECTION_STRINGS`, `GEN_AI_ETLS`, `AI_AGENTS`, `REMOTE_ATTACHMENTS`, `SCHEMA_VALIDATION` | +| **includeExpired** | `boolean` | Should expired documents be imported.
Default: `true` | +| **includeArtificial** | `boolean` | Should artificial documents be imported.
Default: `false` | +| **includeArchived** | `boolean` | Should archived documents be imported.
Default: `true` | +| **removeAnalyzers** | `boolean` | Should analyzers be removed from indexes.
Default: `false` | +| **transformScript** | `String` | JavaScript applied to every imported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **maxStepsForTransformScript** | `int` | Maximum number of steps the transform script can run before failing.
Default: `10000` | +| **skipRevisionCreation** | `boolean` | When `true`, revisions are not created for the imported documents.
Default: `false` | +| **encryptionKey** | `String` | Encryption key used to read an encrypted `.ravendbdump` file.
Default: `null` | +| **skipCorruptedData** | `boolean` | When `true`, the import continues if corrupted data is encountered, recording an error entry in the result report instead of stopping.
Default: `false` | + +
+
+ + diff --git a/docs/client-api/smuggler/content/_what-is-smuggler-nodejs.mdx b/docs/client-api/smuggler/content/_what-is-smuggler-nodejs.mdx index ca9f9b333d..11a7739d76 100644 --- a/docs/client-api/smuggler/content/_what-is-smuggler-nodejs.mdx +++ b/docs/client-api/smuggler/content/_what-is-smuggler-nodejs.mdx @@ -1,150 +1,358 @@ import Admonition from '@theme/Admonition'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -import CodeBlock from '@theme/CodeBlock'; +import Panel from "@site/src/components/Panel"; +import ContentFrame from "@site/src/components/ContentFrame"; -Smuggler gives you the ability to export or import data from or to a database using JSON format. -It is exposed via the `DocumentStore.smuggler`. + -## ForDatabase +* **Smuggler** is a tool for moving data between databases. + Use it to export documents, indexes, and other items from a database as JSON, + and to import that data into another database. -By default, the `DocumentStore.smuggler` works on the default document store database from the `DocumentStore.database` . +* Smuggler is exposed through the `documentStore.smuggler` property. -In order to switch it to a different database use the `.forDatabase` method. +* An export can write to a `.ravendbdump` file. + An import reads that data back into a database. - - -{`const northwindSmuggler = store +* You can narrow what is transferred by collection, by item type, + and by applying a transform script to each document. + +* In this article: + * [ForDatabase](../../../client-api/smuggler/what-is-smuggler.mdx#fordatabase) + * [Export](../../../client-api/smuggler/what-is-smuggler.mdx#export) + * [Export specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#export-specific-collections) + * [Import](../../../client-api/smuggler/what-is-smuggler.mdx#import) + * [Import specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#import-specific-collections) + * [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript) + * [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) + * [Methods](../../../client-api/smuggler/what-is-smuggler.mdx#methods) + * [Classes](../../../client-api/smuggler/what-is-smuggler.mdx#classes) + + + + + +By default, the smuggler works on the database set in the `documentStore.database` property. +To run the smuggler against a different database, switch to it with the `forDatabase` method. + +```js +const northwindSmuggler = store .smuggler .forDatabase("Northwind"); -`} - - +``` + + -## Export +Export writes the selected data to a `.ravendbdump` file. +Use `DatabaseSmugglerExportOptions` to choose what to include. -### Usage +
- - -{`const operation = await store.smuggler.export(options, toDatabase); +#### Example: export indexes and documents to a file -const operation = await store.smuggler.export(options, toFile); -`} - - +```js +// export only Indexes and Documents to a given file +const options = new DatabaseSmugglerExportOptions(); +options.operateOnTypes = ["Indexes", "Documents"]; -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerExportOptions` | Options that will be used during the export. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | -| **toDatabase** | `DatabaseSmuggler` | `DatabaseSmuggler` instance used as a destination | -| **toFile** | `string` | Path to a file where exported data will be written | - -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | - -### DatabaseSmugglerExportOptions - -| Parameters | | | -| - | - | - | -| **collections** | ` string[]` | List of specific collections to export. If empty, then all collections will be exported. Default: `empty` | -| **operateOnTypes** | `DatabaseItemType[]` | Indicates what should be exported. Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `Identities`, `CompareExchange`, `Subscriptions` | -| **operateOnDatabaseRecordTypes** | `DatabaseRecordItemType[]` | Indicates what should be exported from database record. Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications` | -| **includeExpired** | `boolean` | Should expired documents be included in the export. Default: `true` | -| **includeArtificial** | `boolean` | ? | -| **removeAnalyzers** | `boolean` | Should analyzers be removed from Indexes. Default: `false` | -| **transformScript** | `string` | JavaScript-based script applied to every exported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **maxStepsForTransformScript** | `number` | Maximum number of steps that transform script can process before failing. Default: 10000 | -| **skipRevisionCreation** | `boolean` | skip revision creation | -| **encryptionKey** | `string` | Encryption key used for restore | - -### Example - - - -{`const options = new DatabaseSmugglerExportOptions(); -options.operateOnTypes = ["Documents"]; const operation = await store .smuggler - .export(options, "C:\\\\ravendb-exports\\\\Northwind.ravendbdump"); + .export(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + await operation.waitForCompletion(); -`} - - +``` +
+ -## Import +### Export specific collections -### Usage +By default, all collections are exported. +To export only specific collections, list their names in the `collections` option. - - -{`const operation = await store.smuggler.import(options, fromFile); +```js +// Export only the "Orders" and "Categories" collections to a file +const options = new DatabaseSmugglerExportOptions(); +options.collections = ["Orders", "Categories"]; -const operation = await store.smuggler.import(options, stream); -`} - - +const operation = await store + .smuggler + .export(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + +await operation.waitForCompletion(); +``` + + + +
+ +See [export](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. + + + + -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerImportOptions` | Options that will be used during the import. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | -| **stream** | `Stream` | Stream with data to import | -| **fromFile** | `string` | Path to a file from which data will be imported | - -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation-class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | - -### DatabaseSmugglerImportOptions - -| Parameters | | | -| - | - | - | -| **operateOnTypes** | `DatabaseItemType[]` | Indicates what should be imported. Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `Identities`, `CompareExchange`, `Subscriptions` | -| **operateOnDatabaseRecordTypes** | `DatabaseRecordItemType[]` | Indicates what should be imported. Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications` | -| **includeExpired** | `boolean` | Should expired documents be included in the import. Default: `true` | -| **includeArtificial** | `boolean` | ? | -| **removeAnalyzers** | `boolean` | Should analyzers be removed from Indexes. Default: `false` | -| **transformScript** | `string` | JavaScript-based script applied to every imported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **maxStepsForTransformScript** | `number` | Maximum number of steps that transform script can process before failing. Default: 10000 | -| **skipRevisionCreation** | `boolean` | skip revision creation | -| **encryptionKey** | `string` | Encryption key used for restore | - -### Example - - - -{`const options = new DatabaseSmugglerImportOptions(); +Import reads data from a `.ravendbdump` file and writes it into the database. +Use `DatabaseSmugglerImportOptions` to choose what to include. + +
+ +#### Example: import documents from a file + +```js +// import only Documents from a given file +const options = new DatabaseSmugglerImportOptions(); options.operateOnTypes = ["Documents"]; -const operation = await store.smuggler.import(options, "C:\\\\ravendb-exports\\\\Northwind.ravendbdump"); + +// import the .ravendbdump file that you exported (i.e. in the export example above) +const operation = await store + .smuggler + .import(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + await operation.waitForCompletion(); -`} - - +``` + +
+ + + +### Import specific collections + +By default, all collections are imported. +To import only specific collections, list their names in the `collections` option. + +```js +// Import only the "Orders" and "Categories" collections from the file +const options = new DatabaseSmugglerImportOptions(); +options.collections = ["Orders", "Categories"]; + +const operation = await store + .smuggler + .import(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + +await operation.waitForCompletion(); +``` + + + +
+See [import](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. + -## TransformScript + -`TransformScript` exposes the ability to modify or even filter-out the document during the import and export process using the provided JavaScript. +`TransformScript` lets you modify or filter out each document during an export or import, using JavaScript that you provide. -Underneath the JavaScript engine is exactly the same as used for [patching operations](../../../client-api/operations/patching/single-document.mdx) giving you identical syntax and capabilities with additional **ability to filter out documents by throwing a 'skip' exception**. +The JavaScript engine is the same one used for [patching operations](../../../client-api/operations/patching/single-document.mdx), so you have identical syntax and capabilities, plus the ability to filter out a document by throwing a `skip` exception. - - -{`var id = this['@metadata']['@id']; +```javascript +var id = this['@metadata']['@id']; if (id === 'orders/999-A') throw 'skip'; // filter-out this.Freight = 15.3; -`} - +``` + + + + + +## Methods + + + +### Smuggler methods + + + + +Switches the smuggler to operate on a database other than the document store's default database. + +```ts +forDatabase(databaseName: string): DatabaseSmuggler; +``` + +
+ +Usage: + +```js +const northwindSmuggler = store.smuggler.forDatabase("Northwind"); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **databaseName** | `string` | The database that the smuggler will operate on. | + +| Return value | | +|-----------|-------------| +| `DatabaseSmuggler` | A smuggler instance scoped to the specified database. | + +
+ + +Exports the selected data to a file. + +```ts +export(options: DatabaseSmugglerExportOptions, toFile: string): Promise; +``` + +
+ +Usage: + +```js +const operation = await store.smuggler.export(options, toFile); +await operation.waitForCompletion(); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerExportOptions` | Options used during the export. See [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | +| **toFile** | `string` | Path to the file where exported data is written. | + +| Return value | | +|-----------|-------------| +| `Promise` | An awaiter you can use to wait for the operation to complete and subscribe to progress events. | + +
+ + +Imports data from a file into the database. + +```ts +import(options: DatabaseSmugglerImportOptions, fromFile: string): Promise; +``` + +
+ +Usage: + +```js +const operation = await store.smuggler.import(options, fromFile); +await operation.waitForCompletion(); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerImportOptions` | Options used during the import. See [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | +| **fromFile** | `string` | Path to the file the data is imported from. | + +| Return value | | +|-----------|-------------| +| `Promise` | An awaiter you can use to wait for the operation to complete and subscribe to progress events. | +
+
+ +
+ +## Classes + + + +### Smuggler options classes + +
+ + + + + +**Options that control what an export includes.** + +```ts +class DatabaseSmugglerExportOptions +{ + operateOnTypes: DatabaseItemType[] + operateOnDatabaseRecordType: DatabaseRecordItemType[] + includeExpired: boolean + includeArtificial: boolean + includeArchived: boolean + removeAnalyzers: boolean + transformScript: string + maxStepsForTransformScript: number + skipRevisionCreation: boolean + encryptionKey: string + collections: string[] + skipCorruptedData: boolean + compressionAlgorithm?: ExportCompressionAlgorithm +} +``` + +
+ +| Property | Type | Description | +|-----------|------|-------------| +| **collections** | `string[]` | List of specific collections to export. If empty, all collections are exported.
Default: `empty` | +| **operateOnTypes** | `DatabaseItemType[]` | Indicates what should be exported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `ReplicationHubCertificates`, `Identities`, `CompareExchange`, `Attachments`, `CounterGroups`, `Subscriptions`, `TimeSeries` | +| **operateOnDatabaseRecordType** | `DatabaseRecordItemType[]` | Indicates what should be exported from the database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `Settings`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications`, `TimeSeries`, `DocumentsCompression`, `Analyzers`, `LockMode`, `OlapConnectionStrings`, `OlapEtls`, `ElasticSearchConnectionStrings`, `ElasticSearchEtls`, `PostgreSqlIntegration`, `QueueConnectionStrings`, `QueueEtl`, `IndexesHistory`, `Refresh`, `QueueSinks`, `DataArchival` | +| **includeExpired** | `boolean` | Should expired documents be exported.
Default: `true` | +| **includeArtificial** | `boolean` | Should artificial documents be exported.
Default: `false` | +| **includeArchived** | `boolean` | Should archived documents be exported.
Default: `true` | +| **removeAnalyzers** | `boolean` | Should analyzers be removed from indexes.
Default: `false` | +| **transformScript** | `string` | JavaScript applied to every exported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **maxStepsForTransformScript** | `number` | Maximum number of steps the transform script can run before failing.
Default: `10000` | +| **skipRevisionCreation** | `boolean` | When `true`, revisions are not created for the processed documents.
Default: `false` | +| **encryptionKey** | `string` | Encryption key used to encrypt the exported file.
Default: `null` | +| **skipCorruptedData** | `boolean` | When `true`, the export continues if corrupted data is encountered (for example, lost compression dictionaries), recording an error entry in the result report instead of stopping.
Default: `false` | +| **compressionAlgorithm** | `ExportCompressionAlgorithm` | Compression algorithm used for the exported file: `Zstd` or `Gzip`.
Default: `null` | +
+ + +**Options that control what an import includes.** + +```ts +class DatabaseSmugglerImportOptions +{ + operateOnTypes: DatabaseItemType[] + operateOnDatabaseRecordType: DatabaseRecordItemType[] + includeExpired: boolean + includeArtificial: boolean + includeArchived: boolean + removeAnalyzers: boolean + transformScript: string + maxStepsForTransformScript: number + skipRevisionCreation: boolean + encryptionKey: string + collections: string[] + skipCorruptedData: boolean +} +``` + +
+ +| Property | Type | Description | +|-----------|------|-------------| +| **collections** | `string[]` | List of specific collections to import. If empty, all collections are imported.
Default: `empty` | +| **operateOnTypes** | `DatabaseItemType[]` | Indicates what should be imported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `ReplicationHubCertificates`, `Identities`, `CompareExchange`, `Attachments`, `CounterGroups`, `Subscriptions`, `TimeSeries` | +| **operateOnDatabaseRecordType** | `DatabaseRecordItemType[]` | Indicates what should be imported into the database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `Settings`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications`, `TimeSeries`, `DocumentsCompression`, `Analyzers`, `LockMode`, `OlapConnectionStrings`, `OlapEtls`, `ElasticSearchConnectionStrings`, `ElasticSearchEtls`, `PostgreSqlIntegration`, `QueueConnectionStrings`, `QueueEtl`, `IndexesHistory`, `Refresh`, `QueueSinks`, `DataArchival` | +| **includeExpired** | `boolean` | Should expired documents be imported.
Default: `true` | +| **includeArtificial** | `boolean` | Should artificial documents be imported.
Default: `false` | +| **includeArchived** | `boolean` | Should archived documents be imported.
Default: `true` | +| **removeAnalyzers** | `boolean` | Should analyzers be removed from indexes.
Default: `false` | +| **transformScript** | `string` | JavaScript applied to every imported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **maxStepsForTransformScript** | `number` | Maximum number of steps the transform script can run before failing.
Default: `10000` | +| **skipRevisionCreation** | `boolean` | When `true`, revisions are not created for the imported documents.
Default: `false` | +| **encryptionKey** | `string` | Encryption key used to read an encrypted `.ravendbdump` file.
Default: `null` | +| **skipCorruptedData** | `boolean` | When `true`, the import continues if corrupted data is encountered, recording an error entry in the result report instead of stopping.
Default: `false` | +
+
+ + diff --git a/docs/studio/database/tasks/import-data/assets/import-command-powershell.png b/docs/studio/database/tasks/import-data/assets/import-command-powershell.png index 32d855711e..eba063c988 100644 Binary files a/docs/studio/database/tasks/import-data/assets/import-command-powershell.png and b/docs/studio/database/tasks/import-data/assets/import-command-powershell.png differ diff --git a/docs/studio/database/tasks/import-data/assets/import-from-file-advanced-configuration-ongoing-tasks.png b/docs/studio/database/tasks/import-data/assets/import-from-file-advanced-configuration-ongoing-tasks.png index b462061dbb..cddedd4950 100644 Binary files a/docs/studio/database/tasks/import-data/assets/import-from-file-advanced-configuration-ongoing-tasks.png and b/docs/studio/database/tasks/import-data/assets/import-from-file-advanced-configuration-ongoing-tasks.png differ diff --git a/docs/studio/database/tasks/import-data/assets/import-from-file-advanced-transform-options.png b/docs/studio/database/tasks/import-data/assets/import-from-file-advanced-transform-options.png new file mode 100644 index 0000000000..517f60d255 Binary files /dev/null and b/docs/studio/database/tasks/import-data/assets/import-from-file-advanced-transform-options.png differ diff --git a/docs/studio/database/tasks/import-data/assets/import-from-file-advanced-transform-script.png b/docs/studio/database/tasks/import-data/assets/import-from-file-advanced-transform-script.png index 57e6fe1d17..bd32cbbc4c 100644 Binary files a/docs/studio/database/tasks/import-data/assets/import-from-file-advanced-transform-script.png and b/docs/studio/database/tasks/import-data/assets/import-from-file-advanced-transform-script.png differ diff --git a/docs/studio/database/tasks/import-data/assets/import-from-file-import-all-collections.png b/docs/studio/database/tasks/import-data/assets/import-from-file-import-all-collections.png new file mode 100644 index 0000000000..2b2ffa318e Binary files /dev/null and b/docs/studio/database/tasks/import-data/assets/import-from-file-import-all-collections.png differ diff --git a/docs/studio/database/tasks/import-data/assets/import-from-file-max-read-ops-per-second.png b/docs/studio/database/tasks/import-data/assets/import-from-file-max-read-ops-per-second.png new file mode 100644 index 0000000000..8e7c8bc1d7 Binary files /dev/null and b/docs/studio/database/tasks/import-data/assets/import-from-file-max-read-ops-per-second.png differ diff --git a/docs/studio/database/tasks/import-data/assets/import-from-file-options.png b/docs/studio/database/tasks/import-data/assets/import-from-file-options.png index 98ef804ebd..488c5cf5ac 100644 Binary files a/docs/studio/database/tasks/import-data/assets/import-from-file-options.png and b/docs/studio/database/tasks/import-data/assets/import-from-file-options.png differ diff --git a/docs/studio/database/tasks/import-data/assets/snagit/import-from-file-advanced-configuration-ongoing-tasks.snagx b/docs/studio/database/tasks/import-data/assets/snagit/import-from-file-advanced-configuration-ongoing-tasks.snagx new file mode 100644 index 0000000000..6c7755e3d1 Binary files /dev/null and b/docs/studio/database/tasks/import-data/assets/snagit/import-from-file-advanced-configuration-ongoing-tasks.snagx differ diff --git a/docs/studio/database/tasks/import-data/assets/snagit/import-from-file-advanced-transform-options.snagx b/docs/studio/database/tasks/import-data/assets/snagit/import-from-file-advanced-transform-options.snagx new file mode 100644 index 0000000000..ad9c861dc9 Binary files /dev/null and b/docs/studio/database/tasks/import-data/assets/snagit/import-from-file-advanced-transform-options.snagx differ diff --git a/docs/studio/database/tasks/import-data/assets/snagit/import-from-file-options.snagx b/docs/studio/database/tasks/import-data/assets/snagit/import-from-file-options.snagx new file mode 100644 index 0000000000..a2e6382f13 Binary files /dev/null and b/docs/studio/database/tasks/import-data/assets/snagit/import-from-file-options.snagx differ diff --git a/docs/studio/database/tasks/import-data/assets/snagit/studio-view-import-fromfile-steps.snagx b/docs/studio/database/tasks/import-data/assets/snagit/studio-view-import-fromfile-steps.snagx new file mode 100644 index 0000000000..16aef39be7 Binary files /dev/null and b/docs/studio/database/tasks/import-data/assets/snagit/studio-view-import-fromfile-steps.snagx differ diff --git a/docs/studio/database/tasks/import-data/assets/studio-view-import-fromfile-steps.png b/docs/studio/database/tasks/import-data/assets/studio-view-import-fromfile-steps.png index 1bc72fb6a5..b4923b8fa1 100644 Binary files a/docs/studio/database/tasks/import-data/assets/studio-view-import-fromfile-steps.png and b/docs/studio/database/tasks/import-data/assets/studio-view-import-fromfile-steps.png differ diff --git a/docs/studio/database/tasks/import-data/import-data-file.mdx b/docs/studio/database/tasks/import-data/import-data-file.mdx index ff8eea4057..5abc380dba 100644 --- a/docs/studio/database/tasks/import-data/import-data-file.mdx +++ b/docs/studio/database/tasks/import-data/import-data-file.mdx @@ -1,174 +1,202 @@ --- title: "Import data from .ravendbdump file" sidebar_label: Import from File (.ravendbdump) -description: "Import a .ravendbdump file into a RavenDB database through Studio with collection filtering and transform script options." +description: "Import a .ravendbdump file into a RavenDB database through Studio, with collection filtering and transform script options." sidebar_position: 0 --- import Admonition from '@theme/Admonition'; -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; -import CodeBlock from '@theme/CodeBlock'; -import LanguageSwitcher from "@site/src/components/LanguageSwitcher"; -import LanguageContent from "@site/src/components/LanguageContent"; +import Panel from "@site/src/components/Panel"; +import ContentFrame from "@site/src/components/ContentFrame"; # Import data from .ravendbdump file -* `.ravendbdump` is RavenDB's format for [exportation of data](../export-database.mdx) - into and from files. +* `.ravendbdump` is RavenDB's format for [exporting data](../export-database.mdx) into and out of files. -* RavenDB dump files can be used to easily migrate data from one database - to another. They are commonly used when the database is upgraded. +* A `.ravendbdump` file can be used to migrate data from one database to another. + It is commonly used when a database is upgraded. -* an exported `.ravendbdump` file can be used to migrate the data stored - in a non-sharded database into a sharded database. - Learn more about this operation in the [section dedicated to it](../../../../sharding/import-and-export.mdx#import). +* An exported `.ravendbdump` file can also migrate data from a non-sharded database into a sharded database. + Learn more in the [section dedicated to this operation](../../../../sharding/import-and-export.mdx#import). -In this page: - -* [Import data to destination server from file](../../../../studio/database/tasks/import-data/import-data-file.mdx#import-data-to-destination-server-from-file) -* [Import options](../../../../studio/database/tasks/import-data/import-data-file.mdx#import-options) -* [Advanced import options](../../../../studio/database/tasks/import-data/import-data-file.mdx#advanced-import-options) - * [Transform Script](../../../../studio/database/tasks/import-data/import-data-file.mdx#transform-script) - * [Customize Configuration and Ongoing Tasks](../../../../studio/database/tasks/import-data/import-data-file.mdx#customize-configuration-and-ongoing-tasks) - * [Copy command as PowerShell](../../../../studio/database/tasks/import-data/import-data-file.mdx#copy-command-as-powershell) +* In this article: + * [Import data from a file](../../../../studio/database/tasks/import-data/import-data-file.mdx#import-data-from-a-file) + * [Import options](../../../../studio/database/tasks/import-data/import-data-file.mdx#import-options) + * [Advanced import options](../../../../studio/database/tasks/import-data/import-data-file.mdx#advanced-import-options) + * [Import all collections](../../../../studio/database/tasks/import-data/import-data-file.mdx#1-import-all-collections) + * [Use Transform script](../../../../studio/database/tasks/import-data/import-data-file.mdx#2-use-transform-script) + * [Customize Configuration and Ongoing Tasks](../../../../studio/database/tasks/import-data/import-data-file.mdx#3-customize-configuration-and-ongoing-tasks) + * [Set Max Read Operations Per Second](../../../../studio/database/tasks/import-data/import-data-file.mdx#4-set-max-read-operations-per-second) + * [Import Command](../../../../studio/database/tasks/import-data/import-data-file.mdx#5-import-command) + + +If you have already [exported a database](../export-database.mdx), follow these steps in Studio when you are ready to import from the file. + +![Importing a database from a file](./assets/studio-view-import-fromfile-steps.png) + +1. Select the **Tasks menu**. + +2. Open the **Import data** view. + +3. **From file** tab. + Select the **From file (.ravendbdump)** tab to import from a database that was previously [exported](../export-database.mdx) to a file. + +4. **Overwrite warning**. + Be aware that importing from a file will overwrite any existing documents and indexes that share the same IDs. + To avoid rewriting your data, you can [create a new database](../../create-new-database/general-flow.mdx) and import into it. + +5. **Select file**. + Choose the exported `.ravendbdump` file that you want to import data from, by entering the file name or browsing your file system. + +6. **Import options** + Select the data to import from the file. See [Import options](../../../../studio/database/tasks/import-data/import-data-file.mdx#import-options) below. + If you exported an encrypted file, enable **Imported file is encrypted** and provide the encryption key. -## Import data to destination server from file - -If you have already [exported a database](../export-database.mdx), follow these steps in the studio when you're ready to import from file. - -![Figure 1. Importing Database From File ](./assets/studio-view-import-fromfile-steps.png) - -1. **Tasks** - In the destination server, click Tasks tab. -2. **Import Data**. - Click to see various impoprt options. -3. **From file (.ravendbdump)** - Seclect to import from a source database which has previously been [exported](../export-database.mdx). -4. **Warning** - *Make sure that you are not writing over data that you want to keep*. - One option is [to start a new database with the studio](https://ravendb.net/docs/article-page/5.2/csharp/studio/database/create-new-database/general-flow). -5. **Select file** - Select the `.ravendbdump` file that you previously exported from the source server. -6. **Documents and Extensions** - Select desired [options](../../tasks/import-data/import-data-file.mdx#import-options). - *If you encrypted while exporting* make sure to select **imported file is encrypted**. - You will need to paste the encryption key that you saved when creating the encrypted database. 7. **Advanced** - Click to configure [advanced import options] + Present [advanced import options](../../../../studio/database/tasks/import-data/import-data-file.mdx#advanced-import-options). + 8. **Import Database** - Click after configuring the new database to finalize the import task. + Click to import the selected data. + + +![Import options](./assets/import-from-file-options.png) +Select the entities that you want to import: -## Import options +| 1. Documents and Extensions | 2. Cluster Entities | +|-----------------------------|---------------------| +| [Documents](../../../../studio/database/documents/document-view.mdx) | [Indexes](../../../../indexes/what-are-indexes.mdx) | +|    [Attachments](../../../../document-extensions/attachments/overview.mdx) |    [Index history](../../../../studio/database/indexes/index-history.mdx) | +|    [Legacy attachments](../../../../studio/database/create-new-database/from-legacy-files.mdx) |    [Remove analyzers](../../../../indexes/using-analyzers.mdx) | +|    [Counters](../../../../document-extensions/counters/overview.mdx) | [Identities](../../../../client-api/document-identifiers/working-with-document-identifiers.mdx) | +|    [Time series](../../../../document-extensions/timeseries/overview.mdx) | [Subscriptions](../../../../client-api/data-subscriptions/what-are-data-subscriptions.mdx) | +|    [Revisions](../../../../document-extensions/revisions/overview.mdx) | [Configuration and Ongoing Tasks](../../../../studio/database/tasks/import-data/import-data-file.mdx#3-customize-configuration-and-ongoing-tasks) | +| [Artificial documents](../../../../studio/database/indexes/create-map-reduce-index.mdx#artificial-documents--vs--regular-documents) | [Compare exchange items](../../../../compare-exchange/overview.mdx) | +| [Archived documents](../../../../data-archival/overview.mdx) | Compare exchange tombstones | +| [Expired documents](../../../../server/extensions/expiration.mdx) | | +| [Conflicts](../../../../client-api/cluster/document-conflicts-in-client-side.mdx) | | +| [Document tombstones](../../../../glossary/tombstone.mdx) | | +| Time-series deleted ranges | | -Here you can filter the data you want to import, select configuration and apply a transform script on your documents. +Options that are selected but have no matching items in the imported file are skipped during import. -![Figure 2. Import Options](./assets/import-from-file-options.png) +**3. Imported file is encrypted** +Enable this option when importing from an encrypted file, and provide the decryption key. +Make sure that **Encrypt exported file** was enabled when exporting from the source database. + -1. [Include Documents](../../../../studio/database/documents/document-view.mdx) - Toggle to include documents and to enable inclusion of the following document related items: - - [Include Attachments](../../../../document-extensions/attachments/overview.mdx) - - [Include Legacy Attachments](../../../../studio/database/create-new-database/from-legacy-files.mdx) - Determines whether or not legacy attachments contained in the file should be imported where legacy attachments refers to v2.x and v3.x attachments. - - [Include Counters](../../../../document-extensions/counters/overview.mdx) - - [Include Legacy Files](../../../../studio/database/create-new-database/from-legacy-files.mdx) - - [Include Artificial Documents](../../../../studio/database/indexes/create-map-reduce-index.mdx#artificial-documents--vs--regular-documents) - - [Include Expired Documents](../../../../server/extensions/expiration.mdx) - - [Include Revisions](../../../../document-extensions/revisions/overview.mdx) - - [Include Conflicts](../../../../client-api/cluster/document-conflicts-in-client-side.mdx) - -2. [Include Indexes](../../../../indexes/what-are-indexes.mdx) - - [Remove Analyzers](../../../../indexes/using-analyzers.mdx) - - [Include Identities](../../../../client-api/document-identifiers/working-with-document-identifiers.mdx) - - [Include Compare Exchange](../../../../compare-exchange/overview.mdx) - - [Include Subscriptions](../../../../client-api/data-subscriptions/what-are-data-subscriptions.mdx) - - [Include Configuration and OngoingTasks](../../../../studio/database/tasks/import-data/import-from-ravendb.mdx#customize-configuration-and-ongoing-tasks) - - -3. [Imported file is encrypted](../../../../server/security/overview.mdx#encryption) - Toggle to include the decryption key when importing data from encrypted file. - Make sure that **Encrypt exported file** option was selected when exporting from source database so that the encryption key is included. + - -If any of the options is set but the source database doesn't contain any items of that type, the item will be skipped. - +Click **Advanced** to reveal the following options. +![Advanced import options](./assets/import-from-file-advanced-transform-options.png) + +#### 1. Import all collections +![Import specific collections](./assets/import-from-file-import-all-collections.png) -## Advanced import options +By default, **all** exported collections are imported. +To import only specific collections, disable **Import all collections**, enter the name of each collection that you want to import, and click **Add collection**. +Enter collection names exactly as they appear in the source database. -Click the **Advanced** button at the bottom of the options view for the following import features. + -### Transform Script + -In the Studio, select **database** > click **Task** tab > select **Import** > select **From file (.ravendbdump)** > click the **Advanced** button at the bottom > toggle **Use Transform script**. +#### 2. Use Transform script -![Figure 3. Advanced Import Options - Transform Script](./assets/import-from-file-advanced-transform-script.png) +![Use transform script](./assets/import-from-file-advanced-transform-script.png) -- **Use Transform Script** - Enabling this **advanced** option allows you to provide a transform javascript, that would operate on each document imported from the file. +Enable this option to provide a JavaScript transform script that runs on each document imported from the file. - Sample transform javascript: +The current document is available under the `this` variable. +The `@change-vector`, `@id`, and `@last-modified` metadata fields are not available to the script. - - -{`delete this['@metadata']['@change-vector'] -// The script above will delete the change-vector from imported documents -// and will generate new change vectors during import. -// This is very helpfull if the data is imported from a diffrent database group -// and you want to avoid adding old change vector entries to a new environment. -`} - - -### Customize Configuration and Ongoing Tasks +For example, use the following script to skip discontinued products and stamp every imported document with the time it was imported: -This **advanced** option enables you to choose whether to import various ongoing tasks, connection strings and advanced configurations. +```javascript +// Skip documents you don't want to import +// (throwing 'skip' filters the document out) +if (this.Discontinued === true) { + throw 'skip'; +} + +// Add or modify regular fields on the imported document +this.ImportedAt = new Date().toISOString(); +``` + + + + + +#### 3. Customize Configuration and Ongoing Tasks + +![Customize configuration and ongoing tasks](./assets/import-from-file-advanced-configuration-ongoing-tasks.png) + +Use this option to select ongoing tasks, configurations, and connection strings to import. +Selected options with no matching items in the imported file will be skipped during import. + +| Ongoing tasks | Other | Connection Strings | +|---------------|-------|--------------------| +| [Periodic backups](../../../../backup/create/periodic-tasks/database-backup.mdx) | [Settings](../../../../studio/database/settings/database-settings.mdx) | RavenDB connection strings | +| [External replications](../../../../studio/database/tasks/ongoing-tasks/external-replication-task.mdx) | [Conflict solver configuration](../../../../client-api/operations/server-wide/modify-conflict-solver.mdx) | SQL connection strings | +| [RavenDB ETLs](../../../../server/ongoing-tasks/etl/raven.mdx) | [Client configuration](../../../../studio/server/client-configuration.mdx) | Snowflake connection strings | +| [SQL ETLs](../../../../server/ongoing-tasks/etl/sql.mdx) | [Revisions configuration](../../../../document-extensions/revisions/client-api/operations/configure-revisions.mdx) | OLAP connection strings | +| [Snowflake ETLs](../../../../server/ongoing-tasks/etl/snowflake.mdx) | [Document refresh](../../../../server/extensions/refresh.mdx) | Elasticsearch connection strings | +| [OLAP ETLs](../../../../server/ongoing-tasks/etl/olap.mdx) | [Document expiration](../../../../server/extensions/expiration.mdx) | Queue connection strings (Kafka, RabbitMQ, Azure queue storage) | +| [Elasticsearch ETLs](../../../../server/ongoing-tasks/etl/elasticsearch.mdx) | [Documents compression](../../../../server/storage/documents-compression.mdx) | [AI connection strings](../../../../ai-integration/connection-strings/overview.mdx) | +| [Queue ETLs](../../../../server/ongoing-tasks/etl/queue-etl/overview.mdx) (Kafka, RabbitMQ, Azure queue storage) | [Time series configuration](../../../../studio/database/settings/time-series-settings.mdx) | | +| [Replication hubs](../../../../studio/database/tasks/ongoing-tasks/hub-sink-replication/overview.mdx) | [Custom sorters](../../../../querying/sorting-query-results/custom-sorters/overview.mdx) | | +| [Replication sinks](../../../../studio/database/tasks/ongoing-tasks/hub-sink-replication/overview.mdx) | [Custom analyzers](../../../../studio/database/settings/custom-analyzers.mdx) | | +| [Embeddings Generation](../../../../ai-integration/generating-embeddings/overview.mdx) | [PostgreSQL integration](../../../../integrations/postgresql-protocol/overview.mdx) | | +| [GenAI](../../../../ai-integration/gen-ai-integration/overview.mdx) | [AI agents](../../../../ai-integration/ai-agents/overview.mdx) | | +| | [Remote attachments](../../../../document-extensions/attachments/configure-remote-attachments.mdx) | | + + +Imported ongoing tasks are **disabled** by default. + -In the Studio, select **database** > click **Task** tab > select **Import** > select **From file (.ravendbdump)** > click the **Advanced** button at the bottom > toggle **Customize configuration and Ongoing Tasks**. + -![Figure 4. Advanced Import Options - Customize Configuration and Ongoing Tasks](./assets/import-from-file-advanced-configuration-ongoing-tasks.png) + -Toggle to include the following configurations and ongoing tasks. -If a task is included but doesn't exist in your source database, it will be skipped. +#### 4. Set Max Read Operations Per Second -**Ongoing tasks:** +![Set max read operations per second](./assets/import-from-file-max-read-ops-per-second.png) -- [Periodic Backups](../../../../backup/create/periodic-tasks/database-backup#creating-and-managing-periodic-backups-via-studio) -- [External replications](../../../../studio/database/tasks/ongoing-tasks/external-replication-task.mdx) -- [ETL Tasks - Extract, Transform, Load](../../../../server/ongoing-tasks/etl/basics.mdx) -- [Pull Replication Sinks](../../../../studio/database/tasks/ongoing-tasks/hub-sink-replication/overview.mdx) -- [Pull Replication Hubs](../../../../studio/database/tasks/ongoing-tasks/hub-sink-replication/overview.mdx) +Enable this option to set a maximum number of read operations per second for the import. +Capping this rate makes the import lighter on the server, leaving more capacity for other work running on the database while the import is in progress, at the cost of a slower import. -**Other:** + -- [Settings](../../../../studio/database/settings/database-settings.mdx) -- [Conflict Solver Configuration](../../../../client-api/operations/server-wide/modify-conflict-solver.mdx) -- [Revisions Configuration](../../../../document-extensions/revisions/client-api/operations/configure-revisions.mdx) -- [Document Expiration](../../../../server/extensions/expiration.mdx) -- [Client Configuration](../../../../studio/server/client-configuration.mdx) -- [Custom Sorters](../../../../querying/sorting-query-results/custom-sorters/overview.mdx) + -**Connection Strings:** +#### 5. Import Command -- [Connection Strings](../../../../client-api/operations/maintenance/connection-strings/add-connection-string.mdx) used by ETL tasks to authenticate and connect to external databases will be imported. -### Copy command as PowerShell +![Import command](./assets/import-command-powershell.png) -In the Studio, select **database** > click **Task** tab > select **Import** > select **From file (.ravendbdump)** > click the **Advanced** button at the bottom. +Studio can generate a ready-to-run command that performs this same import from a command line, using the options you configured above. +This is useful for repeating the import later or running it from a script. -![Figure 5. Import Command Powershell](./assets/import-command-powershell.png) +Select the format you need, **PowerShell**, **Cmd**, or **Bash**, then click the copy icon to copy the generated command to the clipboard. -- Generates the commands to run the importing logic from PowerShell based on your configurations above. +The command uses `curl` to send your `.ravendbdump` file and the selected options to the database's import endpoint. +For example, the generated PowerShell command looks similar to this: +```powershell +curl.exe -F 'importOptions={\"OperateOnTypes\":\"Documents,Indexes\",\"IncludeExpired\":true}' -F 'file=@.\Northwind.ravendbdump' http://127.0.0.1:8080/databases/Northwind/smuggler/import +``` + + diff --git a/versioned_docs/version-6.2/client-api/smuggler/content/_what-is-smuggler-csharp.mdx b/versioned_docs/version-6.2/client-api/smuggler/content/_what-is-smuggler-csharp.mdx index 4bdf46a8fc..cdcc1e71ba 100644 --- a/versioned_docs/version-6.2/client-api/smuggler/content/_what-is-smuggler-csharp.mdx +++ b/versioned_docs/version-6.2/client-api/smuggler/content/_what-is-smuggler-csharp.mdx @@ -1,179 +1,397 @@ import Admonition from '@theme/Admonition'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -import CodeBlock from '@theme/CodeBlock'; +import Panel from "@site/src/components/Panel"; +import ContentFrame from "@site/src/components/ContentFrame"; -Smuggler gives you the ability to export or import data from or to a database using JSON format. -It is exposed via the `DocumentStore.Smuggler` property. + -## ForDatabase +* **Smuggler** is a tool for moving data between databases. + Use it to export documents, indexes, and other items from a database as JSON, + and to import that data into another database. -By default, the `DocumentStore.Smuggler` works on the default document store database from the `DocumentStore.Database` property. +* Smuggler is exposed through the `DocumentStore.Smuggler` property. -In order to switch it to a different database use the `.ForDatabase` method. +* An export can write to a `.ravendbdump` file or stream directly into another database. + An import reads that data back into a database. - - -{`var northwindSmuggler = store - .Smuggler - .ForDatabase("Northwind"); -`} - - +* You can narrow what is transferred by collection, by item type, + and by applying a transform script to each document. +* In this article: + * [ForDatabase](../../../client-api/smuggler/what-is-smuggler.mdx#fordatabase) + * [Export](../../../client-api/smuggler/what-is-smuggler.mdx#export) + * [Export specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#export-specific-collections) + * [Import](../../../client-api/smuggler/what-is-smuggler.mdx#import) + * [Import specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#import-specific-collections) + * [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript) + * [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) + * [Methods](../../../client-api/smuggler/what-is-smuggler.mdx#methods) + * [Classes](../../../client-api/smuggler/what-is-smuggler.mdx#classes) + -## Export + -### Syntax +By default, `DocumentStore.Smuggler` works on the database set in the `DocumentStore.Database` property. +To run Smuggler against a different database, switch to it with the `ForDatabase` method. - - -{`Task ExportAsync( - DatabaseSmugglerExportOptions options, - DatabaseSmuggler toDatabase, - CancellationToken token = default(CancellationToken)); - -Task ExportAsync( - DatabaseSmugglerExportOptions options, - string toFile, - CancellationToken token = default(CancellationToken)); -`} - - +```csharp +var northwindSmuggler = store + .Smuggler + .ForDatabase("Northwind"); +``` -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerExportOptions` | Options that will be used during the export. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | -| **toDatabase** | `DatabaseSmuggler` | `DatabaseSmuggler` instance used as a destination | -| **toFile** | `string` | Path to a file where exported data will be written | -| **token** | `CancellationToken` | Token used to cancel the operation | + -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | + -### DatabaseSmugglerExportOptions +Export writes the selected data to a `.ravendbdump` file, or streams it directly into another database. +Use `DatabaseSmugglerExportOptions` to choose what to include. -| Parameters | | | -| ------------- | ------------- | ----- | -| **Collections** | `List` | List of specific collections to export. If empty, then all collections will be exported.
Default: `empty` | -| **OperateOnTypes** | `DatabaseItemType` | Indicates what should be exported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `ReplicationHubCertificates`, `Identities`, `CompareExchange`, `Attachments`, `CounterGroups`, `Subscriptions`, `TimeSeries` | -| **OperateOnDatabaseRecordTypes** | `DatabaseRecordItemType` | Indicates what should be exported from database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `Settings`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications`, `TimeSeries`, `DocumentsCompression`, `Analyzers`, `LockMode`, `OlapConnectionStrings`, `OlapEtls`, `ElasticSearchConnectionStrings`, `ElasticSearchEtls`, `PostgreSQLIntegration`, `QueueConnectionStrings`, `QueueEtls`, `IndexesHistory`, `Refresh`, `DataArchival` | -| **IncludeExpired** | `bool` | Should expired documents be exported.
Default: `true` | -| **IncludeArtificial** | `bool` | Should artificial documents be exported.
Default: `false` | -| **IncludeArchived** | `bool` | Should archived documents be exported.
Default: `true` | -| **RemoveAnalyzers** | `bool` | Should analyzers be removed from Indexes.
Default: `false` | -| **TransformScript** | `string` | JavaScript-based script applied to every exported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **MaxStepsForTransformScript** | `int` | Maximum number of steps that the transform script can process before failing.
Default: **10000** | +
-### Example +#### Example: export indexes and documents to a file - - -{`// export only Indexes and Documents to a given file +```csharp +// export only Indexes and Documents to a given file var exportOperation = await store .Smuggler .ExportAsync( new DatabaseSmugglerExportOptions - \{ + { OperateOnTypes = DatabaseItemType.Indexes | DatabaseItemType.Documents - \}, - @"C:\\ravendb-exports\\Northwind.ravendbdump", + }, + @"C:\ravendb-exports\Northwind.ravendbdump", token); await exportOperation.WaitForCompletionAsync(); -`} - - +``` +
+ -## Import +### Export specific collections -### Syntax +By default, all collections are exported. +To export only specific collections, list their names in the `Collections` option. - - -{`Task ImportAsync( - DatabaseSmugglerImportOptions options, - Stream stream, - CancellationToken token = default(CancellationToken)); +```csharp +// Export only the "Orders" and "Categories" collections to a file +var exportOperation = await store + .Smuggler + .ExportAsync( + new DatabaseSmugglerExportOptions + { + Collections = new List { "Orders", "Categories" } + }, + @"C:\ravendb-exports\Northwind.ravendbdump", + token); -Task ImportAsync( - DatabaseSmugglerImportOptions options, - string fromFile, - CancellationToken token = default(CancellationToken)); -`} - - +await exportOperation.WaitForCompletionAsync(); +``` -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerImportOptions` | Options that will be used during the import. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | -| **stream** | `Stream` | Stream with data to import | -| **fromFile** | `string` | Path to a file from which data will be imported | -| **token** | `CancellationToken` | Token used to cancel the operation | + -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation-class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | +
-### DatabaseSmugglerImportOptions +See [ExportAsync](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. -| Parameters | | | -| - | - | - | -| **Collections** | `List` | List of specific collections to import. If empty, then all collections will be imported.
Default: `empty` | -| **OperateOnTypes** | `DatabaseItemType` | Indicates what should be imported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `ReplicationHubCertificates`, `Identities`, `CompareExchange`, `Attachments`, `CounterGroups`, `Subscriptions`, `TimeSeries` | -| **OperateOnDatabaseRecordTypes** | `DatabaseRecordItemType` | Indicates what should be imported from database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `Settings`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications`, `TimeSeries`, `DocumentsCompression`, `Analyzers`, `LockMode`, `OlapConnectionStrings`, `OlapEtls`, `ElasticSearchConnectionStrings`, `ElasticSearchEtls`, `PostgreSQLIntegration`, `QueueConnectionStrings`, `QueueEtls`, `IndexesHistory`, `Refresh`, `DataArchival` | -| **IncludeExpired** | `bool` | Should expired documents be imported.
Default: `true` | -| **IncludeArtificial** | `bool` | Should artificial documents be imported.
Default: `false` | -| **IncludeArchived** | `bool` | Should archived documents be imported.
Default: `true` | -| **RemoveAnalyzers** | `bool` | Should analyzers be removed from Indexes.
Default: `false` | -| **TransformScript** | `string` | JavaScript-based script applied to every imported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **MaxStepsForTransformScript** | `int` | Maximum number of steps that the transform script can process before failing.
Default: **10000** | + + + -### Example +Import reads data from a `.ravendbdump` file or a stream and writes it into the database. +Use `DatabaseSmugglerImportOptions` to choose what to include. - - -{`// import only Documents from a given file +
+ +#### Example: import documents from a file + +```csharp +// import only Documents from a given file var importOperation = await store .Smuggler .ImportAsync( new DatabaseSmugglerImportOptions - \{ + { OperateOnTypes = DatabaseItemType.Documents - \}, + }, // import the .ravendbdump file that you exported (i.e. in the export example above) - @"C:\\ravendb-exports\\Northwind.ravendbdump", + @"C:\ravendb-exports\Northwind.ravendbdump", token); await importOperation.WaitForCompletionAsync(); -`} - - +``` +
+ -## TransformScript +### Import specific collections -`TransformScript` exposes the ability to modify or even filter-out the document during the import and export process using the provided JavaScript. +By default, all collections are imported. +To import only specific collections, list their names in the `Collections` option. -Underneath the JavaScript engine is exactly the same as used for [patching operations](../../../client-api/operations/patching/single-document.mdx) giving you identical syntax and capabilities with additional **ability to filter out documents by throwing a 'skip' exception**. +```csharp +// Import only the "Orders" and "Categories" collections from the file +var importOperation = await store + .Smuggler + .ImportAsync( + new DatabaseSmugglerImportOptions + { + Collections = new List { "Orders", "Categories" } + }, + @"C:\ravendb-exports\Northwind.ravendbdump", + token); + +await importOperation.WaitForCompletionAsync(); +``` - - -{`var id = this['@metadata']['@id']; + + +
+ +See [ImportAsync](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. + + + + + +`TransformScript` lets you modify or filter out each document during an export or import, using JavaScript that you provide. + +The JavaScript engine is the same one used for [patching operations](../../../client-api/operations/patching/single-document.mdx), so you have identical syntax and capabilities, plus the ability to filter out a document by throwing a `skip` exception. + +```javascript +var id = this['@metadata']['@id']; if (id === 'orders/999-A') throw 'skip'; // filter-out this.Freight = 15.3; -`} - +``` + + + + + +## Methods + + + +### Smuggler methods + + + + +Switches Smuggler to operate on a database other than the document store's default database. + +```csharp +public DatabaseSmuggler ForDatabase(string databaseName); +``` + +
+ +Usage: + +```csharp +var northwindSmuggler = store.Smuggler.ForDatabase("Northwind"); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **databaseName** | `string` | The database that Smuggler will operate on. | + +| Return value | | +|-----------|-------------| +| `DatabaseSmuggler` | A Smuggler instance scoped to the specified database. | + +
+ + +Exports the selected data to a file, to a stream, or to another database. + +```csharp +Task ExportAsync( + DatabaseSmugglerExportOptions options, + string toFile, + CancellationToken token = default); + +Task ExportAsync( + DatabaseSmugglerExportOptions options, + Stream toStream, + CancellationToken token = default); + +Task ExportAsync( + DatabaseSmugglerExportOptions options, + DatabaseSmuggler toDatabase, + CancellationToken token = default); +``` + +
+ +Usage: + +```csharp +var exportOperation = await store.Smuggler.ExportAsync(options, toFile, token); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerExportOptions` | Options used during the export. See [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | +| **toFile** | `string` | Path to the file where exported data is written. | +| **toStream** | `Stream` | Stream that exported data is written to. | +| **toDatabase** | `DatabaseSmuggler` | A `DatabaseSmuggler` instance used as the destination. | +| **token** | `CancellationToken` | Token used to cancel the operation. | + +| Return value | | +|-----------|-------------| +| `Operation` | An `Operation` instance you can use to wait for completion and subscribe to progress events. | +
+ + +Imports data from a file or a stream into the database. + +```csharp +Task ImportAsync( + DatabaseSmugglerImportOptions options, + string fromFile, + CancellationToken token = default); + +Task ImportAsync( + DatabaseSmugglerImportOptions options, + Stream stream, + CancellationToken token = default); +``` + +
+ +Usage: + +```csharp +var importOperation = await store.Smuggler.ImportAsync(options, fromFile, token); +``` +
+| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerImportOptions` | Options used during the import. See [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | +| **fromFile** | `string` | Path to the file the data is imported from. | +| **stream** | `Stream` | Stream with the data to import. | +| **token** | `CancellationToken` | Token used to cancel the operation. | + +| Return value | | +|-----------|-------------| +| `Operation` | An `Operation` instance you can use to wait for completion and subscribe to progress events. | + +
+
+ +
+ +## Classes + + + +### Smuggler options classes + +
+ + + + + +**Options that control what an export includes.** + +```csharp +class DatabaseSmugglerExportOptions +{ + DatabaseItemType OperateOnTypes + DatabaseRecordItemType OperateOnDatabaseRecordTypes + bool IncludeExpired + bool IncludeArtificial + bool IncludeArchived + bool RemoveAnalyzers + string TransformScript + int MaxStepsForTransformScript + string EncryptionKey + List Collections + int? MaxReadOpsPerSecond + bool SkipCorruptedData + ExportCompressionAlgorithm? CompressionAlgorithm +} +``` + +
+ +| Property | Type | Description | +|-----------|------|-------------| +| **Collections** | `List` | List of specific collections to export. If empty, all collections are exported.
Default: `empty` | +| **OperateOnTypes** | `DatabaseItemType` | Indicates what should be exported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `ReplicationHubCertificates`, `Identities`, `CompareExchange`, `Attachments`, `CounterGroups`, `Subscriptions`, `TimeSeries`, `TimeSeriesDeletedRanges` | +| **OperateOnDatabaseRecordTypes** | `DatabaseRecordItemType` | Indicates what should be exported from the database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `Settings`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications`, `TimeSeries`, `DocumentsCompression`, `Analyzers`, `LockMode`, `OlapConnectionStrings`, `OlapEtls`, `ElasticSearchConnectionStrings`, `ElasticSearchEtls`, `PostgreSQLIntegration`, `QueueConnectionStrings`, `QueueEtls`, `IndexesHistory`, `Refresh`, `DataArchival`, `QueueSinks` | +| **IncludeExpired** | `bool` | Should expired documents be exported.
Default: `true` | +| **IncludeArtificial** | `bool` | Should artificial documents be exported.
Default: `false` | +| **IncludeArchived** | `bool` | Should archived documents be exported.
Default: `true` | +| **RemoveAnalyzers** | `bool` | Should analyzers be removed from indexes.
Default: `false` | +| **TransformScript** | `string` | JavaScript applied to every exported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **MaxStepsForTransformScript** | `int` | Maximum number of steps the transform script can run before failing.
Default: `10000` | +| **EncryptionKey** | `string` | Encryption key used to encrypt the exported file.
Default: `null` | +| **MaxReadOpsPerSecond** | `int?` | Limits the read-operation rate during the export, to throttle the load placed on the database.
Default: `null` (no limit) | +| **SkipCorruptedData** | `bool` | When `true`, the export continues if corrupted data is encountered (for example, lost compression dictionaries), recording an error entry in the result report instead of stopping.
Default: `false` | +| **CompressionAlgorithm** | `ExportCompressionAlgorithm?` | Compression algorithm used for the exported file: `Zstd` or `Gzip`.
Default: `null` | + +
+ + +**Options that control what an import includes.** + +```csharp +class DatabaseSmugglerImportOptions +{ + DatabaseItemType OperateOnTypes + DatabaseRecordItemType OperateOnDatabaseRecordTypes + bool IncludeExpired + bool IncludeArtificial + bool IncludeArchived + bool RemoveAnalyzers + string TransformScript + int MaxStepsForTransformScript + string EncryptionKey + List Collections + int? MaxReadOpsPerSecond + bool SkipCorruptedData + bool SkipRevisionCreation +} +``` + +
+ +| Property | Type | Description | +|-----------|------|-------------| +| **Collections** | `List` | List of specific collections to import. If empty, all collections are imported.
Default: `empty` | +| **OperateOnTypes** | `DatabaseItemType` | Indicates what should be imported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `ReplicationHubCertificates`, `Identities`, `CompareExchange`, `Attachments`, `CounterGroups`, `Subscriptions`, `TimeSeries`, `TimeSeriesDeletedRanges` | +| **OperateOnDatabaseRecordTypes** | `DatabaseRecordItemType` | Indicates what should be imported into the database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `Settings`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications`, `TimeSeries`, `DocumentsCompression`, `Analyzers`, `LockMode`, `OlapConnectionStrings`, `OlapEtls`, `ElasticSearchConnectionStrings`, `ElasticSearchEtls`, `PostgreSQLIntegration`, `QueueConnectionStrings`, `QueueEtls`, `IndexesHistory`, `Refresh`, `DataArchival`, `QueueSinks` | +| **IncludeExpired** | `bool` | Should expired documents be imported.
Default: `true` | +| **IncludeArtificial** | `bool` | Should artificial documents be imported.
Default: `false` | +| **IncludeArchived** | `bool` | Should archived documents be imported.
Default: `true` | +| **RemoveAnalyzers** | `bool` | Should analyzers be removed from indexes.
Default: `false` | +| **TransformScript** | `string` | JavaScript applied to every imported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **MaxStepsForTransformScript** | `int` | Maximum number of steps the transform script can run before failing.
Default: `10000` | +| **EncryptionKey** | `string` | Encryption key used to read an encrypted `.ravendbdump` file.
Default: `null` | +| **MaxReadOpsPerSecond** | `int?` | Limits the read-operation rate during the import, to throttle the load placed on the database.
Default: `null` (no limit) | +| **SkipCorruptedData** | `bool` | When `true`, the import continues if corrupted data is encountered, recording an error entry in the result report instead of stopping.
Default: `false` | +| **SkipRevisionCreation** | `bool` | When `true`, revisions are not created for the imported documents.
Default: `false` | + +
+
+ + diff --git a/versioned_docs/version-6.2/client-api/smuggler/content/_what-is-smuggler-java.mdx b/versioned_docs/version-6.2/client-api/smuggler/content/_what-is-smuggler-java.mdx index dd4f20fd0c..d604d4267a 100644 --- a/versioned_docs/version-6.2/client-api/smuggler/content/_what-is-smuggler-java.mdx +++ b/versioned_docs/version-6.2/client-api/smuggler/content/_what-is-smuggler-java.mdx @@ -1,137 +1,312 @@ import Admonition from '@theme/Admonition'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -import CodeBlock from '@theme/CodeBlock'; +import Panel from "@site/src/components/Panel"; +import ContentFrame from "@site/src/components/ContentFrame"; -Smuggler gives you the ability to export or import data from or to a database using JSON format. -It is exposed via the `DocumentStore.smuggler()`. + -## ForDatabase +* **Smuggler** is a tool for moving data between databases. + Use it to export documents, indexes, and other items from a database as JSON, + and to import that data into another database. -By default, the `IDocumentStore.smuggler` works on the default document store database from the `IDocumentStore.database` property. +* Smuggler is exposed through the `DocumentStore.smuggler()` method. -In order to switch it to a different database use the `.forDatabase` method. +* An export can write to a `.ravendbdump` file or stream directly into another database. + An import reads that data back into a database. - - -{`DatabaseSmuggler northwindSmuggler = store.smuggler().forDatabase("Northwind"); -`} - - +* You can narrow what is transferred by collection, by item type, + and by applying a transform script to each document. + +* In this article: + * [ForDatabase](../../../client-api/smuggler/what-is-smuggler.mdx#fordatabase) + * [Export](../../../client-api/smuggler/what-is-smuggler.mdx#export) + * [Export specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#export-specific-collections) + * [Import](../../../client-api/smuggler/what-is-smuggler.mdx#import) + * [Import specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#import-specific-collections) + * [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript) + * [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) + * [Methods](../../../client-api/smuggler/what-is-smuggler.mdx#methods) + * [Classes](../../../client-api/smuggler/what-is-smuggler.mdx#classes) + + + + + +By default, the smuggler works on the database set in the `DocumentStore.getDatabase()` property. +To run the smuggler against a different database, switch to it with the `forDatabase` method. + +```java +DatabaseSmuggler northwindSmuggler = store + .smuggler() + .forDatabase("Northwind"); +``` + + + + + +Export writes the selected data to a `.ravendbdump` file, or streams it directly into another database. +Use `DatabaseSmugglerExportOptions` to choose what to include. + +
+ +#### Example: export indexes and documents to a file + +```java +// export only Indexes and Documents to a given file +DatabaseSmugglerExportOptions options = new DatabaseSmugglerExportOptions(); +options.setOperateOnTypes(EnumSet.of( + DatabaseItemType.INDEXES, + DatabaseItemType.DOCUMENTS)); + +Operation exportOperation = store + .smuggler() + .exportAsync(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + +exportOperation.waitForCompletion(); +``` + +
+ + + +### Export specific collections + +By default, all collections are exported. +To export only specific collections, list their names in the `collections` option. + +```java +// Export only the "Orders" and "Categories" collections to a file +DatabaseSmugglerExportOptions options = new DatabaseSmugglerExportOptions(); +options.setCollections(Arrays.asList("Orders", "Categories")); + +Operation exportOperation = store + .smuggler() + .exportAsync(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + +exportOperation.waitForCompletion(); +``` + + + +
+ +See [exportAsync](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. + + + + + +Import reads data from a `.ravendbdump` file or a stream and writes it into the database. +Use `DatabaseSmugglerImportOptions` to choose what to include. + +
+ +#### Example: import documents from a file + +```java +// import only Documents from a given file +DatabaseSmugglerImportOptions options = new DatabaseSmugglerImportOptions(); +options.setOperateOnTypes(EnumSet.of(DatabaseItemType.DOCUMENTS)); + +// import the .ravendbdump file that you exported (i.e. in the export example above) +Operation importOperation = store + .smuggler() + .importAsync(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + +importOperation.waitForCompletion(); +``` + +
+ + + +### Import specific collections + +By default, all collections are imported. +To import only specific collections, list their names in the `collections` option. + +```java +// Import only the "Orders" and "Categories" collections from the file +DatabaseSmugglerImportOptions options = new DatabaseSmugglerImportOptions(); +options.setCollections(Arrays.asList("Orders", "Categories")); + +Operation importOperation = store + .smuggler() + .importAsync(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + +importOperation.waitForCompletion(); +``` + + +
+See [importAsync](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. -## Export + -### Syntax + - - -{`//export +`TransformScript` lets you modify or filter out each document during an export or import, using JavaScript that you provide. + +The JavaScript engine is the same one used for [patching operations](../../../client-api/operations/patching/single-document.mdx), so you have identical syntax and capabilities, plus the ability to filter out a document by throwing a `skip` exception. + +```javascript +var id = this['@metadata']['@id']; +if (id === 'orders/999-A') + throw 'skip'; // filter-out + +this.Freight = 15.3; +``` + + + + + +## Methods + + + +### Smuggler methods + + + + +Switches the smuggler to operate on a database other than the document store's default database. + +```java +public DatabaseSmuggler forDatabase(String databaseName); +``` + +
+ +Usage: + +```java +DatabaseSmuggler northwindSmuggler = store.smuggler().forDatabase("Northwind"); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **databaseName** | `String` | The database that the smuggler will operate on. | + +| Return value | | +|-----------|-------------| +| `DatabaseSmuggler` | A smuggler instance scoped to the specified database. | + +
+ + +Exports the selected data to a file or to another database. + +```java public Operation exportAsync(DatabaseSmugglerExportOptions options, String toFile); public Operation exportAsync(DatabaseSmugglerExportOptions options, DatabaseSmuggler toDatabase); -`} - - +``` -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerExportOptions` | Options that will be used during the export. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | -| **toDatabase** | `DatabaseSmuggler` | `DatabaseSmuggler` instance used as a destination | -| **toFile** | `String` | Path to a file where exported data will be written | - -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | - -### DatabaseSmugglerExportOptions - -| Parameters | | | -| ------------- | ------------- | ----- | -| **Collections** | `List` | List of specific collections to export. If empty, then all collections will be exported. Default: `empty` | -| **operateOnTypes** | `DatabaseItemType` | Indicates what should be exported. Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `Identities`, `CompareExchange`, `Subscriptions` | -| **operateOnDatabaseRecordType** | `DatabaseRecordItemType` | Indicates what should be exported from database record. Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications` | -| **includeExpired** | `boolean` | Should expired documents be included in the export. Default: `true` | -| **removeAnalyzers** | `boolean` | Should analyzers be removed from Indexes. Default: `false` | -| **transformScript** | `String` | JavaScript-based script applied to every exported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **maxStepsForTransformScript** | `int` | Maximum number of steps that transform script can process before failing. Default: 10000 | - -### Example - - - -{`// export only Indexes and Documents to a given file -Operation exportOperation = store.smuggler().exportAsync(exportOptions, "C:\\\\ravendb-exports\\\\Northwind.ravendbdump"); -`} - - +
+Usage: +```java +Operation exportOperation = store.smuggler().exportAsync(options, toFile); +``` -## Import +
-### Syntax +| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerExportOptions` | Options used during the export. See [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | +| **toFile** | `String` | Path to the file where exported data is written. | +| **toDatabase** | `DatabaseSmuggler` | A `DatabaseSmuggler` instance used as the destination. | + +| Return value | | +|-----------|-------------| +| `Operation` | An `Operation` instance you can use to wait for completion and subscribe to progress events. | - - -{`public Operation importAsync(DatabaseSmugglerImportOptions options, String fromFile); -public Operation importAsync(DatabaseSmugglerImportOptions options, InputStream stream); -`} - + -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerImportOptions` | Options that will be used during the import. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | -| **stream** | `InputStream` | Stream with data to import | -| **fromFile** | `String` | Path to a file from which data will be imported | +Imports data from a file or a stream into the database. -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation-class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | +```java +public Operation importAsync(DatabaseSmugglerImportOptions options, String fromFile); + +public Operation importAsync(DatabaseSmugglerImportOptions options, InputStream stream); +``` -### DatabaseSmugglerImportOptions +
-| Parameters | | | -| - | - | - | -| **Collections** | `List` | List of specific collections to import. If empty, then all collections will be exported. Default: `empty` | -| **operateOnTypes** | `DatabaseItemType` | Indicates what should be imported. Default: `INDEXES`, `DOCUMENTS`, `REVISION_DOCUMENTS`, `CONFLICTS`, `DATABASE_RECORD`, `IDENTITIES`, `COMPARE_EXCHANGE`, `SUBSCRIPTIONS` | -| **operateOnDatabaseRecordType** | `DatabaseRecordItemType` | Indicates what should be imported. Default: `CLIENT`, `CONFLICT_SOLVER_CONFIG`, `EXPIRATION`, `EXTERNAL_REPLICATIONS`, `PERIODIC_BACKUPS`, `RAVEN_CONNECTION_STRINGS`, `RAVEN_ETLS`, `REVISIONS`, `SQL_CONNECTION_STRINGS`, `SORTERS`, `SQL_ETLS`, `HUB_PULL_REPLICATIONS`, `SINK_PULL_REPLICATIONS` | -| **includeExpired** | `boolean` | Should expired documents be included in the import. Default: `true` | -| **removeAnalyzers** | `boolean` | Should analyzers be removed from Indexes. Default: `false` | -| **transformScript** | `String` | JavaScript-based script applied to every exported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **maxStepsForTransformScript** | `int` | Maximum number of steps that transform script can process before failing. Default: 10000 | +Usage: +```java +Operation importOperation = store.smuggler().importAsync(options, fromFile); +``` -### Example +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerImportOptions` | Options used during the import. See [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | +| **fromFile** | `String` | Path to the file the data is imported from. | +| **stream** | `InputStream` | Stream with the data to import. | + +| Return value | | +|-----------|-------------| +| `Operation` | An `Operation` instance you can use to wait for completion and subscribe to progress events. | - - -{`Operation importOperation = store.smuggler().importAsync(importOptions, "C:\\\\ravendb-exports\\\\Northwind.ravendbdump"); -`} - +
+
+## Classes -## TransformScript + -`TransformScript` exposes the ability to modify or even filter-out the document during the import and export process using the provided JavaScript. +### Smuggler options classes -Underneath the JavaScript engine is exactly the same as used for [patching operations](../../../client-api/operations/patching/single-document.mdx) giving you identical syntax and capabilities with additional **ability to filter out documents by throwing a 'skip' exception**. +
+ - - -{`var id = this['@metadata']['@id']; -if (id === 'orders/999-A') - throw 'skip'; // filter-out + + + +**Options that control what an export includes.** + +| Property | Type | Description | +|-----------|------|-------------| +| **collections** | `List` | List of specific collections to export. If empty, all collections are exported.
Default: `empty` | +| **operateOnTypes** | `DatabaseItemType` | Indicates what should be exported.
Default: `INDEXES`, `DOCUMENTS`, `REVISION_DOCUMENTS`, `CONFLICTS`, `DATABASE_RECORD`, `IDENTITIES`, `COMPARE_EXCHANGE`, `SUBSCRIPTIONS` | +| **operateOnDatabaseRecordType** | `DatabaseRecordItemType` | Indicates what should be exported from the database record.
Default: `CLIENT`, `CONFLICT_SOLVER_CONFIG`, `EXPIRATION`, `EXTERNAL_REPLICATIONS`, `PERIODIC_BACKUPS`, `RAVEN_CONNECTION_STRINGS`, `RAVEN_ETLS`, `REVISIONS`, `SQL_CONNECTION_STRINGS`, `SORTERS`, `SQL_ETLS`, `HUB_PULL_REPLICATIONS`, `SINK_PULL_REPLICATIONS` | +| **includeExpired** | `boolean` | Should expired documents be exported.
Default: `true` | +| **removeAnalyzers** | `boolean` | Should analyzers be removed from indexes.
Default: `false` | +| **transformScript** | `String` | JavaScript applied to every exported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **maxStepsForTransformScript** | `int` | Maximum number of steps the transform script can run before failing.
Default: `10000` | -this.Freight = 15.3; -`} -
+ +**Options that control what an import includes.** +| Property | Type | Description | +|-----------|------|-------------| +| **collections** | `List` | List of specific collections to import. If empty, all collections are imported.
Default: `empty` | +| **operateOnTypes** | `DatabaseItemType` | Indicates what should be imported.
Default: `INDEXES`, `DOCUMENTS`, `REVISION_DOCUMENTS`, `CONFLICTS`, `DATABASE_RECORD`, `IDENTITIES`, `COMPARE_EXCHANGE`, `SUBSCRIPTIONS` | +| **operateOnDatabaseRecordType** | `DatabaseRecordItemType` | Indicates what should be imported into the database record.
Default: `CLIENT`, `CONFLICT_SOLVER_CONFIG`, `EXPIRATION`, `EXTERNAL_REPLICATIONS`, `PERIODIC_BACKUPS`, `RAVEN_CONNECTION_STRINGS`, `RAVEN_ETLS`, `REVISIONS`, `SQL_CONNECTION_STRINGS`, `SORTERS`, `SQL_ETLS`, `HUB_PULL_REPLICATIONS`, `SINK_PULL_REPLICATIONS` | +| **includeExpired** | `boolean` | Should expired documents be imported.
Default: `true` | +| **removeAnalyzers** | `boolean` | Should analyzers be removed from indexes.
Default: `false` | +| **transformScript** | `String` | JavaScript applied to every imported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **maxStepsForTransformScript** | `int` | Maximum number of steps the transform script can run before failing.
Default: `10000` | + +
+ + + diff --git a/versioned_docs/version-6.2/client-api/smuggler/content/_what-is-smuggler-nodejs.mdx b/versioned_docs/version-6.2/client-api/smuggler/content/_what-is-smuggler-nodejs.mdx index ca9f9b333d..a23f3586d1 100644 --- a/versioned_docs/version-6.2/client-api/smuggler/content/_what-is-smuggler-nodejs.mdx +++ b/versioned_docs/version-6.2/client-api/smuggler/content/_what-is-smuggler-nodejs.mdx @@ -1,150 +1,308 @@ import Admonition from '@theme/Admonition'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -import CodeBlock from '@theme/CodeBlock'; +import Panel from "@site/src/components/Panel"; +import ContentFrame from "@site/src/components/ContentFrame"; -Smuggler gives you the ability to export or import data from or to a database using JSON format. -It is exposed via the `DocumentStore.smuggler`. + -## ForDatabase +* **Smuggler** is a tool for moving data between databases. + Use it to export documents, indexes, and other items from a database as JSON, + and to import that data into another database. -By default, the `DocumentStore.smuggler` works on the default document store database from the `DocumentStore.database` . +* Smuggler is exposed through the `documentStore.smuggler` property. -In order to switch it to a different database use the `.forDatabase` method. +* An export can write to a `.ravendbdump` file. + An import reads that data back into a database. - - -{`const northwindSmuggler = store +* You can narrow what is transferred by collection, by item type, + and by applying a transform script to each document. + +* In this article: + * [ForDatabase](../../../client-api/smuggler/what-is-smuggler.mdx#fordatabase) + * [Export](../../../client-api/smuggler/what-is-smuggler.mdx#export) + * [Export specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#export-specific-collections) + * [Import](../../../client-api/smuggler/what-is-smuggler.mdx#import) + * [Import specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#import-specific-collections) + * [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript) + * [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) + * [Methods](../../../client-api/smuggler/what-is-smuggler.mdx#methods) + * [Classes](../../../client-api/smuggler/what-is-smuggler.mdx#classes) + + + + + +By default, the smuggler works on the database set in the `documentStore.database` property. +To run the smuggler against a different database, switch to it with the `forDatabase` method. + +```js +const northwindSmuggler = store .smuggler .forDatabase("Northwind"); -`} - - +``` + + -## Export +Export writes the selected data to a `.ravendbdump` file. +Use `DatabaseSmugglerExportOptions` to choose what to include. -### Usage +
- - -{`const operation = await store.smuggler.export(options, toDatabase); +#### Example: export indexes and documents to a file -const operation = await store.smuggler.export(options, toFile); -`} - - +```js +// export only Indexes and Documents to a given file +const options = new DatabaseSmugglerExportOptions(); +options.operateOnTypes = ["Indexes", "Documents"]; -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerExportOptions` | Options that will be used during the export. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | -| **toDatabase** | `DatabaseSmuggler` | `DatabaseSmuggler` instance used as a destination | -| **toFile** | `string` | Path to a file where exported data will be written | - -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | - -### DatabaseSmugglerExportOptions - -| Parameters | | | -| - | - | - | -| **collections** | ` string[]` | List of specific collections to export. If empty, then all collections will be exported. Default: `empty` | -| **operateOnTypes** | `DatabaseItemType[]` | Indicates what should be exported. Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `Identities`, `CompareExchange`, `Subscriptions` | -| **operateOnDatabaseRecordTypes** | `DatabaseRecordItemType[]` | Indicates what should be exported from database record. Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications` | -| **includeExpired** | `boolean` | Should expired documents be included in the export. Default: `true` | -| **includeArtificial** | `boolean` | ? | -| **removeAnalyzers** | `boolean` | Should analyzers be removed from Indexes. Default: `false` | -| **transformScript** | `string` | JavaScript-based script applied to every exported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **maxStepsForTransformScript** | `number` | Maximum number of steps that transform script can process before failing. Default: 10000 | -| **skipRevisionCreation** | `boolean` | skip revision creation | -| **encryptionKey** | `string` | Encryption key used for restore | - -### Example - - - -{`const options = new DatabaseSmugglerExportOptions(); -options.operateOnTypes = ["Documents"]; const operation = await store .smuggler - .export(options, "C:\\\\ravendb-exports\\\\Northwind.ravendbdump"); + .export(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + await operation.waitForCompletion(); -`} - - +``` +
+ -## Import +### Export specific collections -### Usage +By default, all collections are exported. +To export only specific collections, list their names in the `collections` option. - - -{`const operation = await store.smuggler.import(options, fromFile); +```js +// Export only the "Orders" and "Categories" collections to a file +const options = new DatabaseSmugglerExportOptions(); +options.collections = ["Orders", "Categories"]; -const operation = await store.smuggler.import(options, stream); -`} - - +const operation = await store + .smuggler + .export(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + +await operation.waitForCompletion(); +``` + + + +
+ +See [export](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. + + + + + +Import reads data from a `.ravendbdump` file and writes it into the database. +Use `DatabaseSmugglerImportOptions` to choose what to include. + +
-| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerImportOptions` | Options that will be used during the import. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | -| **stream** | `Stream` | Stream with data to import | -| **fromFile** | `string` | Path to a file from which data will be imported | - -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation-class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | - -### DatabaseSmugglerImportOptions - -| Parameters | | | -| - | - | - | -| **operateOnTypes** | `DatabaseItemType[]` | Indicates what should be imported. Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `Identities`, `CompareExchange`, `Subscriptions` | -| **operateOnDatabaseRecordTypes** | `DatabaseRecordItemType[]` | Indicates what should be imported. Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications` | -| **includeExpired** | `boolean` | Should expired documents be included in the import. Default: `true` | -| **includeArtificial** | `boolean` | ? | -| **removeAnalyzers** | `boolean` | Should analyzers be removed from Indexes. Default: `false` | -| **transformScript** | `string` | JavaScript-based script applied to every imported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **maxStepsForTransformScript** | `number` | Maximum number of steps that transform script can process before failing. Default: 10000 | -| **skipRevisionCreation** | `boolean` | skip revision creation | -| **encryptionKey** | `string` | Encryption key used for restore | - -### Example - - - -{`const options = new DatabaseSmugglerImportOptions(); +#### Example: import documents from a file + +```js +// import only Documents from a given file +const options = new DatabaseSmugglerImportOptions(); options.operateOnTypes = ["Documents"]; -const operation = await store.smuggler.import(options, "C:\\\\ravendb-exports\\\\Northwind.ravendbdump"); + +// import the .ravendbdump file that you exported (i.e. in the export example above) +const operation = await store + .smuggler + .import(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + await operation.waitForCompletion(); -`} - - +``` +
+ -## TransformScript +### Import specific collections -`TransformScript` exposes the ability to modify or even filter-out the document during the import and export process using the provided JavaScript. +By default, all collections are imported. +To import only specific collections, list their names in the `collections` option. + +```js +// Import only the "Orders" and "Categories" collections from the file +const options = new DatabaseSmugglerImportOptions(); +options.collections = ["Orders", "Categories"]; + +const operation = await store + .smuggler + .import(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + +await operation.waitForCompletion(); +``` -Underneath the JavaScript engine is exactly the same as used for [patching operations](../../../client-api/operations/patching/single-document.mdx) giving you identical syntax and capabilities with additional **ability to filter out documents by throwing a 'skip' exception**. + - - -{`var id = this['@metadata']['@id']; +
+ +See [import](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. + + + + + +`TransformScript` lets you modify or filter out each document during an export or import, using JavaScript that you provide. + +The JavaScript engine is the same one used for [patching operations](../../../client-api/operations/patching/single-document.mdx), so you have identical syntax and capabilities, plus the ability to filter out a document by throwing a `skip` exception. + +```javascript +var id = this['@metadata']['@id']; if (id === 'orders/999-A') throw 'skip'; // filter-out this.Freight = 15.3; -`} -
+``` + + + + + +## Methods + + + +### Smuggler methods + + + + +Switches the smuggler to operate on a database other than the document store's default database. + +```ts +forDatabase(databaseName: string): DatabaseSmuggler; +``` + +
+ +Usage: + +```js +const northwindSmuggler = store.smuggler.forDatabase("Northwind"); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **databaseName** | `string` | The database that the smuggler will operate on. | + +| Return value | | +|-----------|-------------| +| `DatabaseSmuggler` | A smuggler instance scoped to the specified database. | +
+ + +Exports the selected data to a file. + +```ts +export(options: DatabaseSmugglerExportOptions, toFile: string): Promise; +``` + +
+ +Usage: + +```js +const operation = await store.smuggler.export(options, toFile); +await operation.waitForCompletion(); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerExportOptions` | Options used during the export. See [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | +| **toFile** | `string` | Path to the file where exported data is written. | + +| Return value | | +|-----------|-------------| +| `Promise` | An awaiter you can use to wait for the operation to complete and subscribe to progress events. | + +
+ + +Imports data from a file into the database. +```ts +import(options: DatabaseSmugglerImportOptions, fromFile: string): Promise; +``` +
+ +Usage: + +```js +const operation = await store.smuggler.import(options, fromFile); +await operation.waitForCompletion(); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerImportOptions` | Options used during the import. See [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | +| **fromFile** | `string` | Path to the file the data is imported from. | + +| Return value | | +|-----------|-------------| +| `Promise` | An awaiter you can use to wait for the operation to complete and subscribe to progress events. | + +
+
+ +
+ +## Classes + + + +### Smuggler options classes + +
+ + + + + +**Options that control what an export includes.** + +| Property | Type | Description | +|-----------|------|-------------| +| **collections** | `string[]` | List of specific collections to export. If empty, all collections are exported.
Default: `empty` | +| **operateOnTypes** | `DatabaseItemType[]` | Indicates what should be exported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `Identities`, `CompareExchange`, `Subscriptions` | +| **operateOnDatabaseRecordType** | `DatabaseRecordItemType[]` | Indicates what should be exported from the database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications` | +| **includeExpired** | `boolean` | Should expired documents be exported.
Default: `true` | +| **includeArtificial** | `boolean` | Should artificial documents be exported.
Default: `false` | +| **removeAnalyzers** | `boolean` | Should analyzers be removed from indexes.
Default: `false` | +| **transformScript** | `string` | JavaScript applied to every exported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **maxStepsForTransformScript** | `number` | Maximum number of steps the transform script can run before failing.
Default: `10000` | + +
+ + +**Options that control what an import includes.** + +| Property | Type | Description | +|-----------|------|-------------| +| **collections** | `string[]` | List of specific collections to import. If empty, all collections are imported.
Default: `empty` | +| **operateOnTypes** | `DatabaseItemType[]` | Indicates what should be imported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `Identities`, `CompareExchange`, `Subscriptions` | +| **operateOnDatabaseRecordType** | `DatabaseRecordItemType[]` | Indicates what should be imported into the database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications` | +| **includeExpired** | `boolean` | Should expired documents be imported.
Default: `true` | +| **includeArtificial** | `boolean` | Should artificial documents be imported.
Default: `false` | +| **removeAnalyzers** | `boolean` | Should analyzers be removed from indexes.
Default: `false` | +| **transformScript** | `string` | JavaScript applied to every imported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **maxStepsForTransformScript** | `number` | Maximum number of steps the transform script can run before failing.
Default: `10000` | + +
+
+ + diff --git a/versioned_docs/version-7.0/client-api/smuggler/content/_what-is-smuggler-csharp.mdx b/versioned_docs/version-7.0/client-api/smuggler/content/_what-is-smuggler-csharp.mdx index 4bdf46a8fc..b0be29d8b6 100644 --- a/versioned_docs/version-7.0/client-api/smuggler/content/_what-is-smuggler-csharp.mdx +++ b/versioned_docs/version-7.0/client-api/smuggler/content/_what-is-smuggler-csharp.mdx @@ -1,179 +1,397 @@ import Admonition from '@theme/Admonition'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -import CodeBlock from '@theme/CodeBlock'; +import Panel from "@site/src/components/Panel"; +import ContentFrame from "@site/src/components/ContentFrame"; -Smuggler gives you the ability to export or import data from or to a database using JSON format. -It is exposed via the `DocumentStore.Smuggler` property. + -## ForDatabase +* **Smuggler** is a tool for moving data between databases. + Use it to export documents, indexes, and other items from a database as JSON, + and to import that data into another database. -By default, the `DocumentStore.Smuggler` works on the default document store database from the `DocumentStore.Database` property. +* Smuggler is exposed through the `DocumentStore.Smuggler` property. -In order to switch it to a different database use the `.ForDatabase` method. +* An export can write to a `.ravendbdump` file or stream directly into another database. + An import reads that data back into a database. - - -{`var northwindSmuggler = store - .Smuggler - .ForDatabase("Northwind"); -`} - - +* You can narrow what is transferred by collection, by item type, + and by applying a transform script to each document. +* In this article: + * [ForDatabase](../../../client-api/smuggler/what-is-smuggler.mdx#fordatabase) + * [Export](../../../client-api/smuggler/what-is-smuggler.mdx#export) + * [Export specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#export-specific-collections) + * [Import](../../../client-api/smuggler/what-is-smuggler.mdx#import) + * [Import specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#import-specific-collections) + * [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript) + * [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) + * [Methods](../../../client-api/smuggler/what-is-smuggler.mdx#methods) + * [Classes](../../../client-api/smuggler/what-is-smuggler.mdx#classes) + -## Export + -### Syntax +By default, `DocumentStore.Smuggler` works on the database set in the `DocumentStore.Database` property. +To run Smuggler against a different database, switch to it with the `ForDatabase` method. - - -{`Task ExportAsync( - DatabaseSmugglerExportOptions options, - DatabaseSmuggler toDatabase, - CancellationToken token = default(CancellationToken)); - -Task ExportAsync( - DatabaseSmugglerExportOptions options, - string toFile, - CancellationToken token = default(CancellationToken)); -`} - - +```csharp +var northwindSmuggler = store + .Smuggler + .ForDatabase("Northwind"); +``` -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerExportOptions` | Options that will be used during the export. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | -| **toDatabase** | `DatabaseSmuggler` | `DatabaseSmuggler` instance used as a destination | -| **toFile** | `string` | Path to a file where exported data will be written | -| **token** | `CancellationToken` | Token used to cancel the operation | + -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | + -### DatabaseSmugglerExportOptions +Export writes the selected data to a `.ravendbdump` file, or streams it directly into another database. +Use `DatabaseSmugglerExportOptions` to choose what to include. -| Parameters | | | -| ------------- | ------------- | ----- | -| **Collections** | `List` | List of specific collections to export. If empty, then all collections will be exported.
Default: `empty` | -| **OperateOnTypes** | `DatabaseItemType` | Indicates what should be exported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `ReplicationHubCertificates`, `Identities`, `CompareExchange`, `Attachments`, `CounterGroups`, `Subscriptions`, `TimeSeries` | -| **OperateOnDatabaseRecordTypes** | `DatabaseRecordItemType` | Indicates what should be exported from database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `Settings`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications`, `TimeSeries`, `DocumentsCompression`, `Analyzers`, `LockMode`, `OlapConnectionStrings`, `OlapEtls`, `ElasticSearchConnectionStrings`, `ElasticSearchEtls`, `PostgreSQLIntegration`, `QueueConnectionStrings`, `QueueEtls`, `IndexesHistory`, `Refresh`, `DataArchival` | -| **IncludeExpired** | `bool` | Should expired documents be exported.
Default: `true` | -| **IncludeArtificial** | `bool` | Should artificial documents be exported.
Default: `false` | -| **IncludeArchived** | `bool` | Should archived documents be exported.
Default: `true` | -| **RemoveAnalyzers** | `bool` | Should analyzers be removed from Indexes.
Default: `false` | -| **TransformScript** | `string` | JavaScript-based script applied to every exported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **MaxStepsForTransformScript** | `int` | Maximum number of steps that the transform script can process before failing.
Default: **10000** | +
-### Example +#### Example: export indexes and documents to a file - - -{`// export only Indexes and Documents to a given file +```csharp +// export only Indexes and Documents to a given file var exportOperation = await store .Smuggler .ExportAsync( new DatabaseSmugglerExportOptions - \{ + { OperateOnTypes = DatabaseItemType.Indexes | DatabaseItemType.Documents - \}, - @"C:\\ravendb-exports\\Northwind.ravendbdump", + }, + @"C:\ravendb-exports\Northwind.ravendbdump", token); await exportOperation.WaitForCompletionAsync(); -`} - - +``` +
+ -## Import +### Export specific collections -### Syntax +By default, all collections are exported. +To export only specific collections, list their names in the `Collections` option. - - -{`Task ImportAsync( - DatabaseSmugglerImportOptions options, - Stream stream, - CancellationToken token = default(CancellationToken)); +```csharp +// Export only the "Orders" and "Categories" collections to a file +var exportOperation = await store + .Smuggler + .ExportAsync( + new DatabaseSmugglerExportOptions + { + Collections = new List { "Orders", "Categories" } + }, + @"C:\ravendb-exports\Northwind.ravendbdump", + token); -Task ImportAsync( - DatabaseSmugglerImportOptions options, - string fromFile, - CancellationToken token = default(CancellationToken)); -`} - - +await exportOperation.WaitForCompletionAsync(); +``` -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerImportOptions` | Options that will be used during the import. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | -| **stream** | `Stream` | Stream with data to import | -| **fromFile** | `string` | Path to a file from which data will be imported | -| **token** | `CancellationToken` | Token used to cancel the operation | + -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation-class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | +
-### DatabaseSmugglerImportOptions +See [ExportAsync](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. -| Parameters | | | -| - | - | - | -| **Collections** | `List` | List of specific collections to import. If empty, then all collections will be imported.
Default: `empty` | -| **OperateOnTypes** | `DatabaseItemType` | Indicates what should be imported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `ReplicationHubCertificates`, `Identities`, `CompareExchange`, `Attachments`, `CounterGroups`, `Subscriptions`, `TimeSeries` | -| **OperateOnDatabaseRecordTypes** | `DatabaseRecordItemType` | Indicates what should be imported from database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `Settings`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications`, `TimeSeries`, `DocumentsCompression`, `Analyzers`, `LockMode`, `OlapConnectionStrings`, `OlapEtls`, `ElasticSearchConnectionStrings`, `ElasticSearchEtls`, `PostgreSQLIntegration`, `QueueConnectionStrings`, `QueueEtls`, `IndexesHistory`, `Refresh`, `DataArchival` | -| **IncludeExpired** | `bool` | Should expired documents be imported.
Default: `true` | -| **IncludeArtificial** | `bool` | Should artificial documents be imported.
Default: `false` | -| **IncludeArchived** | `bool` | Should archived documents be imported.
Default: `true` | -| **RemoveAnalyzers** | `bool` | Should analyzers be removed from Indexes.
Default: `false` | -| **TransformScript** | `string` | JavaScript-based script applied to every imported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **MaxStepsForTransformScript** | `int` | Maximum number of steps that the transform script can process before failing.
Default: **10000** | + + + -### Example +Import reads data from a `.ravendbdump` file or a stream and writes it into the database. +Use `DatabaseSmugglerImportOptions` to choose what to include. - - -{`// import only Documents from a given file +
+ +#### Example: import documents from a file + +```csharp +// import only Documents from a given file var importOperation = await store .Smuggler .ImportAsync( new DatabaseSmugglerImportOptions - \{ + { OperateOnTypes = DatabaseItemType.Documents - \}, + }, // import the .ravendbdump file that you exported (i.e. in the export example above) - @"C:\\ravendb-exports\\Northwind.ravendbdump", + @"C:\ravendb-exports\Northwind.ravendbdump", token); await importOperation.WaitForCompletionAsync(); -`} - - +``` +
+ -## TransformScript +### Import specific collections -`TransformScript` exposes the ability to modify or even filter-out the document during the import and export process using the provided JavaScript. +By default, all collections are imported. +To import only specific collections, list their names in the `Collections` option. -Underneath the JavaScript engine is exactly the same as used for [patching operations](../../../client-api/operations/patching/single-document.mdx) giving you identical syntax and capabilities with additional **ability to filter out documents by throwing a 'skip' exception**. +```csharp +// Import only the "Orders" and "Categories" collections from the file +var importOperation = await store + .Smuggler + .ImportAsync( + new DatabaseSmugglerImportOptions + { + Collections = new List { "Orders", "Categories" } + }, + @"C:\ravendb-exports\Northwind.ravendbdump", + token); + +await importOperation.WaitForCompletionAsync(); +``` - - -{`var id = this['@metadata']['@id']; + + +
+ +See [ImportAsync](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. + + + + + +`TransformScript` lets you modify or filter out each document during an export or import, using JavaScript that you provide. + +The JavaScript engine is the same one used for [patching operations](../../../client-api/operations/patching/single-document.mdx), so you have identical syntax and capabilities, plus the ability to filter out a document by throwing a `skip` exception. + +```javascript +var id = this['@metadata']['@id']; if (id === 'orders/999-A') throw 'skip'; // filter-out this.Freight = 15.3; -`} - +``` + + + + + +## Methods + + + +### Smuggler methods + + + + +Switches Smuggler to operate on a database other than the document store's default database. + +```csharp +public DatabaseSmuggler ForDatabase(string databaseName); +``` + +
+ +Usage: + +```csharp +var northwindSmuggler = store.Smuggler.ForDatabase("Northwind"); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **databaseName** | `string` | The database that Smuggler will operate on. | + +| Return value | | +|-----------|-------------| +| `DatabaseSmuggler` | A Smuggler instance scoped to the specified database. | + +
+ + +Exports the selected data to a file, to a stream, or to another database. + +```csharp +Task ExportAsync( + DatabaseSmugglerExportOptions options, + string toFile, + CancellationToken token = default); + +Task ExportAsync( + DatabaseSmugglerExportOptions options, + Stream toStream, + CancellationToken token = default); + +Task ExportAsync( + DatabaseSmugglerExportOptions options, + DatabaseSmuggler toDatabase, + CancellationToken token = default); +``` + +
+ +Usage: + +```csharp +var exportOperation = await store.Smuggler.ExportAsync(options, toFile, token); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerExportOptions` | Options used during the export. See [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | +| **toFile** | `string` | Path to the file where exported data is written. | +| **toStream** | `Stream` | Stream that exported data is written to. | +| **toDatabase** | `DatabaseSmuggler` | A `DatabaseSmuggler` instance used as the destination. | +| **token** | `CancellationToken` | Token used to cancel the operation. | + +| Return value | | +|-----------|-------------| +| `Operation` | An `Operation` instance you can use to wait for completion and subscribe to progress events. | +
+ + +Imports data from a file or a stream into the database. + +```csharp +Task ImportAsync( + DatabaseSmugglerImportOptions options, + string fromFile, + CancellationToken token = default); + +Task ImportAsync( + DatabaseSmugglerImportOptions options, + Stream stream, + CancellationToken token = default); +``` + +
+ +Usage: + +```csharp +var importOperation = await store.Smuggler.ImportAsync(options, fromFile, token); +``` +
+| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerImportOptions` | Options used during the import. See [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | +| **fromFile** | `string` | Path to the file the data is imported from. | +| **stream** | `Stream` | Stream with the data to import. | +| **token** | `CancellationToken` | Token used to cancel the operation. | + +| Return value | | +|-----------|-------------| +| `Operation` | An `Operation` instance you can use to wait for completion and subscribe to progress events. | + +
+
+ +
+ +## Classes + + + +### Smuggler options classes + +
+ + + + + +**Options that control what an export includes.** + +```csharp +class DatabaseSmugglerExportOptions +{ + DatabaseItemType OperateOnTypes + DatabaseRecordItemType OperateOnDatabaseRecordTypes + bool IncludeExpired + bool IncludeArtificial + bool IncludeArchived + bool RemoveAnalyzers + string TransformScript + int MaxStepsForTransformScript + string EncryptionKey + List Collections + int? MaxReadOpsPerSecond + bool SkipCorruptedData + ExportCompressionAlgorithm? CompressionAlgorithm +} +``` + +
+ +| Property | Type | Description | +|-----------|------|-------------| +| **Collections** | `List` | List of specific collections to export. If empty, all collections are exported.
Default: `empty` | +| **OperateOnTypes** | `DatabaseItemType` | Indicates what should be exported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `ReplicationHubCertificates`, `Identities`, `CompareExchange`, `Attachments`, `CounterGroups`, `Subscriptions`, `TimeSeries`, `TimeSeriesDeletedRanges` | +| **OperateOnDatabaseRecordTypes** | `DatabaseRecordItemType` | Indicates what should be exported from the database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `Settings`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications`, `TimeSeries`, `DocumentsCompression`, `Analyzers`, `LockMode`, `OlapConnectionStrings`, `OlapEtls`, `ElasticSearchConnectionStrings`, `ElasticSearchEtls`, `PostgreSQLIntegration`, `QueueConnectionStrings`, `QueueEtls`, `IndexesHistory`, `Refresh`, `DataArchival`, `QueueSinks`, `SnowflakeEtls`, `SnowflakeConnectionStrings`, `EmbeddingsGenerations`, `AiConnectionStrings` | +| **IncludeExpired** | `bool` | Should expired documents be exported.
Default: `true` | +| **IncludeArtificial** | `bool` | Should artificial documents be exported.
Default: `false` | +| **IncludeArchived** | `bool` | Should archived documents be exported.
Default: `true` | +| **RemoveAnalyzers** | `bool` | Should analyzers be removed from indexes.
Default: `false` | +| **TransformScript** | `string` | JavaScript applied to every exported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **MaxStepsForTransformScript** | `int` | Maximum number of steps the transform script can run before failing.
Default: `10000` | +| **EncryptionKey** | `string` | Encryption key used to encrypt the exported file.
Default: `null` | +| **MaxReadOpsPerSecond** | `int?` | Limits the read-operation rate during the export, to throttle the load placed on the database.
Default: `null` (no limit) | +| **SkipCorruptedData** | `bool` | When `true`, the export continues if corrupted data is encountered (for example, lost compression dictionaries), recording an error entry in the result report instead of stopping.
Default: `false` | +| **CompressionAlgorithm** | `ExportCompressionAlgorithm?` | Compression algorithm used for the exported file: `Zstd` or `Gzip`.
Default: `null` | + +
+ + +**Options that control what an import includes.** + +```csharp +class DatabaseSmugglerImportOptions +{ + DatabaseItemType OperateOnTypes + DatabaseRecordItemType OperateOnDatabaseRecordTypes + bool IncludeExpired + bool IncludeArtificial + bool IncludeArchived + bool RemoveAnalyzers + string TransformScript + int MaxStepsForTransformScript + string EncryptionKey + List Collections + int? MaxReadOpsPerSecond + bool SkipCorruptedData + bool SkipRevisionCreation +} +``` + +
+ +| Property | Type | Description | +|-----------|------|-------------| +| **Collections** | `List` | List of specific collections to import. If empty, all collections are imported.
Default: `empty` | +| **OperateOnTypes** | `DatabaseItemType` | Indicates what should be imported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `ReplicationHubCertificates`, `Identities`, `CompareExchange`, `Attachments`, `CounterGroups`, `Subscriptions`, `TimeSeries`, `TimeSeriesDeletedRanges` | +| **OperateOnDatabaseRecordTypes** | `DatabaseRecordItemType` | Indicates what should be imported into the database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `Settings`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications`, `TimeSeries`, `DocumentsCompression`, `Analyzers`, `LockMode`, `OlapConnectionStrings`, `OlapEtls`, `ElasticSearchConnectionStrings`, `ElasticSearchEtls`, `PostgreSQLIntegration`, `QueueConnectionStrings`, `QueueEtls`, `IndexesHistory`, `Refresh`, `DataArchival`, `QueueSinks`, `SnowflakeEtls`, `SnowflakeConnectionStrings`, `EmbeddingsGenerations`, `AiConnectionStrings` | +| **IncludeExpired** | `bool` | Should expired documents be imported.
Default: `true` | +| **IncludeArtificial** | `bool` | Should artificial documents be imported.
Default: `false` | +| **IncludeArchived** | `bool` | Should archived documents be imported.
Default: `true` | +| **RemoveAnalyzers** | `bool` | Should analyzers be removed from indexes.
Default: `false` | +| **TransformScript** | `string` | JavaScript applied to every imported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **MaxStepsForTransformScript** | `int` | Maximum number of steps the transform script can run before failing.
Default: `10000` | +| **EncryptionKey** | `string` | Encryption key used to read an encrypted `.ravendbdump` file.
Default: `null` | +| **MaxReadOpsPerSecond** | `int?` | Limits the read-operation rate during the import, to throttle the load placed on the database.
Default: `null` (no limit) | +| **SkipCorruptedData** | `bool` | When `true`, the import continues if corrupted data is encountered, recording an error entry in the result report instead of stopping.
Default: `false` | +| **SkipRevisionCreation** | `bool` | When `true`, revisions are not created for the imported documents.
Default: `false` | + +
+
+ + diff --git a/versioned_docs/version-7.0/client-api/smuggler/content/_what-is-smuggler-java.mdx b/versioned_docs/version-7.0/client-api/smuggler/content/_what-is-smuggler-java.mdx index dd4f20fd0c..d604d4267a 100644 --- a/versioned_docs/version-7.0/client-api/smuggler/content/_what-is-smuggler-java.mdx +++ b/versioned_docs/version-7.0/client-api/smuggler/content/_what-is-smuggler-java.mdx @@ -1,137 +1,312 @@ import Admonition from '@theme/Admonition'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -import CodeBlock from '@theme/CodeBlock'; +import Panel from "@site/src/components/Panel"; +import ContentFrame from "@site/src/components/ContentFrame"; -Smuggler gives you the ability to export or import data from or to a database using JSON format. -It is exposed via the `DocumentStore.smuggler()`. + -## ForDatabase +* **Smuggler** is a tool for moving data between databases. + Use it to export documents, indexes, and other items from a database as JSON, + and to import that data into another database. -By default, the `IDocumentStore.smuggler` works on the default document store database from the `IDocumentStore.database` property. +* Smuggler is exposed through the `DocumentStore.smuggler()` method. -In order to switch it to a different database use the `.forDatabase` method. +* An export can write to a `.ravendbdump` file or stream directly into another database. + An import reads that data back into a database. - - -{`DatabaseSmuggler northwindSmuggler = store.smuggler().forDatabase("Northwind"); -`} - - +* You can narrow what is transferred by collection, by item type, + and by applying a transform script to each document. + +* In this article: + * [ForDatabase](../../../client-api/smuggler/what-is-smuggler.mdx#fordatabase) + * [Export](../../../client-api/smuggler/what-is-smuggler.mdx#export) + * [Export specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#export-specific-collections) + * [Import](../../../client-api/smuggler/what-is-smuggler.mdx#import) + * [Import specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#import-specific-collections) + * [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript) + * [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) + * [Methods](../../../client-api/smuggler/what-is-smuggler.mdx#methods) + * [Classes](../../../client-api/smuggler/what-is-smuggler.mdx#classes) + + + + + +By default, the smuggler works on the database set in the `DocumentStore.getDatabase()` property. +To run the smuggler against a different database, switch to it with the `forDatabase` method. + +```java +DatabaseSmuggler northwindSmuggler = store + .smuggler() + .forDatabase("Northwind"); +``` + + + + + +Export writes the selected data to a `.ravendbdump` file, or streams it directly into another database. +Use `DatabaseSmugglerExportOptions` to choose what to include. + +
+ +#### Example: export indexes and documents to a file + +```java +// export only Indexes and Documents to a given file +DatabaseSmugglerExportOptions options = new DatabaseSmugglerExportOptions(); +options.setOperateOnTypes(EnumSet.of( + DatabaseItemType.INDEXES, + DatabaseItemType.DOCUMENTS)); + +Operation exportOperation = store + .smuggler() + .exportAsync(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + +exportOperation.waitForCompletion(); +``` + +
+ + + +### Export specific collections + +By default, all collections are exported. +To export only specific collections, list their names in the `collections` option. + +```java +// Export only the "Orders" and "Categories" collections to a file +DatabaseSmugglerExportOptions options = new DatabaseSmugglerExportOptions(); +options.setCollections(Arrays.asList("Orders", "Categories")); + +Operation exportOperation = store + .smuggler() + .exportAsync(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + +exportOperation.waitForCompletion(); +``` + + + +
+ +See [exportAsync](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. + + + + + +Import reads data from a `.ravendbdump` file or a stream and writes it into the database. +Use `DatabaseSmugglerImportOptions` to choose what to include. + +
+ +#### Example: import documents from a file + +```java +// import only Documents from a given file +DatabaseSmugglerImportOptions options = new DatabaseSmugglerImportOptions(); +options.setOperateOnTypes(EnumSet.of(DatabaseItemType.DOCUMENTS)); + +// import the .ravendbdump file that you exported (i.e. in the export example above) +Operation importOperation = store + .smuggler() + .importAsync(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + +importOperation.waitForCompletion(); +``` + +
+ + + +### Import specific collections + +By default, all collections are imported. +To import only specific collections, list their names in the `collections` option. + +```java +// Import only the "Orders" and "Categories" collections from the file +DatabaseSmugglerImportOptions options = new DatabaseSmugglerImportOptions(); +options.setCollections(Arrays.asList("Orders", "Categories")); + +Operation importOperation = store + .smuggler() + .importAsync(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + +importOperation.waitForCompletion(); +``` + + +
+See [importAsync](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. -## Export + -### Syntax + - - -{`//export +`TransformScript` lets you modify or filter out each document during an export or import, using JavaScript that you provide. + +The JavaScript engine is the same one used for [patching operations](../../../client-api/operations/patching/single-document.mdx), so you have identical syntax and capabilities, plus the ability to filter out a document by throwing a `skip` exception. + +```javascript +var id = this['@metadata']['@id']; +if (id === 'orders/999-A') + throw 'skip'; // filter-out + +this.Freight = 15.3; +``` + + + + + +## Methods + + + +### Smuggler methods + + + + +Switches the smuggler to operate on a database other than the document store's default database. + +```java +public DatabaseSmuggler forDatabase(String databaseName); +``` + +
+ +Usage: + +```java +DatabaseSmuggler northwindSmuggler = store.smuggler().forDatabase("Northwind"); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **databaseName** | `String` | The database that the smuggler will operate on. | + +| Return value | | +|-----------|-------------| +| `DatabaseSmuggler` | A smuggler instance scoped to the specified database. | + +
+ + +Exports the selected data to a file or to another database. + +```java public Operation exportAsync(DatabaseSmugglerExportOptions options, String toFile); public Operation exportAsync(DatabaseSmugglerExportOptions options, DatabaseSmuggler toDatabase); -`} - - +``` -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerExportOptions` | Options that will be used during the export. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | -| **toDatabase** | `DatabaseSmuggler` | `DatabaseSmuggler` instance used as a destination | -| **toFile** | `String` | Path to a file where exported data will be written | - -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | - -### DatabaseSmugglerExportOptions - -| Parameters | | | -| ------------- | ------------- | ----- | -| **Collections** | `List` | List of specific collections to export. If empty, then all collections will be exported. Default: `empty` | -| **operateOnTypes** | `DatabaseItemType` | Indicates what should be exported. Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `Identities`, `CompareExchange`, `Subscriptions` | -| **operateOnDatabaseRecordType** | `DatabaseRecordItemType` | Indicates what should be exported from database record. Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications` | -| **includeExpired** | `boolean` | Should expired documents be included in the export. Default: `true` | -| **removeAnalyzers** | `boolean` | Should analyzers be removed from Indexes. Default: `false` | -| **transformScript** | `String` | JavaScript-based script applied to every exported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **maxStepsForTransformScript** | `int` | Maximum number of steps that transform script can process before failing. Default: 10000 | - -### Example - - - -{`// export only Indexes and Documents to a given file -Operation exportOperation = store.smuggler().exportAsync(exportOptions, "C:\\\\ravendb-exports\\\\Northwind.ravendbdump"); -`} - - +
+Usage: +```java +Operation exportOperation = store.smuggler().exportAsync(options, toFile); +``` -## Import +
-### Syntax +| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerExportOptions` | Options used during the export. See [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | +| **toFile** | `String` | Path to the file where exported data is written. | +| **toDatabase** | `DatabaseSmuggler` | A `DatabaseSmuggler` instance used as the destination. | + +| Return value | | +|-----------|-------------| +| `Operation` | An `Operation` instance you can use to wait for completion and subscribe to progress events. | - - -{`public Operation importAsync(DatabaseSmugglerImportOptions options, String fromFile); -public Operation importAsync(DatabaseSmugglerImportOptions options, InputStream stream); -`} - + -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerImportOptions` | Options that will be used during the import. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | -| **stream** | `InputStream` | Stream with data to import | -| **fromFile** | `String` | Path to a file from which data will be imported | +Imports data from a file or a stream into the database. -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation-class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | +```java +public Operation importAsync(DatabaseSmugglerImportOptions options, String fromFile); + +public Operation importAsync(DatabaseSmugglerImportOptions options, InputStream stream); +``` -### DatabaseSmugglerImportOptions +
-| Parameters | | | -| - | - | - | -| **Collections** | `List` | List of specific collections to import. If empty, then all collections will be exported. Default: `empty` | -| **operateOnTypes** | `DatabaseItemType` | Indicates what should be imported. Default: `INDEXES`, `DOCUMENTS`, `REVISION_DOCUMENTS`, `CONFLICTS`, `DATABASE_RECORD`, `IDENTITIES`, `COMPARE_EXCHANGE`, `SUBSCRIPTIONS` | -| **operateOnDatabaseRecordType** | `DatabaseRecordItemType` | Indicates what should be imported. Default: `CLIENT`, `CONFLICT_SOLVER_CONFIG`, `EXPIRATION`, `EXTERNAL_REPLICATIONS`, `PERIODIC_BACKUPS`, `RAVEN_CONNECTION_STRINGS`, `RAVEN_ETLS`, `REVISIONS`, `SQL_CONNECTION_STRINGS`, `SORTERS`, `SQL_ETLS`, `HUB_PULL_REPLICATIONS`, `SINK_PULL_REPLICATIONS` | -| **includeExpired** | `boolean` | Should expired documents be included in the import. Default: `true` | -| **removeAnalyzers** | `boolean` | Should analyzers be removed from Indexes. Default: `false` | -| **transformScript** | `String` | JavaScript-based script applied to every exported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **maxStepsForTransformScript** | `int` | Maximum number of steps that transform script can process before failing. Default: 10000 | +Usage: +```java +Operation importOperation = store.smuggler().importAsync(options, fromFile); +``` -### Example +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerImportOptions` | Options used during the import. See [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | +| **fromFile** | `String` | Path to the file the data is imported from. | +| **stream** | `InputStream` | Stream with the data to import. | + +| Return value | | +|-----------|-------------| +| `Operation` | An `Operation` instance you can use to wait for completion and subscribe to progress events. | - - -{`Operation importOperation = store.smuggler().importAsync(importOptions, "C:\\\\ravendb-exports\\\\Northwind.ravendbdump"); -`} - +
+
+## Classes -## TransformScript + -`TransformScript` exposes the ability to modify or even filter-out the document during the import and export process using the provided JavaScript. +### Smuggler options classes -Underneath the JavaScript engine is exactly the same as used for [patching operations](../../../client-api/operations/patching/single-document.mdx) giving you identical syntax and capabilities with additional **ability to filter out documents by throwing a 'skip' exception**. +
+ - - -{`var id = this['@metadata']['@id']; -if (id === 'orders/999-A') - throw 'skip'; // filter-out + + + +**Options that control what an export includes.** + +| Property | Type | Description | +|-----------|------|-------------| +| **collections** | `List` | List of specific collections to export. If empty, all collections are exported.
Default: `empty` | +| **operateOnTypes** | `DatabaseItemType` | Indicates what should be exported.
Default: `INDEXES`, `DOCUMENTS`, `REVISION_DOCUMENTS`, `CONFLICTS`, `DATABASE_RECORD`, `IDENTITIES`, `COMPARE_EXCHANGE`, `SUBSCRIPTIONS` | +| **operateOnDatabaseRecordType** | `DatabaseRecordItemType` | Indicates what should be exported from the database record.
Default: `CLIENT`, `CONFLICT_SOLVER_CONFIG`, `EXPIRATION`, `EXTERNAL_REPLICATIONS`, `PERIODIC_BACKUPS`, `RAVEN_CONNECTION_STRINGS`, `RAVEN_ETLS`, `REVISIONS`, `SQL_CONNECTION_STRINGS`, `SORTERS`, `SQL_ETLS`, `HUB_PULL_REPLICATIONS`, `SINK_PULL_REPLICATIONS` | +| **includeExpired** | `boolean` | Should expired documents be exported.
Default: `true` | +| **removeAnalyzers** | `boolean` | Should analyzers be removed from indexes.
Default: `false` | +| **transformScript** | `String` | JavaScript applied to every exported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **maxStepsForTransformScript** | `int` | Maximum number of steps the transform script can run before failing.
Default: `10000` | -this.Freight = 15.3; -`} -
+ +**Options that control what an import includes.** +| Property | Type | Description | +|-----------|------|-------------| +| **collections** | `List` | List of specific collections to import. If empty, all collections are imported.
Default: `empty` | +| **operateOnTypes** | `DatabaseItemType` | Indicates what should be imported.
Default: `INDEXES`, `DOCUMENTS`, `REVISION_DOCUMENTS`, `CONFLICTS`, `DATABASE_RECORD`, `IDENTITIES`, `COMPARE_EXCHANGE`, `SUBSCRIPTIONS` | +| **operateOnDatabaseRecordType** | `DatabaseRecordItemType` | Indicates what should be imported into the database record.
Default: `CLIENT`, `CONFLICT_SOLVER_CONFIG`, `EXPIRATION`, `EXTERNAL_REPLICATIONS`, `PERIODIC_BACKUPS`, `RAVEN_CONNECTION_STRINGS`, `RAVEN_ETLS`, `REVISIONS`, `SQL_CONNECTION_STRINGS`, `SORTERS`, `SQL_ETLS`, `HUB_PULL_REPLICATIONS`, `SINK_PULL_REPLICATIONS` | +| **includeExpired** | `boolean` | Should expired documents be imported.
Default: `true` | +| **removeAnalyzers** | `boolean` | Should analyzers be removed from indexes.
Default: `false` | +| **transformScript** | `String` | JavaScript applied to every imported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **maxStepsForTransformScript** | `int` | Maximum number of steps the transform script can run before failing.
Default: `10000` | + +
+ + + diff --git a/versioned_docs/version-7.0/client-api/smuggler/content/_what-is-smuggler-nodejs.mdx b/versioned_docs/version-7.0/client-api/smuggler/content/_what-is-smuggler-nodejs.mdx index ca9f9b333d..11a7739d76 100644 --- a/versioned_docs/version-7.0/client-api/smuggler/content/_what-is-smuggler-nodejs.mdx +++ b/versioned_docs/version-7.0/client-api/smuggler/content/_what-is-smuggler-nodejs.mdx @@ -1,150 +1,358 @@ import Admonition from '@theme/Admonition'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -import CodeBlock from '@theme/CodeBlock'; +import Panel from "@site/src/components/Panel"; +import ContentFrame from "@site/src/components/ContentFrame"; -Smuggler gives you the ability to export or import data from or to a database using JSON format. -It is exposed via the `DocumentStore.smuggler`. + -## ForDatabase +* **Smuggler** is a tool for moving data between databases. + Use it to export documents, indexes, and other items from a database as JSON, + and to import that data into another database. -By default, the `DocumentStore.smuggler` works on the default document store database from the `DocumentStore.database` . +* Smuggler is exposed through the `documentStore.smuggler` property. -In order to switch it to a different database use the `.forDatabase` method. +* An export can write to a `.ravendbdump` file. + An import reads that data back into a database. - - -{`const northwindSmuggler = store +* You can narrow what is transferred by collection, by item type, + and by applying a transform script to each document. + +* In this article: + * [ForDatabase](../../../client-api/smuggler/what-is-smuggler.mdx#fordatabase) + * [Export](../../../client-api/smuggler/what-is-smuggler.mdx#export) + * [Export specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#export-specific-collections) + * [Import](../../../client-api/smuggler/what-is-smuggler.mdx#import) + * [Import specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#import-specific-collections) + * [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript) + * [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) + * [Methods](../../../client-api/smuggler/what-is-smuggler.mdx#methods) + * [Classes](../../../client-api/smuggler/what-is-smuggler.mdx#classes) + + + + + +By default, the smuggler works on the database set in the `documentStore.database` property. +To run the smuggler against a different database, switch to it with the `forDatabase` method. + +```js +const northwindSmuggler = store .smuggler .forDatabase("Northwind"); -`} - - +``` + + -## Export +Export writes the selected data to a `.ravendbdump` file. +Use `DatabaseSmugglerExportOptions` to choose what to include. -### Usage +
- - -{`const operation = await store.smuggler.export(options, toDatabase); +#### Example: export indexes and documents to a file -const operation = await store.smuggler.export(options, toFile); -`} - - +```js +// export only Indexes and Documents to a given file +const options = new DatabaseSmugglerExportOptions(); +options.operateOnTypes = ["Indexes", "Documents"]; -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerExportOptions` | Options that will be used during the export. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | -| **toDatabase** | `DatabaseSmuggler` | `DatabaseSmuggler` instance used as a destination | -| **toFile** | `string` | Path to a file where exported data will be written | - -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | - -### DatabaseSmugglerExportOptions - -| Parameters | | | -| - | - | - | -| **collections** | ` string[]` | List of specific collections to export. If empty, then all collections will be exported. Default: `empty` | -| **operateOnTypes** | `DatabaseItemType[]` | Indicates what should be exported. Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `Identities`, `CompareExchange`, `Subscriptions` | -| **operateOnDatabaseRecordTypes** | `DatabaseRecordItemType[]` | Indicates what should be exported from database record. Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications` | -| **includeExpired** | `boolean` | Should expired documents be included in the export. Default: `true` | -| **includeArtificial** | `boolean` | ? | -| **removeAnalyzers** | `boolean` | Should analyzers be removed from Indexes. Default: `false` | -| **transformScript** | `string` | JavaScript-based script applied to every exported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **maxStepsForTransformScript** | `number` | Maximum number of steps that transform script can process before failing. Default: 10000 | -| **skipRevisionCreation** | `boolean` | skip revision creation | -| **encryptionKey** | `string` | Encryption key used for restore | - -### Example - - - -{`const options = new DatabaseSmugglerExportOptions(); -options.operateOnTypes = ["Documents"]; const operation = await store .smuggler - .export(options, "C:\\\\ravendb-exports\\\\Northwind.ravendbdump"); + .export(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + await operation.waitForCompletion(); -`} - - +``` +
+ -## Import +### Export specific collections -### Usage +By default, all collections are exported. +To export only specific collections, list their names in the `collections` option. - - -{`const operation = await store.smuggler.import(options, fromFile); +```js +// Export only the "Orders" and "Categories" collections to a file +const options = new DatabaseSmugglerExportOptions(); +options.collections = ["Orders", "Categories"]; -const operation = await store.smuggler.import(options, stream); -`} - - +const operation = await store + .smuggler + .export(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + +await operation.waitForCompletion(); +``` + + + +
+ +See [export](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. + + + + -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerImportOptions` | Options that will be used during the import. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | -| **stream** | `Stream` | Stream with data to import | -| **fromFile** | `string` | Path to a file from which data will be imported | - -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation-class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | - -### DatabaseSmugglerImportOptions - -| Parameters | | | -| - | - | - | -| **operateOnTypes** | `DatabaseItemType[]` | Indicates what should be imported. Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `Identities`, `CompareExchange`, `Subscriptions` | -| **operateOnDatabaseRecordTypes** | `DatabaseRecordItemType[]` | Indicates what should be imported. Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications` | -| **includeExpired** | `boolean` | Should expired documents be included in the import. Default: `true` | -| **includeArtificial** | `boolean` | ? | -| **removeAnalyzers** | `boolean` | Should analyzers be removed from Indexes. Default: `false` | -| **transformScript** | `string` | JavaScript-based script applied to every imported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **maxStepsForTransformScript** | `number` | Maximum number of steps that transform script can process before failing. Default: 10000 | -| **skipRevisionCreation** | `boolean` | skip revision creation | -| **encryptionKey** | `string` | Encryption key used for restore | - -### Example - - - -{`const options = new DatabaseSmugglerImportOptions(); +Import reads data from a `.ravendbdump` file and writes it into the database. +Use `DatabaseSmugglerImportOptions` to choose what to include. + +
+ +#### Example: import documents from a file + +```js +// import only Documents from a given file +const options = new DatabaseSmugglerImportOptions(); options.operateOnTypes = ["Documents"]; -const operation = await store.smuggler.import(options, "C:\\\\ravendb-exports\\\\Northwind.ravendbdump"); + +// import the .ravendbdump file that you exported (i.e. in the export example above) +const operation = await store + .smuggler + .import(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + await operation.waitForCompletion(); -`} - - +``` + +
+ + + +### Import specific collections + +By default, all collections are imported. +To import only specific collections, list their names in the `collections` option. + +```js +// Import only the "Orders" and "Categories" collections from the file +const options = new DatabaseSmugglerImportOptions(); +options.collections = ["Orders", "Categories"]; + +const operation = await store + .smuggler + .import(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + +await operation.waitForCompletion(); +``` + + + +
+See [import](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. + -## TransformScript + -`TransformScript` exposes the ability to modify or even filter-out the document during the import and export process using the provided JavaScript. +`TransformScript` lets you modify or filter out each document during an export or import, using JavaScript that you provide. -Underneath the JavaScript engine is exactly the same as used for [patching operations](../../../client-api/operations/patching/single-document.mdx) giving you identical syntax and capabilities with additional **ability to filter out documents by throwing a 'skip' exception**. +The JavaScript engine is the same one used for [patching operations](../../../client-api/operations/patching/single-document.mdx), so you have identical syntax and capabilities, plus the ability to filter out a document by throwing a `skip` exception. - - -{`var id = this['@metadata']['@id']; +```javascript +var id = this['@metadata']['@id']; if (id === 'orders/999-A') throw 'skip'; // filter-out this.Freight = 15.3; -`} - +``` + + + + + +## Methods + + + +### Smuggler methods + + + + +Switches the smuggler to operate on a database other than the document store's default database. + +```ts +forDatabase(databaseName: string): DatabaseSmuggler; +``` + +
+ +Usage: + +```js +const northwindSmuggler = store.smuggler.forDatabase("Northwind"); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **databaseName** | `string` | The database that the smuggler will operate on. | + +| Return value | | +|-----------|-------------| +| `DatabaseSmuggler` | A smuggler instance scoped to the specified database. | + +
+ + +Exports the selected data to a file. + +```ts +export(options: DatabaseSmugglerExportOptions, toFile: string): Promise; +``` + +
+ +Usage: + +```js +const operation = await store.smuggler.export(options, toFile); +await operation.waitForCompletion(); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerExportOptions` | Options used during the export. See [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | +| **toFile** | `string` | Path to the file where exported data is written. | + +| Return value | | +|-----------|-------------| +| `Promise` | An awaiter you can use to wait for the operation to complete and subscribe to progress events. | + +
+ + +Imports data from a file into the database. + +```ts +import(options: DatabaseSmugglerImportOptions, fromFile: string): Promise; +``` + +
+ +Usage: + +```js +const operation = await store.smuggler.import(options, fromFile); +await operation.waitForCompletion(); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerImportOptions` | Options used during the import. See [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | +| **fromFile** | `string` | Path to the file the data is imported from. | + +| Return value | | +|-----------|-------------| +| `Promise` | An awaiter you can use to wait for the operation to complete and subscribe to progress events. | +
+
+ +
+ +## Classes + + + +### Smuggler options classes + +
+ + + + + +**Options that control what an export includes.** + +```ts +class DatabaseSmugglerExportOptions +{ + operateOnTypes: DatabaseItemType[] + operateOnDatabaseRecordType: DatabaseRecordItemType[] + includeExpired: boolean + includeArtificial: boolean + includeArchived: boolean + removeAnalyzers: boolean + transformScript: string + maxStepsForTransformScript: number + skipRevisionCreation: boolean + encryptionKey: string + collections: string[] + skipCorruptedData: boolean + compressionAlgorithm?: ExportCompressionAlgorithm +} +``` + +
+ +| Property | Type | Description | +|-----------|------|-------------| +| **collections** | `string[]` | List of specific collections to export. If empty, all collections are exported.
Default: `empty` | +| **operateOnTypes** | `DatabaseItemType[]` | Indicates what should be exported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `ReplicationHubCertificates`, `Identities`, `CompareExchange`, `Attachments`, `CounterGroups`, `Subscriptions`, `TimeSeries` | +| **operateOnDatabaseRecordType** | `DatabaseRecordItemType[]` | Indicates what should be exported from the database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `Settings`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications`, `TimeSeries`, `DocumentsCompression`, `Analyzers`, `LockMode`, `OlapConnectionStrings`, `OlapEtls`, `ElasticSearchConnectionStrings`, `ElasticSearchEtls`, `PostgreSqlIntegration`, `QueueConnectionStrings`, `QueueEtl`, `IndexesHistory`, `Refresh`, `QueueSinks`, `DataArchival` | +| **includeExpired** | `boolean` | Should expired documents be exported.
Default: `true` | +| **includeArtificial** | `boolean` | Should artificial documents be exported.
Default: `false` | +| **includeArchived** | `boolean` | Should archived documents be exported.
Default: `true` | +| **removeAnalyzers** | `boolean` | Should analyzers be removed from indexes.
Default: `false` | +| **transformScript** | `string` | JavaScript applied to every exported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **maxStepsForTransformScript** | `number` | Maximum number of steps the transform script can run before failing.
Default: `10000` | +| **skipRevisionCreation** | `boolean` | When `true`, revisions are not created for the processed documents.
Default: `false` | +| **encryptionKey** | `string` | Encryption key used to encrypt the exported file.
Default: `null` | +| **skipCorruptedData** | `boolean` | When `true`, the export continues if corrupted data is encountered (for example, lost compression dictionaries), recording an error entry in the result report instead of stopping.
Default: `false` | +| **compressionAlgorithm** | `ExportCompressionAlgorithm` | Compression algorithm used for the exported file: `Zstd` or `Gzip`.
Default: `null` | +
+ + +**Options that control what an import includes.** + +```ts +class DatabaseSmugglerImportOptions +{ + operateOnTypes: DatabaseItemType[] + operateOnDatabaseRecordType: DatabaseRecordItemType[] + includeExpired: boolean + includeArtificial: boolean + includeArchived: boolean + removeAnalyzers: boolean + transformScript: string + maxStepsForTransformScript: number + skipRevisionCreation: boolean + encryptionKey: string + collections: string[] + skipCorruptedData: boolean +} +``` + +
+ +| Property | Type | Description | +|-----------|------|-------------| +| **collections** | `string[]` | List of specific collections to import. If empty, all collections are imported.
Default: `empty` | +| **operateOnTypes** | `DatabaseItemType[]` | Indicates what should be imported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `ReplicationHubCertificates`, `Identities`, `CompareExchange`, `Attachments`, `CounterGroups`, `Subscriptions`, `TimeSeries` | +| **operateOnDatabaseRecordType** | `DatabaseRecordItemType[]` | Indicates what should be imported into the database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `Settings`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications`, `TimeSeries`, `DocumentsCompression`, `Analyzers`, `LockMode`, `OlapConnectionStrings`, `OlapEtls`, `ElasticSearchConnectionStrings`, `ElasticSearchEtls`, `PostgreSqlIntegration`, `QueueConnectionStrings`, `QueueEtl`, `IndexesHistory`, `Refresh`, `QueueSinks`, `DataArchival` | +| **includeExpired** | `boolean` | Should expired documents be imported.
Default: `true` | +| **includeArtificial** | `boolean` | Should artificial documents be imported.
Default: `false` | +| **includeArchived** | `boolean` | Should archived documents be imported.
Default: `true` | +| **removeAnalyzers** | `boolean` | Should analyzers be removed from indexes.
Default: `false` | +| **transformScript** | `string` | JavaScript applied to every imported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **maxStepsForTransformScript** | `number` | Maximum number of steps the transform script can run before failing.
Default: `10000` | +| **skipRevisionCreation** | `boolean` | When `true`, revisions are not created for the imported documents.
Default: `false` | +| **encryptionKey** | `string` | Encryption key used to read an encrypted `.ravendbdump` file.
Default: `null` | +| **skipCorruptedData** | `boolean` | When `true`, the import continues if corrupted data is encountered, recording an error entry in the result report instead of stopping.
Default: `false` | +
+
+ + diff --git a/versioned_docs/version-7.1/client-api/smuggler/content/_what-is-smuggler-csharp.mdx b/versioned_docs/version-7.1/client-api/smuggler/content/_what-is-smuggler-csharp.mdx index 4bdf46a8fc..4ecfc4f75d 100644 --- a/versioned_docs/version-7.1/client-api/smuggler/content/_what-is-smuggler-csharp.mdx +++ b/versioned_docs/version-7.1/client-api/smuggler/content/_what-is-smuggler-csharp.mdx @@ -1,179 +1,397 @@ import Admonition from '@theme/Admonition'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -import CodeBlock from '@theme/CodeBlock'; +import Panel from "@site/src/components/Panel"; +import ContentFrame from "@site/src/components/ContentFrame"; -Smuggler gives you the ability to export or import data from or to a database using JSON format. -It is exposed via the `DocumentStore.Smuggler` property. + -## ForDatabase +* **Smuggler** is a tool for moving data between databases. + Use it to export documents, indexes, and other items from a database as JSON, + and to import that data into another database. -By default, the `DocumentStore.Smuggler` works on the default document store database from the `DocumentStore.Database` property. +* Smuggler is exposed through the `DocumentStore.Smuggler` property. -In order to switch it to a different database use the `.ForDatabase` method. +* An export can write to a `.ravendbdump` file or stream directly into another database. + An import reads that data back into a database. - - -{`var northwindSmuggler = store - .Smuggler - .ForDatabase("Northwind"); -`} - - +* You can narrow what is transferred by collection, by item type, + and by applying a transform script to each document. +* In this article: + * [ForDatabase](../../../client-api/smuggler/what-is-smuggler.mdx#fordatabase) + * [Export](../../../client-api/smuggler/what-is-smuggler.mdx#export) + * [Export specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#export-specific-collections) + * [Import](../../../client-api/smuggler/what-is-smuggler.mdx#import) + * [Import specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#import-specific-collections) + * [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript) + * [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) + * [Methods](../../../client-api/smuggler/what-is-smuggler.mdx#methods) + * [Classes](../../../client-api/smuggler/what-is-smuggler.mdx#classes) + -## Export + -### Syntax +By default, `DocumentStore.Smuggler` works on the database set in the `DocumentStore.Database` property. +To run Smuggler against a different database, switch to it with the `ForDatabase` method. - - -{`Task ExportAsync( - DatabaseSmugglerExportOptions options, - DatabaseSmuggler toDatabase, - CancellationToken token = default(CancellationToken)); - -Task ExportAsync( - DatabaseSmugglerExportOptions options, - string toFile, - CancellationToken token = default(CancellationToken)); -`} - - +```csharp +var northwindSmuggler = store + .Smuggler + .ForDatabase("Northwind"); +``` -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerExportOptions` | Options that will be used during the export. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | -| **toDatabase** | `DatabaseSmuggler` | `DatabaseSmuggler` instance used as a destination | -| **toFile** | `string` | Path to a file where exported data will be written | -| **token** | `CancellationToken` | Token used to cancel the operation | + -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | + -### DatabaseSmugglerExportOptions +Export writes the selected data to a `.ravendbdump` file, or streams it directly into another database. +Use `DatabaseSmugglerExportOptions` to choose what to include. -| Parameters | | | -| ------------- | ------------- | ----- | -| **Collections** | `List` | List of specific collections to export. If empty, then all collections will be exported.
Default: `empty` | -| **OperateOnTypes** | `DatabaseItemType` | Indicates what should be exported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `ReplicationHubCertificates`, `Identities`, `CompareExchange`, `Attachments`, `CounterGroups`, `Subscriptions`, `TimeSeries` | -| **OperateOnDatabaseRecordTypes** | `DatabaseRecordItemType` | Indicates what should be exported from database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `Settings`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications`, `TimeSeries`, `DocumentsCompression`, `Analyzers`, `LockMode`, `OlapConnectionStrings`, `OlapEtls`, `ElasticSearchConnectionStrings`, `ElasticSearchEtls`, `PostgreSQLIntegration`, `QueueConnectionStrings`, `QueueEtls`, `IndexesHistory`, `Refresh`, `DataArchival` | -| **IncludeExpired** | `bool` | Should expired documents be exported.
Default: `true` | -| **IncludeArtificial** | `bool` | Should artificial documents be exported.
Default: `false` | -| **IncludeArchived** | `bool` | Should archived documents be exported.
Default: `true` | -| **RemoveAnalyzers** | `bool` | Should analyzers be removed from Indexes.
Default: `false` | -| **TransformScript** | `string` | JavaScript-based script applied to every exported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **MaxStepsForTransformScript** | `int` | Maximum number of steps that the transform script can process before failing.
Default: **10000** | +
-### Example +#### Example: export indexes and documents to a file - - -{`// export only Indexes and Documents to a given file +```csharp +// export only Indexes and Documents to a given file var exportOperation = await store .Smuggler .ExportAsync( new DatabaseSmugglerExportOptions - \{ + { OperateOnTypes = DatabaseItemType.Indexes | DatabaseItemType.Documents - \}, - @"C:\\ravendb-exports\\Northwind.ravendbdump", + }, + @"C:\ravendb-exports\Northwind.ravendbdump", token); await exportOperation.WaitForCompletionAsync(); -`} - - +``` +
+ -## Import +### Export specific collections -### Syntax +By default, all collections are exported. +To export only specific collections, list their names in the `Collections` option. - - -{`Task ImportAsync( - DatabaseSmugglerImportOptions options, - Stream stream, - CancellationToken token = default(CancellationToken)); +```csharp +// Export only the "Orders" and "Categories" collections to a file +var exportOperation = await store + .Smuggler + .ExportAsync( + new DatabaseSmugglerExportOptions + { + Collections = new List { "Orders", "Categories" } + }, + @"C:\ravendb-exports\Northwind.ravendbdump", + token); -Task ImportAsync( - DatabaseSmugglerImportOptions options, - string fromFile, - CancellationToken token = default(CancellationToken)); -`} - - +await exportOperation.WaitForCompletionAsync(); +``` -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerImportOptions` | Options that will be used during the import. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | -| **stream** | `Stream` | Stream with data to import | -| **fromFile** | `string` | Path to a file from which data will be imported | -| **token** | `CancellationToken` | Token used to cancel the operation | + -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation-class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | +
-### DatabaseSmugglerImportOptions +See [ExportAsync](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. -| Parameters | | | -| - | - | - | -| **Collections** | `List` | List of specific collections to import. If empty, then all collections will be imported.
Default: `empty` | -| **OperateOnTypes** | `DatabaseItemType` | Indicates what should be imported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `ReplicationHubCertificates`, `Identities`, `CompareExchange`, `Attachments`, `CounterGroups`, `Subscriptions`, `TimeSeries` | -| **OperateOnDatabaseRecordTypes** | `DatabaseRecordItemType` | Indicates what should be imported from database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `Settings`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications`, `TimeSeries`, `DocumentsCompression`, `Analyzers`, `LockMode`, `OlapConnectionStrings`, `OlapEtls`, `ElasticSearchConnectionStrings`, `ElasticSearchEtls`, `PostgreSQLIntegration`, `QueueConnectionStrings`, `QueueEtls`, `IndexesHistory`, `Refresh`, `DataArchival` | -| **IncludeExpired** | `bool` | Should expired documents be imported.
Default: `true` | -| **IncludeArtificial** | `bool` | Should artificial documents be imported.
Default: `false` | -| **IncludeArchived** | `bool` | Should archived documents be imported.
Default: `true` | -| **RemoveAnalyzers** | `bool` | Should analyzers be removed from Indexes.
Default: `false` | -| **TransformScript** | `string` | JavaScript-based script applied to every imported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **MaxStepsForTransformScript** | `int` | Maximum number of steps that the transform script can process before failing.
Default: **10000** | + + + -### Example +Import reads data from a `.ravendbdump` file or a stream and writes it into the database. +Use `DatabaseSmugglerImportOptions` to choose what to include. - - -{`// import only Documents from a given file +
+ +#### Example: import documents from a file + +```csharp +// import only Documents from a given file var importOperation = await store .Smuggler .ImportAsync( new DatabaseSmugglerImportOptions - \{ + { OperateOnTypes = DatabaseItemType.Documents - \}, + }, // import the .ravendbdump file that you exported (i.e. in the export example above) - @"C:\\ravendb-exports\\Northwind.ravendbdump", + @"C:\ravendb-exports\Northwind.ravendbdump", token); await importOperation.WaitForCompletionAsync(); -`} - - +``` +
+ -## TransformScript +### Import specific collections -`TransformScript` exposes the ability to modify or even filter-out the document during the import and export process using the provided JavaScript. +By default, all collections are imported. +To import only specific collections, list their names in the `Collections` option. -Underneath the JavaScript engine is exactly the same as used for [patching operations](../../../client-api/operations/patching/single-document.mdx) giving you identical syntax and capabilities with additional **ability to filter out documents by throwing a 'skip' exception**. +```csharp +// Import only the "Orders" and "Categories" collections from the file +var importOperation = await store + .Smuggler + .ImportAsync( + new DatabaseSmugglerImportOptions + { + Collections = new List { "Orders", "Categories" } + }, + @"C:\ravendb-exports\Northwind.ravendbdump", + token); + +await importOperation.WaitForCompletionAsync(); +``` - - -{`var id = this['@metadata']['@id']; + + +
+ +See [ImportAsync](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. + + + + + +`TransformScript` lets you modify or filter out each document during an export or import, using JavaScript that you provide. + +The JavaScript engine is the same one used for [patching operations](../../../client-api/operations/patching/single-document.mdx), so you have identical syntax and capabilities, plus the ability to filter out a document by throwing a `skip` exception. + +```javascript +var id = this['@metadata']['@id']; if (id === 'orders/999-A') throw 'skip'; // filter-out this.Freight = 15.3; -`} - +``` + + + + + +## Methods + + + +### Smuggler methods + + + + +Switches Smuggler to operate on a database other than the document store's default database. + +```csharp +public DatabaseSmuggler ForDatabase(string databaseName); +``` + +
+ +Usage: + +```csharp +var northwindSmuggler = store.Smuggler.ForDatabase("Northwind"); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **databaseName** | `string` | The database that Smuggler will operate on. | + +| Return value | | +|-----------|-------------| +| `DatabaseSmuggler` | A Smuggler instance scoped to the specified database. | + +
+ + +Exports the selected data to a file, to a stream, or to another database. + +```csharp +Task ExportAsync( + DatabaseSmugglerExportOptions options, + string toFile, + CancellationToken token = default); + +Task ExportAsync( + DatabaseSmugglerExportOptions options, + Stream toStream, + CancellationToken token = default); + +Task ExportAsync( + DatabaseSmugglerExportOptions options, + DatabaseSmuggler toDatabase, + CancellationToken token = default); +``` + +
+ +Usage: + +```csharp +var exportOperation = await store.Smuggler.ExportAsync(options, toFile, token); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerExportOptions` | Options used during the export. See [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | +| **toFile** | `string` | Path to the file where exported data is written. | +| **toStream** | `Stream` | Stream that exported data is written to. | +| **toDatabase** | `DatabaseSmuggler` | A `DatabaseSmuggler` instance used as the destination. | +| **token** | `CancellationToken` | Token used to cancel the operation. | + +| Return value | | +|-----------|-------------| +| `Operation` | An `Operation` instance you can use to wait for completion and subscribe to progress events. | +
+ + +Imports data from a file or a stream into the database. + +```csharp +Task ImportAsync( + DatabaseSmugglerImportOptions options, + string fromFile, + CancellationToken token = default); + +Task ImportAsync( + DatabaseSmugglerImportOptions options, + Stream stream, + CancellationToken token = default); +``` + +
+ +Usage: + +```csharp +var importOperation = await store.Smuggler.ImportAsync(options, fromFile, token); +``` +
+| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerImportOptions` | Options used during the import. See [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | +| **fromFile** | `string` | Path to the file the data is imported from. | +| **stream** | `Stream` | Stream with the data to import. | +| **token** | `CancellationToken` | Token used to cancel the operation. | + +| Return value | | +|-----------|-------------| +| `Operation` | An `Operation` instance you can use to wait for completion and subscribe to progress events. | + +
+
+ +
+ +## Classes + + + +### Smuggler options classes + +
+ + + + + +**Options that control what an export includes.** + +```csharp +class DatabaseSmugglerExportOptions +{ + DatabaseItemType OperateOnTypes + DatabaseRecordItemType OperateOnDatabaseRecordTypes + bool IncludeExpired + bool IncludeArtificial + bool IncludeArchived + bool RemoveAnalyzers + string TransformScript + int MaxStepsForTransformScript + string EncryptionKey + List Collections + int? MaxReadOpsPerSecond + bool SkipCorruptedData + ExportCompressionAlgorithm? CompressionAlgorithm +} +``` + +
+ +| Property | Type | Description | +|-----------|------|-------------| +| **Collections** | `List` | List of specific collections to export. If empty, all collections are exported.
Default: `empty` | +| **OperateOnTypes** | `DatabaseItemType` | Indicates what should be exported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `ReplicationHubCertificates`, `Identities`, `CompareExchange`, `Attachments`, `CounterGroups`, `Subscriptions`, `TimeSeries`, `TimeSeriesDeletedRanges` | +| **OperateOnDatabaseRecordTypes** | `DatabaseRecordItemType` | Indicates what should be exported from the database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `Settings`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications`, `TimeSeries`, `DocumentsCompression`, `Analyzers`, `LockMode`, `OlapConnectionStrings`, `OlapEtls`, `ElasticSearchConnectionStrings`, `ElasticSearchEtls`, `PostgreSQLIntegration`, `QueueConnectionStrings`, `QueueEtls`, `IndexesHistory`, `Refresh`, `DataArchival`, `QueueSinks`, `SnowflakeEtls`, `SnowflakeConnectionStrings`, `EmbeddingsGenerations`, `AiConnectionStrings`, `GenAiEtls`, `AiAgents` | +| **IncludeExpired** | `bool` | Should expired documents be exported.
Default: `true` | +| **IncludeArtificial** | `bool` | Should artificial documents be exported.
Default: `false` | +| **IncludeArchived** | `bool` | Should archived documents be exported.
Default: `true` | +| **RemoveAnalyzers** | `bool` | Should analyzers be removed from indexes.
Default: `false` | +| **TransformScript** | `string` | JavaScript applied to every exported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **MaxStepsForTransformScript** | `int` | Maximum number of steps the transform script can run before failing.
Default: `10000` | +| **EncryptionKey** | `string` | Encryption key used to encrypt the exported file.
Default: `null` | +| **MaxReadOpsPerSecond** | `int?` | Limits the read-operation rate during the export, to throttle the load placed on the database.
Default: `null` (no limit) | +| **SkipCorruptedData** | `bool` | When `true`, the export continues if corrupted data is encountered (for example, lost compression dictionaries), recording an error entry in the result report instead of stopping.
Default: `false` | +| **CompressionAlgorithm** | `ExportCompressionAlgorithm?` | Compression algorithm used for the exported file: `Zstd` or `Gzip`.
Default: `null` | + +
+ + +**Options that control what an import includes.** + +```csharp +class DatabaseSmugglerImportOptions +{ + DatabaseItemType OperateOnTypes + DatabaseRecordItemType OperateOnDatabaseRecordTypes + bool IncludeExpired + bool IncludeArtificial + bool IncludeArchived + bool RemoveAnalyzers + string TransformScript + int MaxStepsForTransformScript + string EncryptionKey + List Collections + int? MaxReadOpsPerSecond + bool SkipCorruptedData + bool SkipRevisionCreation +} +``` + +
+ +| Property | Type | Description | +|-----------|------|-------------| +| **Collections** | `List` | List of specific collections to import. If empty, all collections are imported.
Default: `empty` | +| **OperateOnTypes** | `DatabaseItemType` | Indicates what should be imported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `ReplicationHubCertificates`, `Identities`, `CompareExchange`, `Attachments`, `CounterGroups`, `Subscriptions`, `TimeSeries`, `TimeSeriesDeletedRanges` | +| **OperateOnDatabaseRecordTypes** | `DatabaseRecordItemType` | Indicates what should be imported into the database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `Settings`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications`, `TimeSeries`, `DocumentsCompression`, `Analyzers`, `LockMode`, `OlapConnectionStrings`, `OlapEtls`, `ElasticSearchConnectionStrings`, `ElasticSearchEtls`, `PostgreSQLIntegration`, `QueueConnectionStrings`, `QueueEtls`, `IndexesHistory`, `Refresh`, `DataArchival`, `QueueSinks`, `SnowflakeEtls`, `SnowflakeConnectionStrings`, `EmbeddingsGenerations`, `AiConnectionStrings`, `GenAiEtls`, `AiAgents` | +| **IncludeExpired** | `bool` | Should expired documents be imported.
Default: `true` | +| **IncludeArtificial** | `bool` | Should artificial documents be imported.
Default: `false` | +| **IncludeArchived** | `bool` | Should archived documents be imported.
Default: `true` | +| **RemoveAnalyzers** | `bool` | Should analyzers be removed from indexes.
Default: `false` | +| **TransformScript** | `string` | JavaScript applied to every imported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **MaxStepsForTransformScript** | `int` | Maximum number of steps the transform script can run before failing.
Default: `10000` | +| **EncryptionKey** | `string` | Encryption key used to read an encrypted `.ravendbdump` file.
Default: `null` | +| **MaxReadOpsPerSecond** | `int?` | Limits the read-operation rate during the import, to throttle the load placed on the database.
Default: `null` (no limit) | +| **SkipCorruptedData** | `bool` | When `true`, the import continues if corrupted data is encountered, recording an error entry in the result report instead of stopping.
Default: `false` | +| **SkipRevisionCreation** | `bool` | When `true`, revisions are not created for the imported documents.
Default: `false` | + +
+
+ + diff --git a/versioned_docs/version-7.1/client-api/smuggler/content/_what-is-smuggler-java.mdx b/versioned_docs/version-7.1/client-api/smuggler/content/_what-is-smuggler-java.mdx index dd4f20fd0c..1691730696 100644 --- a/versioned_docs/version-7.1/client-api/smuggler/content/_what-is-smuggler-java.mdx +++ b/versioned_docs/version-7.1/client-api/smuggler/content/_what-is-smuggler-java.mdx @@ -1,137 +1,364 @@ import Admonition from '@theme/Admonition'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -import CodeBlock from '@theme/CodeBlock'; +import Panel from "@site/src/components/Panel"; +import ContentFrame from "@site/src/components/ContentFrame"; -Smuggler gives you the ability to export or import data from or to a database using JSON format. -It is exposed via the `DocumentStore.smuggler()`. + -## ForDatabase +* **Smuggler** is a tool for moving data between databases. + Use it to export documents, indexes, and other items from a database as JSON, + and to import that data into another database. -By default, the `IDocumentStore.smuggler` works on the default document store database from the `IDocumentStore.database` property. +* Smuggler is exposed through the `DocumentStore.smuggler()` method. -In order to switch it to a different database use the `.forDatabase` method. +* An export can write to a `.ravendbdump` file or stream directly into another database. + An import reads that data back into a database. - - -{`DatabaseSmuggler northwindSmuggler = store.smuggler().forDatabase("Northwind"); -`} - - +* You can narrow what is transferred by collection, by item type, + and by applying a transform script to each document. +* In this article: + * [ForDatabase](../../../client-api/smuggler/what-is-smuggler.mdx#fordatabase) + * [Export](../../../client-api/smuggler/what-is-smuggler.mdx#export) + * [Export specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#export-specific-collections) + * [Import](../../../client-api/smuggler/what-is-smuggler.mdx#import) + * [Import specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#import-specific-collections) + * [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript) + * [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) + * [Methods](../../../client-api/smuggler/what-is-smuggler.mdx#methods) + * [Classes](../../../client-api/smuggler/what-is-smuggler.mdx#classes) + -## Export + -### Syntax +By default, the smuggler works on the database set in the `DocumentStore.getDatabase()` property. +To run the smuggler against a different database, switch to it with the `forDatabase` method. - - -{`//export -public Operation exportAsync(DatabaseSmugglerExportOptions options, String toFile); +```java +DatabaseSmuggler northwindSmuggler = store + .smuggler() + .forDatabase("Northwind"); +``` -public Operation exportAsync(DatabaseSmugglerExportOptions options, DatabaseSmuggler toDatabase); -`} - - + -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerExportOptions` | Options that will be used during the export. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | -| **toDatabase** | `DatabaseSmuggler` | `DatabaseSmuggler` instance used as a destination | -| **toFile** | `String` | Path to a file where exported data will be written | - -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | - -### DatabaseSmugglerExportOptions - -| Parameters | | | -| ------------- | ------------- | ----- | -| **Collections** | `List` | List of specific collections to export. If empty, then all collections will be exported. Default: `empty` | -| **operateOnTypes** | `DatabaseItemType` | Indicates what should be exported. Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `Identities`, `CompareExchange`, `Subscriptions` | -| **operateOnDatabaseRecordType** | `DatabaseRecordItemType` | Indicates what should be exported from database record. Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications` | -| **includeExpired** | `boolean` | Should expired documents be included in the export. Default: `true` | -| **removeAnalyzers** | `boolean` | Should analyzers be removed from Indexes. Default: `false` | -| **transformScript** | `String` | JavaScript-based script applied to every exported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **maxStepsForTransformScript** | `int` | Maximum number of steps that transform script can process before failing. Default: 10000 | - -### Example - - - -{`// export only Indexes and Documents to a given file -Operation exportOperation = store.smuggler().exportAsync(exportOptions, "C:\\\\ravendb-exports\\\\Northwind.ravendbdump"); -`} - - + +Export writes the selected data to a `.ravendbdump` file, or streams it directly into another database. +Use `DatabaseSmugglerExportOptions` to choose what to include. +
-## Import +#### Example: export indexes and documents to a file -### Syntax +```java +// export only Indexes and Documents to a given file +DatabaseSmugglerExportOptions options = new DatabaseSmugglerExportOptions(); +options.setOperateOnTypes(EnumSet.of( + DatabaseItemType.INDEXES, + DatabaseItemType.DOCUMENTS)); - - -{`public Operation importAsync(DatabaseSmugglerImportOptions options, String fromFile); -public Operation importAsync(DatabaseSmugglerImportOptions options, InputStream stream); -`} - - +Operation exportOperation = store + .smuggler() + .exportAsync(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerImportOptions` | Options that will be used during the import. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | -| **stream** | `InputStream` | Stream with data to import | -| **fromFile** | `String` | Path to a file from which data will be imported | +exportOperation.waitForCompletion(); +``` -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation-class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | +
-### DatabaseSmugglerImportOptions + -| Parameters | | | -| - | - | - | -| **Collections** | `List` | List of specific collections to import. If empty, then all collections will be exported. Default: `empty` | -| **operateOnTypes** | `DatabaseItemType` | Indicates what should be imported. Default: `INDEXES`, `DOCUMENTS`, `REVISION_DOCUMENTS`, `CONFLICTS`, `DATABASE_RECORD`, `IDENTITIES`, `COMPARE_EXCHANGE`, `SUBSCRIPTIONS` | -| **operateOnDatabaseRecordType** | `DatabaseRecordItemType` | Indicates what should be imported. Default: `CLIENT`, `CONFLICT_SOLVER_CONFIG`, `EXPIRATION`, `EXTERNAL_REPLICATIONS`, `PERIODIC_BACKUPS`, `RAVEN_CONNECTION_STRINGS`, `RAVEN_ETLS`, `REVISIONS`, `SQL_CONNECTION_STRINGS`, `SORTERS`, `SQL_ETLS`, `HUB_PULL_REPLICATIONS`, `SINK_PULL_REPLICATIONS` | -| **includeExpired** | `boolean` | Should expired documents be included in the import. Default: `true` | -| **removeAnalyzers** | `boolean` | Should analyzers be removed from Indexes. Default: `false` | -| **transformScript** | `String` | JavaScript-based script applied to every exported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **maxStepsForTransformScript** | `int` | Maximum number of steps that transform script can process before failing. Default: 10000 | +### Export specific collections +By default, all collections are exported. +To export only specific collections, list their names in the `collections` option. -### Example +```java +// Export only the "Orders" and "Categories" collections to a file +DatabaseSmugglerExportOptions options = new DatabaseSmugglerExportOptions(); +options.setCollections(Arrays.asList("Orders", "Categories")); - - -{`Operation importOperation = store.smuggler().importAsync(importOptions, "C:\\\\ravendb-exports\\\\Northwind.ravendbdump"); -`} - - +Operation exportOperation = store + .smuggler() + .exportAsync(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + +exportOperation.waitForCompletion(); +``` + + + +
+ +See [exportAsync](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. + + + + + +Import reads data from a `.ravendbdump` file or a stream and writes it into the database. +Use `DatabaseSmugglerImportOptions` to choose what to include. + +
+ +#### Example: import documents from a file + +```java +// import only Documents from a given file +DatabaseSmugglerImportOptions options = new DatabaseSmugglerImportOptions(); +options.setOperateOnTypes(EnumSet.of(DatabaseItemType.DOCUMENTS)); +// import the .ravendbdump file that you exported (i.e. in the export example above) +Operation importOperation = store + .smuggler() + .importAsync(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); +importOperation.waitForCompletion(); +``` -## TransformScript +
-`TransformScript` exposes the ability to modify or even filter-out the document during the import and export process using the provided JavaScript. + -Underneath the JavaScript engine is exactly the same as used for [patching operations](../../../client-api/operations/patching/single-document.mdx) giving you identical syntax and capabilities with additional **ability to filter out documents by throwing a 'skip' exception**. +### Import specific collections - - -{`var id = this['@metadata']['@id']; +By default, all collections are imported. +To import only specific collections, list their names in the `collections` option. + +```java +// Import only the "Orders" and "Categories" collections from the file +DatabaseSmugglerImportOptions options = new DatabaseSmugglerImportOptions(); +options.setCollections(Arrays.asList("Orders", "Categories")); + +Operation importOperation = store + .smuggler() + .importAsync(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + +importOperation.waitForCompletion(); +``` + + + +
+ +See [importAsync](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. + + + + + +`TransformScript` lets you modify or filter out each document during an export or import, using JavaScript that you provide. + +The JavaScript engine is the same one used for [patching operations](../../../client-api/operations/patching/single-document.mdx), so you have identical syntax and capabilities, plus the ability to filter out a document by throwing a `skip` exception. + +```javascript +var id = this['@metadata']['@id']; if (id === 'orders/999-A') throw 'skip'; // filter-out this.Freight = 15.3; -`} - +``` + + + + + +## Methods + + + +### Smuggler methods + + + + +Switches the smuggler to operate on a database other than the document store's default database. + +```java +public DatabaseSmuggler forDatabase(String databaseName); +``` + +
+ +Usage: + +```java +DatabaseSmuggler northwindSmuggler = store.smuggler().forDatabase("Northwind"); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **databaseName** | `String` | The database that the smuggler will operate on. | + +| Return value | | +|-----------|-------------| +| `DatabaseSmuggler` | A smuggler instance scoped to the specified database. | +
+ + +Exports the selected data to a file or to another database. + +```java +public Operation exportAsync(DatabaseSmugglerExportOptions options, String toFile); + +public Operation exportAsync(DatabaseSmugglerExportOptions options, DatabaseSmuggler toDatabase); +``` + +
+ +Usage: + +```java +Operation exportOperation = store.smuggler().exportAsync(options, toFile); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerExportOptions` | Options used during the export. See [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | +| **toFile** | `String` | Path to the file where exported data is written. | +| **toDatabase** | `DatabaseSmuggler` | A `DatabaseSmuggler` instance used as the destination. | +| Return value | | +|-----------|-------------| +| `Operation` | An `Operation` instance you can use to wait for completion and subscribe to progress events. | +
+ + +Imports data from a file or a stream into the database. + +```java +public Operation importAsync(DatabaseSmugglerImportOptions options, String fromFile); + +public Operation importAsync(DatabaseSmugglerImportOptions options, InputStream stream); +``` + +
+ +Usage: + +```java +Operation importOperation = store.smuggler().importAsync(options, fromFile); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerImportOptions` | Options used during the import. See [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | +| **fromFile** | `String` | Path to the file the data is imported from. | +| **stream** | `InputStream` | Stream with the data to import. | + +| Return value | | +|-----------|-------------| +| `Operation` | An `Operation` instance you can use to wait for completion and subscribe to progress events. | + +
+
+ +
+ +## Classes + + + +### Smuggler options classes + +
+ + + + + +**Options that control what an export includes.** + +```java +class DatabaseSmugglerExportOptions +{ + EnumSet operateOnTypes + EnumSet operateOnDatabaseRecordType + boolean includeExpired + boolean includeArtificial + boolean includeArchived + boolean removeAnalyzers + String transformScript + int maxStepsForTransformScript + boolean skipRevisionCreation + String encryptionKey + List collections + boolean skipCorruptedData + ExportCompressionAlgorithm compressionAlgorithm +} +``` + +
+ +| Property | Type | Description | +|-----------|------|-------------| +| **collections** | `List` | List of specific collections to export. If empty, all collections are exported.
Default: `empty` | +| **operateOnTypes** | `EnumSet` | Indicates what should be exported.
Default: `INDEXES`, `DOCUMENTS`, `REVISION_DOCUMENTS`, `CONFLICTS`, `DATABASE_RECORD`, `REPLICATION_HUB_CERTIFICATES`, `IDENTITIES`, `COMPARE_EXCHANGE`, `ATTACHMENTS`, `COUNTER_GROUPS`, `SUBSCRIPTIONS`, `TIME_SERIES` | +| **operateOnDatabaseRecordType** | `EnumSet` | Indicates what should be exported from the database record.
Default: `CLIENT`, `CONFLICT_SOLVER_CONFIG`, `EXPIRATION`, `EXTERNAL_REPLICATIONS`, `PERIODIC_BACKUPS`, `RAVEN_CONNECTION_STRINGS`, `RAVEN_ETLS`, `REVISIONS`, `SETTINGS`, `SQL_CONNECTION_STRINGS`, `SORTERS`, `SQL_ETLS`, `HUB_PULL_REPLICATIONS`, `SINK_PULL_REPLICATIONS`, `TIME_SERIES`, `DOCUMENTS_COMPRESSION`, `ANALYZERS`, `LOCK_MODE`, `OLAP_CONNECTION_STRINGS`, `OLAP_ETLS`, `ELASTIC_SEARCH_CONNECTION_STRINGS`, `ELASTIC_SEARCH_ETLS`, `POSTGRE_SQL_INTEGRATION`, `QUEUE_CONNECTION_STRINGS`, `QUEUE_ETLS`, `INDEXES_HISTORY`, `REFRESH`, `QUEUE_SINKS`, `DATA_ARCHIVAL` | +| **includeExpired** | `boolean` | Should expired documents be exported.
Default: `true` | +| **includeArtificial** | `boolean` | Should artificial documents be exported.
Default: `false` | +| **includeArchived** | `boolean` | Should archived documents be exported.
Default: `true` | +| **removeAnalyzers** | `boolean` | Should analyzers be removed from indexes.
Default: `false` | +| **transformScript** | `String` | JavaScript applied to every exported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **maxStepsForTransformScript** | `int` | Maximum number of steps the transform script can run before failing.
Default: `10000` | +| **skipRevisionCreation** | `boolean` | When `true`, revisions are not created for the processed documents.
Default: `false` | +| **encryptionKey** | `String` | Encryption key used to encrypt the exported file.
Default: `null` | +| **skipCorruptedData** | `boolean` | When `true`, the export continues if corrupted data is encountered (for example, lost compression dictionaries), recording an error entry in the result report instead of stopping.
Default: `false` | +| **compressionAlgorithm** | `ExportCompressionAlgorithm` | Compression algorithm used for the exported file: `Zstd` or `Gzip`.
Default: `null` | + +
+ + +**Options that control what an import includes.** + +```java +class DatabaseSmugglerImportOptions +{ + EnumSet operateOnTypes + EnumSet operateOnDatabaseRecordType + boolean includeExpired + boolean includeArtificial + boolean includeArchived + boolean removeAnalyzers + String transformScript + int maxStepsForTransformScript + boolean skipRevisionCreation + String encryptionKey + List collections + boolean skipCorruptedData +} +``` + +
+ +| Property | Type | Description | +|-----------|------|-------------| +| **collections** | `List` | List of specific collections to import. If empty, all collections are imported.
Default: `empty` | +| **operateOnTypes** | `EnumSet` | Indicates what should be imported.
Default: `INDEXES`, `DOCUMENTS`, `REVISION_DOCUMENTS`, `CONFLICTS`, `DATABASE_RECORD`, `REPLICATION_HUB_CERTIFICATES`, `IDENTITIES`, `COMPARE_EXCHANGE`, `ATTACHMENTS`, `COUNTER_GROUPS`, `SUBSCRIPTIONS`, `TIME_SERIES` | +| **operateOnDatabaseRecordType** | `EnumSet` | Indicates what should be imported into the database record.
Default: `CLIENT`, `CONFLICT_SOLVER_CONFIG`, `EXPIRATION`, `EXTERNAL_REPLICATIONS`, `PERIODIC_BACKUPS`, `RAVEN_CONNECTION_STRINGS`, `RAVEN_ETLS`, `REVISIONS`, `SETTINGS`, `SQL_CONNECTION_STRINGS`, `SORTERS`, `SQL_ETLS`, `HUB_PULL_REPLICATIONS`, `SINK_PULL_REPLICATIONS`, `TIME_SERIES`, `DOCUMENTS_COMPRESSION`, `ANALYZERS`, `LOCK_MODE`, `OLAP_CONNECTION_STRINGS`, `OLAP_ETLS`, `ELASTIC_SEARCH_CONNECTION_STRINGS`, `ELASTIC_SEARCH_ETLS`, `POSTGRE_SQL_INTEGRATION`, `QUEUE_CONNECTION_STRINGS`, `QUEUE_ETLS`, `INDEXES_HISTORY`, `REFRESH`, `QUEUE_SINKS`, `DATA_ARCHIVAL` | +| **includeExpired** | `boolean` | Should expired documents be imported.
Default: `true` | +| **includeArtificial** | `boolean` | Should artificial documents be imported.
Default: `false` | +| **includeArchived** | `boolean` | Should archived documents be imported.
Default: `true` | +| **removeAnalyzers** | `boolean` | Should analyzers be removed from indexes.
Default: `false` | +| **transformScript** | `String` | JavaScript applied to every imported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **maxStepsForTransformScript** | `int` | Maximum number of steps the transform script can run before failing.
Default: `10000` | +| **skipRevisionCreation** | `boolean` | When `true`, revisions are not created for the imported documents.
Default: `false` | +| **encryptionKey** | `String` | Encryption key used to read an encrypted `.ravendbdump` file.
Default: `null` | +| **skipCorruptedData** | `boolean` | When `true`, the import continues if corrupted data is encountered, recording an error entry in the result report instead of stopping.
Default: `false` | + +
+
+ + diff --git a/versioned_docs/version-7.1/client-api/smuggler/content/_what-is-smuggler-nodejs.mdx b/versioned_docs/version-7.1/client-api/smuggler/content/_what-is-smuggler-nodejs.mdx index ca9f9b333d..11a7739d76 100644 --- a/versioned_docs/version-7.1/client-api/smuggler/content/_what-is-smuggler-nodejs.mdx +++ b/versioned_docs/version-7.1/client-api/smuggler/content/_what-is-smuggler-nodejs.mdx @@ -1,150 +1,358 @@ import Admonition from '@theme/Admonition'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -import CodeBlock from '@theme/CodeBlock'; +import Panel from "@site/src/components/Panel"; +import ContentFrame from "@site/src/components/ContentFrame"; -Smuggler gives you the ability to export or import data from or to a database using JSON format. -It is exposed via the `DocumentStore.smuggler`. + -## ForDatabase +* **Smuggler** is a tool for moving data between databases. + Use it to export documents, indexes, and other items from a database as JSON, + and to import that data into another database. -By default, the `DocumentStore.smuggler` works on the default document store database from the `DocumentStore.database` . +* Smuggler is exposed through the `documentStore.smuggler` property. -In order to switch it to a different database use the `.forDatabase` method. +* An export can write to a `.ravendbdump` file. + An import reads that data back into a database. - - -{`const northwindSmuggler = store +* You can narrow what is transferred by collection, by item type, + and by applying a transform script to each document. + +* In this article: + * [ForDatabase](../../../client-api/smuggler/what-is-smuggler.mdx#fordatabase) + * [Export](../../../client-api/smuggler/what-is-smuggler.mdx#export) + * [Export specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#export-specific-collections) + * [Import](../../../client-api/smuggler/what-is-smuggler.mdx#import) + * [Import specific collections](../../../client-api/smuggler/what-is-smuggler.mdx#import-specific-collections) + * [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript) + * [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) + * [Methods](../../../client-api/smuggler/what-is-smuggler.mdx#methods) + * [Classes](../../../client-api/smuggler/what-is-smuggler.mdx#classes) + + + + + +By default, the smuggler works on the database set in the `documentStore.database` property. +To run the smuggler against a different database, switch to it with the `forDatabase` method. + +```js +const northwindSmuggler = store .smuggler .forDatabase("Northwind"); -`} - - +``` + + -## Export +Export writes the selected data to a `.ravendbdump` file. +Use `DatabaseSmugglerExportOptions` to choose what to include. -### Usage +
- - -{`const operation = await store.smuggler.export(options, toDatabase); +#### Example: export indexes and documents to a file -const operation = await store.smuggler.export(options, toFile); -`} - - +```js +// export only Indexes and Documents to a given file +const options = new DatabaseSmugglerExportOptions(); +options.operateOnTypes = ["Indexes", "Documents"]; -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerExportOptions` | Options that will be used during the export. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | -| **toDatabase** | `DatabaseSmuggler` | `DatabaseSmuggler` instance used as a destination | -| **toFile** | `string` | Path to a file where exported data will be written | - -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | - -### DatabaseSmugglerExportOptions - -| Parameters | | | -| - | - | - | -| **collections** | ` string[]` | List of specific collections to export. If empty, then all collections will be exported. Default: `empty` | -| **operateOnTypes** | `DatabaseItemType[]` | Indicates what should be exported. Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `Identities`, `CompareExchange`, `Subscriptions` | -| **operateOnDatabaseRecordTypes** | `DatabaseRecordItemType[]` | Indicates what should be exported from database record. Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications` | -| **includeExpired** | `boolean` | Should expired documents be included in the export. Default: `true` | -| **includeArtificial** | `boolean` | ? | -| **removeAnalyzers** | `boolean` | Should analyzers be removed from Indexes. Default: `false` | -| **transformScript** | `string` | JavaScript-based script applied to every exported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **maxStepsForTransformScript** | `number` | Maximum number of steps that transform script can process before failing. Default: 10000 | -| **skipRevisionCreation** | `boolean` | skip revision creation | -| **encryptionKey** | `string` | Encryption key used for restore | - -### Example - - - -{`const options = new DatabaseSmugglerExportOptions(); -options.operateOnTypes = ["Documents"]; const operation = await store .smuggler - .export(options, "C:\\\\ravendb-exports\\\\Northwind.ravendbdump"); + .export(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + await operation.waitForCompletion(); -`} - - +``` +
+ -## Import +### Export specific collections -### Usage +By default, all collections are exported. +To export only specific collections, list their names in the `collections` option. - - -{`const operation = await store.smuggler.import(options, fromFile); +```js +// Export only the "Orders" and "Categories" collections to a file +const options = new DatabaseSmugglerExportOptions(); +options.collections = ["Orders", "Categories"]; -const operation = await store.smuggler.import(options, stream); -`} - - +const operation = await store + .smuggler + .export(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + +await operation.waitForCompletion(); +``` + + + +
+ +See [export](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. + + + + -| Parameters | | | -| ------------- | ------------- | ----- | -| **options** | `DatabaseSmugglerImportOptions` | Options that will be used during the import. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | -| **stream** | `Stream` | Stream with data to import | -| **fromFile** | `string` | Path to a file from which data will be imported | - -| Return Value | | -| ------------- | ----- | -| `Operation` | Instance of Operation-class which gives you an ability to wait for the operation to complete and subscribe to operation progress events | - -### DatabaseSmugglerImportOptions - -| Parameters | | | -| - | - | - | -| **operateOnTypes** | `DatabaseItemType[]` | Indicates what should be imported. Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `Identities`, `CompareExchange`, `Subscriptions` | -| **operateOnDatabaseRecordTypes** | `DatabaseRecordItemType[]` | Indicates what should be imported. Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications` | -| **includeExpired** | `boolean` | Should expired documents be included in the import. Default: `true` | -| **includeArtificial** | `boolean` | ? | -| **removeAnalyzers** | `boolean` | Should analyzers be removed from Indexes. Default: `false` | -| **transformScript** | `string` | JavaScript-based script applied to every imported document. Read more [here](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | -| **maxStepsForTransformScript** | `number` | Maximum number of steps that transform script can process before failing. Default: 10000 | -| **skipRevisionCreation** | `boolean` | skip revision creation | -| **encryptionKey** | `string` | Encryption key used for restore | - -### Example - - - -{`const options = new DatabaseSmugglerImportOptions(); +Import reads data from a `.ravendbdump` file and writes it into the database. +Use `DatabaseSmugglerImportOptions` to choose what to include. + +
+ +#### Example: import documents from a file + +```js +// import only Documents from a given file +const options = new DatabaseSmugglerImportOptions(); options.operateOnTypes = ["Documents"]; -const operation = await store.smuggler.import(options, "C:\\\\ravendb-exports\\\\Northwind.ravendbdump"); + +// import the .ravendbdump file that you exported (i.e. in the export example above) +const operation = await store + .smuggler + .import(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + await operation.waitForCompletion(); -`} - - +``` + +
+ + + +### Import specific collections + +By default, all collections are imported. +To import only specific collections, list their names in the `collections` option. + +```js +// Import only the "Orders" and "Categories" collections from the file +const options = new DatabaseSmugglerImportOptions(); +options.collections = ["Orders", "Categories"]; + +const operation = await store + .smuggler + .import(options, "C:\\ravendb-exports\\Northwind.ravendbdump"); + +await operation.waitForCompletion(); +``` + + + +
+See [import](../../../client-api/smuggler/what-is-smuggler.mdx#methods) and [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions) in the [Syntax](../../../client-api/smuggler/what-is-smuggler.mdx#syntax) section. + -## TransformScript + -`TransformScript` exposes the ability to modify or even filter-out the document during the import and export process using the provided JavaScript. +`TransformScript` lets you modify or filter out each document during an export or import, using JavaScript that you provide. -Underneath the JavaScript engine is exactly the same as used for [patching operations](../../../client-api/operations/patching/single-document.mdx) giving you identical syntax and capabilities with additional **ability to filter out documents by throwing a 'skip' exception**. +The JavaScript engine is the same one used for [patching operations](../../../client-api/operations/patching/single-document.mdx), so you have identical syntax and capabilities, plus the ability to filter out a document by throwing a `skip` exception. - - -{`var id = this['@metadata']['@id']; +```javascript +var id = this['@metadata']['@id']; if (id === 'orders/999-A') throw 'skip'; // filter-out this.Freight = 15.3; -`} - +``` + + + + + +## Methods + + + +### Smuggler methods + + + + +Switches the smuggler to operate on a database other than the document store's default database. + +```ts +forDatabase(databaseName: string): DatabaseSmuggler; +``` + +
+ +Usage: + +```js +const northwindSmuggler = store.smuggler.forDatabase("Northwind"); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **databaseName** | `string` | The database that the smuggler will operate on. | + +| Return value | | +|-----------|-------------| +| `DatabaseSmuggler` | A smuggler instance scoped to the specified database. | + +
+ + +Exports the selected data to a file. + +```ts +export(options: DatabaseSmugglerExportOptions, toFile: string): Promise; +``` + +
+ +Usage: + +```js +const operation = await store.smuggler.export(options, toFile); +await operation.waitForCompletion(); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerExportOptions` | Options used during the export. See [DatabaseSmugglerExportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerexportoptions). | +| **toFile** | `string` | Path to the file where exported data is written. | + +| Return value | | +|-----------|-------------| +| `Promise` | An awaiter you can use to wait for the operation to complete and subscribe to progress events. | + +
+ + +Imports data from a file into the database. + +```ts +import(options: DatabaseSmugglerImportOptions, fromFile: string): Promise; +``` + +
+ +Usage: + +```js +const operation = await store.smuggler.import(options, fromFile); +await operation.waitForCompletion(); +``` + +
+ +| Parameter | Type | Description | +|-----------|------|-------------| +| **options** | `DatabaseSmugglerImportOptions` | Options used during the import. See [DatabaseSmugglerImportOptions](../../../client-api/smuggler/what-is-smuggler.mdx#databasesmugglerimportoptions). | +| **fromFile** | `string` | Path to the file the data is imported from. | + +| Return value | | +|-----------|-------------| +| `Promise` | An awaiter you can use to wait for the operation to complete and subscribe to progress events. | +
+
+ +
+ +## Classes + + + +### Smuggler options classes + +
+ + + + + +**Options that control what an export includes.** + +```ts +class DatabaseSmugglerExportOptions +{ + operateOnTypes: DatabaseItemType[] + operateOnDatabaseRecordType: DatabaseRecordItemType[] + includeExpired: boolean + includeArtificial: boolean + includeArchived: boolean + removeAnalyzers: boolean + transformScript: string + maxStepsForTransformScript: number + skipRevisionCreation: boolean + encryptionKey: string + collections: string[] + skipCorruptedData: boolean + compressionAlgorithm?: ExportCompressionAlgorithm +} +``` + +
+ +| Property | Type | Description | +|-----------|------|-------------| +| **collections** | `string[]` | List of specific collections to export. If empty, all collections are exported.
Default: `empty` | +| **operateOnTypes** | `DatabaseItemType[]` | Indicates what should be exported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `ReplicationHubCertificates`, `Identities`, `CompareExchange`, `Attachments`, `CounterGroups`, `Subscriptions`, `TimeSeries` | +| **operateOnDatabaseRecordType** | `DatabaseRecordItemType[]` | Indicates what should be exported from the database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `Settings`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications`, `TimeSeries`, `DocumentsCompression`, `Analyzers`, `LockMode`, `OlapConnectionStrings`, `OlapEtls`, `ElasticSearchConnectionStrings`, `ElasticSearchEtls`, `PostgreSqlIntegration`, `QueueConnectionStrings`, `QueueEtl`, `IndexesHistory`, `Refresh`, `QueueSinks`, `DataArchival` | +| **includeExpired** | `boolean` | Should expired documents be exported.
Default: `true` | +| **includeArtificial** | `boolean` | Should artificial documents be exported.
Default: `false` | +| **includeArchived** | `boolean` | Should archived documents be exported.
Default: `true` | +| **removeAnalyzers** | `boolean` | Should analyzers be removed from indexes.
Default: `false` | +| **transformScript** | `string` | JavaScript applied to every exported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **maxStepsForTransformScript** | `number` | Maximum number of steps the transform script can run before failing.
Default: `10000` | +| **skipRevisionCreation** | `boolean` | When `true`, revisions are not created for the processed documents.
Default: `false` | +| **encryptionKey** | `string` | Encryption key used to encrypt the exported file.
Default: `null` | +| **skipCorruptedData** | `boolean` | When `true`, the export continues if corrupted data is encountered (for example, lost compression dictionaries), recording an error entry in the result report instead of stopping.
Default: `false` | +| **compressionAlgorithm** | `ExportCompressionAlgorithm` | Compression algorithm used for the exported file: `Zstd` or `Gzip`.
Default: `null` | +
+ + +**Options that control what an import includes.** + +```ts +class DatabaseSmugglerImportOptions +{ + operateOnTypes: DatabaseItemType[] + operateOnDatabaseRecordType: DatabaseRecordItemType[] + includeExpired: boolean + includeArtificial: boolean + includeArchived: boolean + removeAnalyzers: boolean + transformScript: string + maxStepsForTransformScript: number + skipRevisionCreation: boolean + encryptionKey: string + collections: string[] + skipCorruptedData: boolean +} +``` + +
+ +| Property | Type | Description | +|-----------|------|-------------| +| **collections** | `string[]` | List of specific collections to import. If empty, all collections are imported.
Default: `empty` | +| **operateOnTypes** | `DatabaseItemType[]` | Indicates what should be imported.
Default: `Indexes`, `Documents`, `RevisionDocuments`, `Conflicts`, `DatabaseRecord`, `ReplicationHubCertificates`, `Identities`, `CompareExchange`, `Attachments`, `CounterGroups`, `Subscriptions`, `TimeSeries` | +| **operateOnDatabaseRecordType** | `DatabaseRecordItemType[]` | Indicates what should be imported into the database record.
Default: `Client`, `ConflictSolverConfig`, `Expiration`, `ExternalReplications`, `PeriodicBackups`, `RavenConnectionStrings`, `RavenEtls`, `Revisions`, `Settings`, `SqlConnectionStrings`, `Sorters`, `SqlEtls`, `HubPullReplications`, `SinkPullReplications`, `TimeSeries`, `DocumentsCompression`, `Analyzers`, `LockMode`, `OlapConnectionStrings`, `OlapEtls`, `ElasticSearchConnectionStrings`, `ElasticSearchEtls`, `PostgreSqlIntegration`, `QueueConnectionStrings`, `QueueEtl`, `IndexesHistory`, `Refresh`, `QueueSinks`, `DataArchival` | +| **includeExpired** | `boolean` | Should expired documents be imported.
Default: `true` | +| **includeArtificial** | `boolean` | Should artificial documents be imported.
Default: `false` | +| **includeArchived** | `boolean` | Should archived documents be imported.
Default: `true` | +| **removeAnalyzers** | `boolean` | Should analyzers be removed from indexes.
Default: `false` | +| **transformScript** | `string` | JavaScript applied to every imported document. See [TransformScript](../../../client-api/smuggler/what-is-smuggler.mdx#transformscript). | +| **maxStepsForTransformScript** | `number` | Maximum number of steps the transform script can run before failing.
Default: `10000` | +| **skipRevisionCreation** | `boolean` | When `true`, revisions are not created for the imported documents.
Default: `false` | +| **encryptionKey** | `string` | Encryption key used to read an encrypted `.ravendbdump` file.
Default: `null` | +| **skipCorruptedData** | `boolean` | When `true`, the import continues if corrupted data is encountered, recording an error entry in the result report instead of stopping.
Default: `false` | +
+
+ +