Skip to content
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,8 @@
public interface Document {

String toJson();

default DocumentType getDocumentType() {
return DocumentType.DOCUMENT_STORE;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
package org.hypertrace.core.documentstore;

public enum DocumentType {
SQL_STORE,
DOCUMENT_STORE

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SQL_STORE and DOCUMENT_STORE represent the underlying store types, not exactly the document types.

Can we go with something like
FLAT_STRUCTURE,
NESTED_STRUCTURE,
?

Or, even simply, FLAT, NESTED so that we can call them Flat document and Nested document in our future discussions.

@suddendust suddendust Sep 23, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea behind adding this attribute is to identify whether the document contains annotated values (valueList, valueMap, etc.) in the attributes field/column or not. For documents containing just one column, does it make sense to call them nested or flat? Do you think smth like PLAIN and ANNOTATED would make more sense?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The nesting of valueList, valueMap, etc. is not handled by the document-store itself. It is something maintained by the clients. So, in a true-sense the annotation is the responsibility of the clients and the document-store itself is agnostic of that.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, makes sense. Using FLAT and NESTED now.

}
Original file line number Diff line number Diff line change
Expand Up @@ -11,19 +11,40 @@ public class JSONDocument implements Document {

private static ObjectMapper mapper = new ObjectMapper();
private JsonNode node;
private DocumentType documentType;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid issues like NullPointerException, I'd default it to NESTED.


public JSONDocument(String json) throws IOException {
node = mapper.readTree(json);
}

public JSONDocument(String json, DocumentType documentType) throws IOException {
node = mapper.readTree(json);
this.documentType = documentType;
}

public JSONDocument(Object object) throws IOException {
node = mapper.readTree(mapper.writeValueAsString(object));
}

public JSONDocument(Object object, DocumentType documentType) throws IOException {
node = mapper.readTree(mapper.writeValueAsString(object));
this.documentType = documentType;
}

public JSONDocument(JsonNode node) {
this.node = node;
}

public JSONDocument(JsonNode node, DocumentType documentType) {
this.node = node;
this.documentType = documentType;
}

@Override
public DocumentType getDocumentType() {
return this.documentType;
}

@Override
public String toJson() {
try {
Expand All @@ -39,6 +60,12 @@ public static JSONDocument errorDocument(String message) {
return new JSONDocument(objectNode);
}

public static JSONDocument errorDocument(String message, DocumentType documentType) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please explain why this is significant?

@suddendust suddendust Sep 23, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returning the type of an error json doc indicates that that error was encountered while generating a doc of this particular type. Callers can use this information in future.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's not needed for feature parity, I'd keep this change separate (and probably defer it until it's needed).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed.

ObjectNode objectNode = mapper.createObjectNode();
objectNode.put("errorMessage", message);
return new JSONDocument(objectNode, documentType);
}

@Override
public String toString() {
return toJson();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,12 @@
import org.bson.json.JsonMode;
import org.bson.json.JsonWriterSettings;
import org.hypertrace.core.documentstore.Document;
import org.hypertrace.core.documentstore.DocumentType;
import org.hypertrace.core.documentstore.JSONDocument;
import org.hypertrace.core.documentstore.model.options.ReturnDocumentType;

public final class MongoUtils {

public static final String FIELD_SEPARATOR = ".";
public static final String PREFIX = "$";
private static final String LITERAL = PREFIX + "literal";
Expand Down Expand Up @@ -137,10 +139,10 @@ public static Document dbObjectToDocument(BasicDBObject dbObject) {
jsonString = dbObject.toJson(relaxed);
JsonNode jsonNode = MAPPER.readTree(jsonString);
JsonNode decodedJsonNode = recursiveClone(jsonNode, MongoUtils::decodeKey, identity());
return new JSONDocument(decodedJsonNode);
return new JSONDocument(decodedJsonNode, DocumentType.DOCUMENT_STORE);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once you start having a default, these changes are redundant.

} catch (IOException e) {
// throwing exception is not very useful here.
return JSONDocument.errorDocument(e.getMessage());
return JSONDocument.errorDocument(e.getMessage(), DocumentType.DOCUMENT_STORE);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@
import org.hypertrace.core.documentstore.Collection;
import org.hypertrace.core.documentstore.CreateResult;
import org.hypertrace.core.documentstore.Document;
import org.hypertrace.core.documentstore.DocumentType;
import org.hypertrace.core.documentstore.Filter;
import org.hypertrace.core.documentstore.JSONDocument;
import org.hypertrace.core.documentstore.Key;
Expand Down Expand Up @@ -1276,7 +1277,7 @@ public Document next() {
} catch (IOException | SQLException e) {
System.out.println("prepare document failed!");
closeResultSet();
return JSONDocument.errorDocument(e.getMessage());
return JSONDocument.errorDocument(e.getMessage(), DocumentType.SQL_STORE);
}
}

Expand All @@ -1299,7 +1300,7 @@ protected Document prepareDocument() throws SQLException, IOException {
jsonNode.remove(DOCUMENT_ID);
}

return new JSONDocument(MAPPER.writeValueAsString(jsonNode));
return new JSONDocument(MAPPER.writeValueAsString(jsonNode), DocumentType.SQL_STORE);
}

private void addColumnToJsonNode(
Expand Down Expand Up @@ -1430,7 +1431,7 @@ public Document next() {
return prepareDocument();
} catch (IOException | SQLException e) {
closeResultSet();
return JSONDocument.errorDocument(e.getMessage());
return JSONDocument.errorDocument(e.getMessage(), DocumentType.SQL_STORE);
}
}

Expand All @@ -1447,7 +1448,7 @@ protected Document prepareDocument() throws SQLException, IOException {
jsonNode.put(CREATED_AT, String.valueOf(createdAt));
jsonNode.put(UPDATED_AT, String.valueOf(updatedAt));

return new JSONDocument(MAPPER.writeValueAsString(jsonNode));
return new JSONDocument(MAPPER.writeValueAsString(jsonNode), DocumentType.SQL_STORE);
}

protected void closeResultSet() {
Expand Down Expand Up @@ -1508,7 +1509,7 @@ protected Document prepareDocument() throws SQLException, IOException {
}
}
}
return new JSONDocument(MAPPER.writeValueAsString(jsonNode));
return new JSONDocument(MAPPER.writeValueAsString(jsonNode), DocumentType.SQL_STORE);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same everywhere.

}

private String getColumnValue(
Expand Down