Repository navigation
Description
The hardening PRs linked from #969 leave a few questions open. They are choices about the intended contract rather than bugs with one obvious fix, so they are collected here instead of being decided inside a PR.
-
Largest var-data length and numInGroup an encoder may write. The generated encoders and the Go struct encoder ([Go] Return an error when a struct-codec var-data length or group count exceeds its type's maximum #1158) stop at
applicableMaxValue, which for an unsigned type without a schemamaxValueis one below the type maximum (254 foruint8), because the type maximum is the null value. The C++ OTF encoder in [C++] Reject a numInGroup or var data length that does not fit its wire field in the OTF encoder #1159 allows the full type range (255 foruint8). Which limit should all encoders use? -
Schema names that are not valid identifiers.
XmlSchemaParser.checkForValidNamereports such names only as warnings, and names read from a.sbeirfile are not checked. [Java] Reject generated file names that would leave the output directory #1181 stops a name from leaving the output directory, but a name likea borx;ystill reaches the generators. Should a name that is invalid in every target language be an error by default? -
Fields newer than the acting version in OTF.
decodeFieldsin the Java and C++ OTF decoders hands every field to the listener, including fields added after the acting version, so a listener that reads them without checking the version reads beyond an older, shorter block. Should the decoders skip those fields, or is checking the version the listener's job? -
Groups whose fields are all constant. Such a group legitimately has blockLength 0, so numInGroup is not bounded by the remaining buffer and the Java DTO, Java OTF and C++ OTF decoders iterate up to the count type's maximum on a few bytes of input. Should numInGroup be capped, for example by the remaining buffer length when blockLength is 0?
-
Decoding with bounds checks turned off. With
agrona.disable.bounds.checks=true(Java) orSBE_NO_BOUNDS_CHECK(C++), several of the new length checks disappear along with the existing ones, including the DTO var-data sizing in [Java] Size DTO var-data byte arrays from a bounds-checked wrap #1151. Is that the intended contract for those switches, or should checks that prevent large allocations stay on? -
Composite members with their own
sinceVersion.IrGeneratorgives a primitive member of a composite used by a field the versionmax(field.sinceVersion, composite.sinceVersion)and drops the member's ownsinceVersion; enum, set and nested composite members keep theirs. For<composite><type name="mantissa" primitiveType="int64"/><type name="exp" primitiveType="int8" sinceVersion="1"/></composite>on a version 0 field, a version 0 producer writes 8 bytes, but the generated accessors readexpunguarded at acting version 0, and the blockLength minimum in [Java] Reject a group blockLength below the fields of the acting version #1153, [Java] Check var-data length and group blockLength in the OTF decoder #1154, [C++] Reject a wire blockLength shorter than the fixed fields of the acting version #1176, [C] Reject a wire blockLength shorter than the fixed fields of the acting version #1177, [C++] Reject a wire blockLength shorter than the fixed fields of the acting version in the OTF decoder #1179, [Java] Reject a message blockLength shorter than the fields of the acting version #1183, [Go] Reject OTF group entries whose blockLength is below the fixed fields #1187 and [Go] Reject generated flyweight group headers whose blockLength is below the fixed fields #1188 is 9, so that valid version 0 block is rejected. Should the IR keep the member'ssinceVersion(which also guards the accessor), or is a membersinceVersionon a primitive not meant to be supported? -
Decoding a length above
maxValue. No decoder rejects a var-data length above the length type'smaxValuetoday, while [Go] Return an error when a struct-codec var-data length or group count exceeds its type's maximum #1158 makes the Go encoder refuse to write one and the C++put<X>(const char *, length)does not check it. Should decoders reject such lengths, and if so only when range checks are enabled, as the Go struct codec's other range checks are? -
E108in the C++ codecs. It is used for "index >= count" and "buffer too short for next group index", and [C++] Reject a wire blockLength shorter than the fixed fields of the acting version #1176 adds "blockLength too short for group". Is one code for all three fine, or should the blockLength check get its own?
Activity