"Data read, but end of buffer not reached [64]" when using structures

[shirecoding]

Im not sure if this is the right way, but when i add the structure methods i get an error

let packr = new Packr({
    useRecords: true,
    // Enable saving/restoring structures
    getStructures() {
        return this.structures || [];
    },
    saveStructures(structures) {
        this.structures = structures;
    },
});

or alternatively

let packr = new Packr({
    useRecords: true,
    structures: [["event", "type", "message"]], // my data is a list of objects
});

231 |                   restoreStructures()
232 |           clearSource()
233 |           if (error instanceof RangeError || error.message.startsWith('Unexpected end of buffer') || position > srcEnd) {
234 |                   error.incomplete = true
235 |           }
236 |           throw error
              ^
error: Data read, but end of buffer not reached [64]
      at checkedRead (/Users/benjaminhon/Developer/christmas/christmas/app/node_modules/msgpackr/unpack.js:236:9)

I am on version: "msgpackr": "^1.11.2",

It works if i remove it. But what i want to acheive is having the records shared across multiple calls to packr.pack(data)

Im not sure why but i suspect that structures only work with dicts and not list of dicts?

[MohammedAlkindi]

This isn't a bug in msgpackr — the structure store in that snippet isn't actually shared, and the error is the expected consequence.

getStructures() { return this.structures || [] } reads structures off the Packr instance, so each instance gets its own store. Pack with one instance and unpack with another and the second one starts from [], so the record ids in the buffer refer to structures it has never seen. That produces exactly the error you saw.

Reproduced both shapes on your version and on current, Windows 11 / Node v24.18.0:

msgpackr 1.11.2
  per-instance store (your shape)  -> FAIL Data read, but end of buffer not reached [64,"a"]
  genuinely shared store           -> ok   [{"event":"a","type":"t","message":"m"}, ...]

msgpackr 2.0.5
  per-instance store (your shape)  -> FAIL Data read, but end of buffer not reached [64,"a"]
  genuinely shared store           -> ok   [{"event":"a","type":"t","message":"m"}, ...]

The only difference between the two rows is where the array lives:

// fails - one store per Packr instance
getStructures() { return this.structures || [] },
saveStructures(s) { this.structures = s }

// works - one store, outside the instances
let shared = []
getStructures() { return shared },
saveStructures(s) { shared = s }

Two other things worth separating out, because they are what made this look like a structures-vs-lists problem:

• A list of objects is fine. Your second config — structures: [["event","type","message"]] with a static array — round-trips correctly on both 1.11.2 and 2.0.5, repeated calls included. So the suspicion at the end of your report, that structures only work with dicts and not a list of dicts, isn't the cause.
• Same-instance reuse is fine too. Packing and unpacking repeatedly through one instance with your original getStructures/saveStructures works on both versions. The failure needs two instances.

The README example stores structures in a file precisely because the store has to outlive any one Packr, and it notes that shape is what makes multi-process use work. this.structures is per-instance, so it is closer to no store at all.

This looks closeable to me unless you want the error message to be more specific about an unknown record id, which is a different request.