Skip to content

core/rawdb: add QKC raw database support - #49

Open
ping-ke wants to merge 7 commits into
goshard/basefrom
qkc-rawdb
Open

core/rawdb: add QKC raw database support#49
ping-ke wants to merge 7 commits into
goshard/basefrom
qkc-rawdb

Conversation

@ping-ke

@ping-ke ping-ke commented Aug 8, 2026

Copy link
Copy Markdown

Summary

Integrate QuarkChain database support into core/rawdb instead of maintaining a separate qkc/rawdb implementation.

The storage architecture follows these rules:

  • When QKC and geth have the same storage semantics, QKC directly uses the existing core/rawdb functions and keys.
  • QKC-only records are implemented in *_qkc.go files under core/rawdb.
  • Every newly introduced QKC-specific key is namespaced with the global qkc_ prefix:
    qkcPrefix + qkcSpecificPrefix + keyParts...
    
    This prevents QKC schema additions from colliding with prefixes that geth may introduce in the future.
  • Root and minor block data use separate QKC keys, such as qkc_rb<hash> and qkc_mb<hash>.

Moving the QKC accessors into core/rawdb makes core/rawdb depend on qkc/types. The previous qkc/types -> trie dependency would then form the cycle:

core/rawdb -> qkc/types -> trie -> core/rawdb

To remove this cycle, qkc/types.DeriveSha no longer constructs or imports a trie directly. It accepts the core/types.ListHasher interface from its caller. This preserves the goquarkchain/pyquarkchain insertion order while leaving trie construction to the higher-level package.

Changes

  • Add QKC-specific schema and root/minor chain accessors in schema_qkc.go and accessors_chain_qkc.go.
  • Add QKC chain config persistence in accessors_metadata_qkc.go.
  • Add minor-block transaction and receipt lookup support in accessors_indexes_qkc.go, reusing geth's existing lookup key where applicable.
  • Add tests for QKC keys, storage accessors, metadata, and lookup behavior.
  • Update qkc/types.DeriveSha to accept a core/types.ListHasher, removing the trie dependency and the resulting import cycle while preserving QKC hash behavior.

Adding QKC RawDB Data

Future QKC-specific records should follow the same structure.

For example, to add storage for the latest minor headers:

  1. Define its prefix in schema_qkc.go and generate the complete qkc_-namespaced key through qkcKey.
    var latestMinorHeaderPrefixQKC = []byte("lmh")
    
    func qkcLatestMinorHeaderKey(hash common.Hash) []byte {
        return qkcKey(latestMinorHeaderPrefixQKC, hash.Bytes())
    }
  2. Add the required read/write/delete/has accessors in an appropriate *_qkc.go file.
    func HasLatestMinorHeaders(db ethdb.KeyValueReader, hash common.Hash) bool
    func ReadLatestMinorHeaders(db ethdb.KeyValueReader, hash common.Hash) ...
    func WriteLatestMinorHeaders(db ethdb.KeyValueWriter, hash common.Hash, value ...) 
    func DeleteLatestMinorHeaders(db ethdb.KeyValueWriter, hash common.Hash)
  3. Test the exact key and storage behavior.

If geth already provides an accessor with identical semantics and key layout, use it directly instead of adding a QKC wrapper.

Database Compatibility

QKC-specific keys intentionally use the new namespaced schema. Existing goquarkchain database contents are not migrated; nodes are expected to resync.

Testing

go test -count=1 ./core/rawdb ./qkc/types ./qkc/...
go vet ./core/rawdb ./qkc/types

@ping-ke
ping-ke changed the base branch from qkc-3-types-05-blocks to goshard/base August 31, 2026 10:23
@qizhou

qizhou commented Sep 1, 2026

Copy link
Copy Markdown

Quick question: will the old schema.go still be used, or will we completely replace the old one?

@syntrust

syntrust commented Sep 2, 2026

Copy link
Copy Markdown
go test ./qkc/... 
# github.com/ethereum/go-ethereum/qkc/rawdb [github.com/ethereum/go-ethereum/qkc/rawdb.test]
qkc/rawdb/accessors_chain_test.go:88:5: types.NewRootBlockWithHeader(&types.RootBlockHeader{…}).WithBody undefined (type *"github.com/ethereum/go-ethereum/qkc/types".RootBlock has no field or method WithBody)
ok      github.com/ethereum/go-ethereum/qkc     0.381s
ok      github.com/ethereum/go-ethereum/qkc/account     2.889s
ok      github.com/ethereum/go-ethereum/qkc/cluster/conn        1.171s
ok      github.com/ethereum/go-ethereum/qkc/cluster/wire        0.065s
ok      github.com/ethereum/go-ethereum/qkc/common      (cached)
ok      github.com/ethereum/go-ethereum/qkc/common/hexutil      (cached)
ok      github.com/ethereum/go-ethereum/qkc/config      (cached)
?       github.com/ethereum/go-ethereum/qkc/params      [no test files]
FAIL    github.com/ethereum/go-ethereum/qkc/rawdb [build failed]
ok      github.com/ethereum/go-ethereum/qkc/serialize   (cached)
ok      github.com/ethereum/go-ethereum/qkc/shard       6.430s
ok      github.com/ethereum/go-ethereum/qkc/slave       4.688s
?       github.com/ethereum/go-ethereum/qkc/state       [no test files]
ok      github.com/ethereum/go-ethereum/qkc/types       0.071s
FAIL

@qzhodl

qzhodl commented Sep 3, 2026

Copy link
Copy Markdown

I have a similar question to #49 (comment). The existing core/rawdb still contains accessors_trie.go, which is used directly by geth's triedb, while this PR introduces another rawdb package under qkc/rawdb. What is the intended relationship between these two packages? If qkc/rawdb is only responsible for the additional QKC data, how are future callers expected to use it together with core/rawdb?

@ping-ke ping-ke changed the title qkc/rawdb: add qkc chain database accessors core/rawdb: add QKC raw database support Sep 8, 2026
@ping-ke

ping-ke commented Sep 8, 2026

Copy link
Copy Markdown
Author

Quick question: will the old schema.go still be used, or will we completely replace the old one?

When QKC and geth have the same storage semantics, QKC directly uses the existing core/rawdb functions and keys.
QKC-only records are implemented in *_qkc.go files under core/rawdb.

@ping-ke

ping-ke commented Sep 8, 2026

Copy link
Copy Markdown
Author
go test ./qkc/... 
# github.com/ethereum/go-ethereum/qkc/rawdb [github.com/ethereum/go-ethereum/qkc/rawdb.test]
qkc/rawdb/accessors_chain_test.go:88:5: types.NewRootBlockWithHeader(&types.RootBlockHeader{…}).WithBody undefined (type *"github.com/ethereum/go-ethereum/qkc/types".RootBlock has no field or method WithBody)
ok      github.com/ethereum/go-ethereum/qkc     0.381s
ok      github.com/ethereum/go-ethereum/qkc/account     2.889s
ok      github.com/ethereum/go-ethereum/qkc/cluster/conn        1.171s
ok      github.com/ethereum/go-ethereum/qkc/cluster/wire        0.065s
ok      github.com/ethereum/go-ethereum/qkc/common      (cached)
ok      github.com/ethereum/go-ethereum/qkc/common/hexutil      (cached)
ok      github.com/ethereum/go-ethereum/qkc/config      (cached)
?       github.com/ethereum/go-ethereum/qkc/params      [no test files]
FAIL    github.com/ethereum/go-ethereum/qkc/rawdb [build failed]
ok      github.com/ethereum/go-ethereum/qkc/serialize   (cached)
ok      github.com/ethereum/go-ethereum/qkc/shard       6.430s
ok      github.com/ethereum/go-ethereum/qkc/slave       4.688s
?       github.com/ethereum/go-ethereum/qkc/state       [no test files]
ok      github.com/ethereum/go-ethereum/qkc/types       0.071s
FAIL

All related test passed now.

go test -count=1 ./core/rawdb ./qkc/types ./qkc/...
ok      github.com/ethereum/go-ethereum/core/rawdb      9.920s
ok      github.com/ethereum/go-ethereum/qkc/types       0.012s
ok      github.com/ethereum/go-ethereum/qkc     0.155s
ok      github.com/ethereum/go-ethereum/qkc/account     0.556s
ok      github.com/ethereum/go-ethereum/qkc/cluster/conn        1.112s
ok      github.com/ethereum/go-ethereum/qkc/cluster/wire        0.022s
ok      github.com/ethereum/go-ethereum/qkc/common      0.073s
ok      github.com/ethereum/go-ethereum/qkc/common/hexutil      0.016s
ok      github.com/ethereum/go-ethereum/qkc/config      0.154s
?       github.com/ethereum/go-ethereum/qkc/params      [no test files]
ok      github.com/ethereum/go-ethereum/qkc/serialize   0.017s
ok      github.com/ethereum/go-ethereum/qkc/shard       0.570s
ok      github.com/ethereum/go-ethereum/qkc/slave       0.337s
?       github.com/ethereum/go-ethereum/qkc/state       [no test files]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants