Open network: remove the node whitelist, harden consensus behind a fork; 0.9.0

README.md

@@ -93,7 +93,7 @@ Four stages:
 
 - [x] Decrease the threshold of mining pool users 
 
-- [ ] Open the whitelist to achieve complete decentralization
+- [x] Open the whitelist to achieve complete decentralization (0.9.0: no whitelist; see [docs/OPEN_NETWORK.md](docs/OPEN_NETWORK.md))
 
 
 #### Prosperity phase: XDAGJ & DeFi

docs/README_zh.md

@@ -87,7 +87,7 @@ XDAGJ教程可以让您快速加入并体验XDAGJ的钱包及挖矿功能，私
 
 - [ ] 支持XRC协议
 
-- [ ] 降低矿池门槛，逐步开放白名单从而实现完全去中心化
+- [x] 降低矿池门槛，逐步开放白名单从而实现完全去中心化（0.9.0 起不再有节点白名单，见 [docs/OPEN_NETWORK.md](OPEN_NETWORK.md)）
 
 #### 繁荣阶段：XDAGJ & DeFi
 

docs/XDAGJ_Devnet_Tutorial_en.md

@@ -77,7 +77,7 @@
 
 ## Modify pool config
 
-The configuration file is located in `src/main/resources/xdag-devnet.conf`, the specific meaning is as follows, if you do not modify it, the default configuration is enabled. The whitelist of XDAGJ is an optional mode. If the configuration item is empty, all nodes are allowed to join, and only the corresponding ip access is allowed after restriction
+The configuration file is located in `src/main/resources/xdag-devnet.conf`, the specific meaning is as follows, if you do not modify it, the default configuration is enabled. Since 0.9.0 there is no node whitelist: `node.seeds` are the entry points into the network and `node.trustedPeers` are peers the node always keeps; anybody may connect once the open-network fork is in force on the chain (from genesis on a devnet). See `docs/OPEN_NETWORK.md`.
 
 ```yaml
 # Admin Config
@@ -100,7 +100,8 @@ pool.directRation = 5          #Participation reward ratio (1-100)
 node.ip = 127.0.0.1            #The ip used to bind the Full XdagJ Node service
 node.port = 8001               #The port used to bind the Full XdagJ Node service
 node.maxInboundConnectionsPerIp = 8 #The max Inbound Connection used to bind the Full XdagJ Node service
-node.whiteIPs = 127.0.0.1:8001,127.0.0.1:8002 #The white IP list used to bind the Full XdagJ Node service(separated by ‘,’)
+node.seeds = ["127.0.0.1:8001","127.0.0.1:8002"]   # entry points into the network (node.whiteIPs is still read and treated as seeds + trusted peers)
+node.trustedPeers = []                              # peers that are always kept connected and never banned
 
 # Node libp2p Config
 node.libp2p.port = 9001        #The port for libp2p used to bind the Full XdagJ Node service

docs/XDAGJ_Devnet_Tutorial_zh.md

@@ -86,7 +86,7 @@ TODO
 
 ## 修改矿池参数
 
-配置文件位于`src/main/resources/xdag-xxx.conf`，打包后位于程序的根目录下（./pool）具体的含义如下，不修改则启用默认配置。其中XDAGJ的白名单为可选模式，配置项为空则允许所有节点加入，限定后只允许对应的ip接入
+配置文件位于`src/main/resources/xdag-xxx.conf`，打包后位于程序的根目录下（./pool）具体的含义如下，不修改则启用默认配置。0.9.0 起不再有节点白名单：`node.seeds` 是入网的种子节点，`node.trustedPeers` 是始终保持连接的节点；链上开放网络分叉生效后（devnet 从创世起）任何节点都可以接入，见 `docs/OPEN_NETWORK.md`
 
 - xdag-devnet.conf 为例
 
@@ -111,7 +111,8 @@ directRation              # 参与奖励比例(1-100)，默认为 5
 node.ip                                          # 矿池之间相互连接的 ip，默认为 127.0.0.1
 node.port                                      # 矿池之间相互连接的端口，默认为 8001
 node.maxInboundConnectionsPerIp  # 矿池之间允许入站连接数，默认为 8
-node.whiteIPs                               # 白名单列表，可选。在该列表内的 ip 才允许被连接
+node.seeds                                  # 种子节点列表（旧的 node.whiteIPs 仍会被读取，作为种子及可信节点）
+node.trustedPeers                           # 始终保持连接、不会被封禁的节点
 
 # Node RPC Config
 rpc.enabled              # 是否开启 RPC 功能，默认为 true

pom.xml

@@ -6,7 +6,7 @@
 
     <groupId>io.xdag</groupId>
     <artifactId>xdagj</artifactId>
-    <version>0.8.4</version>
+    <version>0.9.0</version>
 
     <properties>
         <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
@@ -16,18 +16,19 @@
         <dist.phase>package</dist.phase>
         <dist.base>${project.basedir}/dist</dist.base>
 
-        <xdagj-crypto.version>0.1.2</xdagj-crypto.version>
+        <xdagj-crypto.version>0.1.5</xdagj-crypto.version>
+        <xdagj-p2p.version>0.1.8</xdagj-p2p.version>
         <xdagj-native-randomx.version>0.2.5</xdagj-native-randomx.version>
-        <netty.version>4.2.1.Final</netty.version>
-        <jackson.version>2.18.6</jackson.version>
+        <netty.version>4.2.9.Final</netty.version>
+        <jackson.version>2.18.9</jackson.version>
         <kryo.version>5.5.0</kryo.version>
         <lombok.version>1.18.38</lombok.version>
         <caffeine.version>3.2.0</caffeine.version>
         <jline.version>3.30.0</jline.version>
         <junit.version>4.13.2</junit.version>
         <mockito.version>5.17.0</mockito.version>
         <system-stubs-jupiter.version>2.1.8</system-stubs-jupiter.version>
-        <log4j.version>2.25.4</log4j.version>
+        <log4j.version>2.25.5</log4j.version>
         <commons-lang3.version>3.18.0</commons-lang3.version>
         <commons-rng.version>1.6</commons-rng.version>
         <commons-collections4.version>4.5.0</commons-collections4.version>
@@ -281,6 +282,32 @@
             <version>${xdagj-crypto.version}</version>
         </dependency>
 
+        <!-- peer-to-peer layer: transport, handshake, discovery -->
+        <dependency>
+            <groupId>io.xdag</groupId>
+            <artifactId>xdagj-p2p</artifactId>
+            <version>${xdagj-p2p.version}</version>
+            <exclusions>
+                <!-- DNS tree publishing (AWS Route53) is not used by the node -->
+                <exclusion>
+                    <groupId>software.amazon.awssdk</groupId>
+                    <artifactId>*</artifactId>
+                </exclusion>
+                <exclusion>
+                    <groupId>dnsjava</groupId>
+                    <artifactId>dnsjava</artifactId>
+                </exclusion>
+                <exclusion>
+                    <groupId>org.slf4j</groupId>
+                    <artifactId>*</artifactId>
+                </exclusion>
+                <exclusion>
+                    <groupId>org.apache.logging.log4j</groupId>
+                    <artifactId>*</artifactId>
+                </exclusion>
+            </exclusions>
+        </dependency>
+
         <dependency>
             <groupId>io.xdag</groupId>
             <artifactId>xdagj-native-randomx</artifactId>

src/main/java/io/xdag/Kernel.java

@@ -41,7 +41,6 @@
 import io.xdag.db.mysql.TransactionHistoryStoreImpl;
 import io.xdag.db.rocksdb.*;
 import io.xdag.net.*;
-import io.xdag.net.message.MessageQueue;
 import io.xdag.net.node.NodeManager;
 import io.xdag.pool.WebSocketServer;
 import io.xdag.pool.PoolAwardManagerImpl;
@@ -74,12 +73,8 @@ public class Kernel {
 
     protected SnapshotStore snapshotStore;
     protected Blockchain blockchain;
-    protected NetDB netDB;
-    protected PeerClient client;
     protected ChannelManager channelMgr;
     protected NodeManager nodeMgr;
-    protected NetDBManager netDBMgr;
-    protected PeerServer p2p;
     protected XdagSync sync;
     protected XdagPow pow;
     private SyncManager syncMgr;
@@ -128,12 +123,9 @@ public synchronized void testStart() {
         isRunning.set(true);
         startEpoch = XdagTime.getCurrentEpoch();
 
-        // Initialize channel manager
+        // The network layer is created now (other components refer to it) and started once the chain and the
+        // synchronisation are in place, so that no peer is talked to before the node can handle it.
         channelMgr = new ChannelManager(this);
-        channelMgr.start();
-
-        netDBMgr = new NetDBManager(this.config);
-        netDBMgr.start();
 
         // Initialize database components
         dbFactory = new RocksdbFactory(this.config);
@@ -158,9 +150,6 @@ public synchronized void testStart() {
             log.info("Transaction History Store init.");
         }
 
-        // Initialize network components
-        netDB = new NetDB();
-
         // Initialize RandomX
         randomx = new RandomX(config);
         randomx.start();
@@ -210,15 +199,6 @@ public synchronized void testStart() {
             xdagState = XdagState.WDST;
         }
 
-        // Initialize P2P networking
-        p2p = new PeerServer(this);
-        p2p.start();
-        client = new PeerClient(this.config, this.coinbase);
-
-        // Initialize node management
-        nodeMgr = new NodeManager(this);
-        nodeMgr.start();
-
         // Initialize synchronization
         sync = new XdagSync(this);
         sync.start();
@@ -231,6 +211,11 @@ public synchronized void testStart() {
         // Initialize mining
         pow = new XdagPow(this);
 
+        // Initialize P2P networking (listening, seeds, discovery) and node management
+        channelMgr.start();
+        nodeMgr = new NodeManager(this);
+        nodeMgr.start();
+
         if (webSocketServer == null) {
             webSocketServer = new WebSocketServer(this, config.getPoolWhiteIPList(), config.getWebsocketServerPort());
         }
@@ -273,13 +258,6 @@ public synchronized void testStop() {
         channelMgr.stop();
         nodeMgr.stop();
 
-        // Close message queue timer
-        MessageQueue.timer.shutdown();
-
-        // Close P2P networking
-        p2p.close();
-        client.close();
-
         // Stop data layer
         blockchain.stopCheckMain();
 

src/main/java/io/xdag/cli/Commands.java

@@ -251,35 +251,29 @@ public String xfer(double sendAmount, Bytes32 address, String remark, double fee
         MutableBytes32 to = MutableBytes32.create();
         to.set(8, address.slice(8, 20));
 
-        // Track remaining amount to send
-        AtomicReference<XAmount> remain = new AtomicReference<>(amount);
-
-        // Collect input accounts
+        // An account transaction spends from exactly one account and carries that account's nonce. (The
+        // former code gathered the amount from several accounts under the nonce of the last one, which no
+        // node accepts.) The default account is tried first, then the others.
         Map<Address, ECKeyPair> ourAccounts = Maps.newHashMap();
-        List<ECKeyPair> accounts = kernel.getWallet().getAccounts();
         UInt64 txNonce = null;
-
+        List<ECKeyPair> accounts = Lists.newArrayList(kernel.getWallet().getAccounts());
+        ECKeyPair defKey = kernel.getWallet().getDefKey();
+        if (defKey != null && accounts.remove(defKey)) {
+            accounts.addFirst(defKey);
+        }
         for (ECKeyPair account : accounts) {
             byte[] addr = toBytesAddress(account).toArray();
             XAmount addrBalance = kernel.getAddressStore().getBalanceByAddress(addr);
-            UInt64 currentTxQuantity = kernel.getAddressStore().getTxQuantity(addr);
-            txNonce = currentTxQuantity.add(UInt64.ONE);
-
-            if (compareAmountTo(remain.get(), addrBalance) <= 0) {
-                ourAccounts.put(new Address(keyPair2Hash(account), XDAG_FIELD_INPUT, remain.get(), true), account);
-                remain.set(XAmount.ZERO);
+            if (compareAmountTo(amount, addrBalance) <= 0) {
+                ourAccounts.put(new Address(keyPair2Hash(account), XDAG_FIELD_INPUT, amount, true), account);
+                txNonce = kernel.getAddressStore().getTxQuantity(addr).add(UInt64.ONE);
                 break;
-            } else {
-                if (compareAmountTo(addrBalance, XAmount.ZERO) > 0) {
-                    remain.set(remain.get().subtract(addrBalance));
-                    ourAccounts.put(new Address(keyPair2Hash(account), XDAG_FIELD_INPUT, addrBalance, true), account);
-                }
             }
         }
 
         // Check if enough balance
-        if (compareAmountTo(remain.get(), XAmount.ZERO) > 0) {
-            return "Balance not enough.";
+        if (ourAccounts.isEmpty()) {
+            return "Balance not enough (no single account holds " + amount.toDecimal(9, XUnit.XDAG).toPlainString() + ").";
         }
 
         // Create and broadcast transaction blocks
@@ -429,7 +423,7 @@ public String stats() {
                           XDAG in address: %s
                         4 hr hashrate KHs: %.9f of %.9f
                         Number of Address: %d""",
-                kernel.getNetDB().getSize(), kernel.getNetDBMgr().getWhiteDB().getSize(),
+                kernel.getChannelMgr().getActiveChannels().size(), xdagStats.getTotalnhosts(),
                 xdagStats.getNblocks(), Math.max(xdagStats.getTotalnblocks(), xdagStats.getNblocks()),
                 xdagStats.getNmain(), Math.max(xdagStats.getTotalnmain(), xdagStats.getNmain()),
                 xdagStats.nextra,