fix(port): plug mp-pool retain and fd/buffer leaks in IPC reply paths

.github/workflows/ci.yml

@@ -394,6 +394,14 @@ jobs:
       ## Tests
       ##
 
+      - name: Build fake_upstream
+        if: matrix.build == 'unit' || matrix.build == 'python-3.11' || matrix.build == 'python-3.12'
+        run: |
+          curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
+          source "$HOME/.cargo/env"
+          cargo build --release --manifest-path test/fake_upstream/Cargo.toml
+          sudo cp test/fake_upstream/target/release/fake_upstream /usr/local/bin/
+
       # /home/runner will be root only after calling sudo above
       # Ensure all users and processes can execute
       - name: Fix permissions

.gitignore

@@ -18,3 +18,8 @@ __pycache__/
 /FIX.md
 /CLAUDE.original.md
 /.qwen/skills/clang-ast/SKILL.md
+/.qwen/.qwen/settings.json
+/.qwen/.qwen/settings.json.orig
+/.qwen/.qwen/skills/clang-ast/SKILL.md
+test/fake_upstream/target/
+test/fake_upstream/Cargo.lock

CHANGES

@@ -1,5 +1,36 @@
 
-Changes with FreeUnit 1.35.4                                     xx xxx 2026
+Changes with FreeUnit 1.35.5                                     29 May 2026
+
+    *) Feature: automatically convert chunked request bodies to Content-Length
+       when forwarding to upstream servers via proxy action. This enables
+       compatibility with backends that do not support Transfer-Encoding:
+       chunked (e.g., Gitea, servers requiring Content-Length). Fixes
+       freeunitorg/freeunit#58, resolves nginx/unit#445 (client chunked),
+       nginx/unit#1088 (duplicate TE), and nginx/unit#1278 (RFC 9112 epic).
+
+    *) Change: chunked_transform feature is no longer experimental. Chunked
+       request bodies can be accepted and transparently converted to
+       Content-Length via configuration: { "settings": { "http":
+       { "chunked_transform": true } } }
+
+    *) Bugfix: fix TLS library busy-loop on peer-initiated close in SSL_write
+       when connection is aborted by remote peer; prevents high CPU usage and
+       ensures proper connection cleanup.
+
+    *) Feature: add unfreeze-sync.sh script for automated migration of issues
+       from nginx/unit to freeunitorg/freeunit with label mapping, deduplication,
+       and dry-run preview support.
+
+    *) Change: upgrade contrib njs to 0.9.8.
+
+    *) Bugfix: fix mem-pool retain leak in cert/script-store IPC paths
+       (router side) and fd/buffer leaks in cert/script/socket/access-log
+       reply paths and the controller config-store path (main process
+       side); all reachable when nxt_port_msg_alloc fails inside the
+       port machinery.
+
+
+Changes with FreeUnit 1.35.4                                     30 Apr 2026
 
     *) Bugfix: fix router process CPU spin and connection hang under port
        scanning load; CLOSE-WAIT sockets are now cleaned up properly on

TODO.md

@@ -135,8 +135,11 @@ Before the OpenSSL 3.6 migration can be considered fully validated:
       `OBJ_sn2nid` / `OpenSSL_version_num` replacements.
 - [ ] Run the full CI matrix (`ci.yml`) and confirm the new "Build OpenSSL 3.6"
       step succeeds on both `amd64` and `arm64` runners.
-- [ ] Confirm `clang-ast` workflow passes end-to-end on `debian:testing`
-      (was broken by `EVP_PKEY_asn1_find_str` / `SSLeay` deprecations).
+- [x] `clang-ast` workflow passes on `debian:testing` + system OpenSSL 1.1
+      via `./test/run-local-full.sh` (verified on `pre-1.35.5` branch).
+- [ ] Confirm `clang-ast` still passes when linked against OpenSSL 3.6
+      (previously broken by `EVP_PKEY_asn1_find_str` / `SSLeay` deprecations
+      — fixes need re-verification on the 3.6 build).
 - [ ] Smoke-test TLS in a Docker image built from `Dockerfile.minimal`
       (now `debian:trixie-slim`) — load a certificate via the REST API and
       make an HTTPS request.
@@ -180,3 +183,56 @@ inside a chroot/rootfs-isolated Unit application.
 1. Run `ldd $(which php)` with PHP 8.5 and compare against the rootfs fixture contents
 2. Check `unit.log` for the full path that caused the segfault (needs core dump or `strace`)
 3. Check if `php 8.5 --define open_basedir=...` reproduces outside of Unit
+
+---
+
+## Test Infrastructure
+
+### Prebuild `fake_upstream` binary via packages.freeunit.org
+
+`test/fake_upstream/` — Rust HTTP mock used by `test_proxy_chunked.py`.
+Currently built from source in Docker (`cargo build --release`), adding ~0.5s per run.
+
+**Improvement:**
+- [ ] Build `fake_upstream` binary and publish to `packages.freeunit.org`
+- [ ] Update `run-local.sh` to download prebuilt binary instead of `cargo build`
+- [ ] Add SHA-512 checksum validation (like `pkg/contrib/Makefile` does for njs/wasmtime)
+- [ ] Fallback to cargo build if download fails
+
+**Benefits:**
+- Faster test image builds
+- Reproducible binaries across platforms (AMD64 + ARM64)
+- No Rust toolchain required in Docker image
+
+---
+
+### clang-ast Docker build: debian:testing + `clang llvm-dev libclang-dev`
+
+`test/run-local-full.sh` builds a Docker image for clang-ast analysis.
+Fixed: use `clang llvm-dev libclang-dev` (not `clang-21 llvm-21-dev libclang-21-dev`).
+
+**Current state:** Works on `debian:testing` (clang 21 + llvm 21).
+
+**Future improvements:**
+- [ ] Prebuild `freeunit-test-full:local` image and publish to GHCR
+- [ ] Or add packages.freeunit.org binary for clang-ast plugin
+- [ ] Cache Docker layers for apt install + clang-ast build
+
+---
+
+## Chunked Encoding (RFC 9112) — Implemented in pre-1.35.5-i58
+
+Branch `pre-1.35.5-i58` implements automatic chunked → Content-Length conversion
+for proxy request forwarding. Key files:
+
+- `src/nxt_h1proto.c` — buffer fix (L1149-1171) + CL injection (L2414-2475)
+- `test/test_proxy_chunked.py` — 10 tests (all passing)
+- `test/fake_upstream/` — Rust HTTP mock with strict CL validation
+
+**Tests:** 10/10 passed ✅
+**clang-ast:** PASSED ✅
+
+**Pending upstream:**
+- Consider making the conversion configurable (currently always-on when `r->chunked`)
+- Add metrics/counter for chunked → CL conversions
+- Consider adding `Transfer-Encoding` removal for HTTP/2 upstream (HTTP/2 doesn't use TE header)

src/nxt_cert.c

@@ -1114,7 +1114,6 @@ nxt_cert_store_get(nxt_task_t *task, nxt_str_t *name, nxt_mp_t *mp,
         goto fail;
     }
 
-    nxt_mp_retain(mp);
     b->completion_handler = nxt_cert_buf_completion;
 
     nxt_buf_cpystr(b, name);
@@ -1138,6 +1137,13 @@ nxt_cert_store_get(nxt_task_t *task, nxt_str_t *name, nxt_mp_t *mp,
         goto fail;
     }
 
+    /*
+     * Retain only after the buffer has been handed off to the port machinery,
+     * so that the failure paths above do not leave the pool with a refcount
+     * that the completion handler can never release.
+     */
+    nxt_mp_retain(mp);
+
     return;
 
 fail:
@@ -1224,8 +1230,21 @@ nxt_cert_store_get_handler(nxt_task_t *task, nxt_port_recv_msg_t *msg)
 
 error:
 
-    (void) nxt_port_socket_write(task, port, type, file.fd,
-                                 msg->port_msg.stream, 0, NULL);
+    if (nxt_port_socket_write(task, port, type, file.fd,
+                              msg->port_msg.stream, 0, NULL)
+        != NXT_OK
+        && file.fd != -1)
+    {
+        /*
+         * On send failure (e.g. malloc failure inside the port machinery)
+         * the port layer never takes ownership of the fd, so close it
+         * here to avoid leaking an open file descriptor in the privileged
+         * main process.  Use nxt_fd_close() rather than nxt_file_close():
+         * file.name has already been freed above and the latter would
+         * dereference it through "%FN" on a close-failure log path.
+         */
+        nxt_fd_close(file.fd);
+    }
 }
 
 

src/nxt_controller.c

@@ -2446,6 +2446,7 @@ nxt_controller_conf_store(nxt_task_t *task, nxt_conf_value_t *conf)
     u_char         *end;
     size_t         size;
     nxt_fd_t       fd;
+    nxt_int_t      rc;
     nxt_buf_t      *b;
     nxt_port_t     *main_port;
     nxt_runtime_t  *rt;
@@ -2479,9 +2480,22 @@ nxt_controller_conf_store(nxt_task_t *task, nxt_conf_value_t *conf)
 
     b->mem.free = nxt_cpymem(b->mem.pos, &size, sizeof(size_t));
 
-    (void) nxt_port_socket_write(task, main_port,
-                                NXT_PORT_MSG_CONF_STORE | NXT_PORT_MSG_CLOSE_FD,
-                                 fd, 0, -1, b);
+    rc = nxt_port_socket_write(task, main_port,
+                               NXT_PORT_MSG_CONF_STORE | NXT_PORT_MSG_CLOSE_FD,
+                               fd, 0, -1, b);
+
+    if (nxt_slow_path(rc != NXT_OK)) {
+        /*
+         * Port layer did not take ownership of fd or b (e.g. malloc
+         * failure inside nxt_port_msg_alloc); close the shm fd and
+         * queue the buffer completion so the engine memory pool is
+         * not left with an unreclaimed buffer.
+         */
+        nxt_fd_close(fd);
+
+        nxt_work_queue_add(&task->thread->engine->fast_work_queue,
+                           b->completion_handler, task, b, b->parent);
+    }
 
     return;
 

src/nxt_h1proto.c

@@ -1149,6 +1149,24 @@ nxt_h1p_conn_request_body_read(nxt_task_t *task, void *obj, void *data)
 
             if (h1p->chunked_parse.last) {
                 body_rest = 0;
+
+            } else if (h1p->chunked_parse.chunk_size > 0) {
+                /* Mid-chunk: chunk_parse consumed the entire buffer but did not
+                 * advance b->mem.pos (CHUNK_MIDDLE path in chunk_buffer).
+                 * Reset so nxt_conn_read has space on the next iteration. */
+                b->mem.free = b->mem.start;
+                b->mem.pos = b->mem.start;
+
+            } else {
+                /* Between chunks: chunk_parse advanced b->mem.pos past all
+                 * framing.  Compact any leftover bytes to the front so
+                 * nxt_conn_read appends after them. */
+                size = (size_t) (b->mem.free - b->mem.pos);
+                if (size > 0) {
+                    nxt_memmove(b->mem.start, b->mem.pos, size);
+                }
+                b->mem.free = b->mem.start + size;
+                b->mem.pos = b->mem.start;
             }
 
         } else {
@@ -2380,6 +2398,7 @@ nxt_h1p_peer_header_send(nxt_task_t *task, nxt_http_peer_t *peer)
     nxt_conn_t          *c;
     nxt_http_field_t    *field;
     nxt_http_request_t  *r;
+    nxt_off_t           content_length;
 
     nxt_debug(task, "h1p peer header send");
 
@@ -2395,9 +2414,34 @@ nxt_h1p_peer_header_send(nxt_task_t *task, nxt_http_peer_t *peer)
            + sizeof("Connection: close\r\n")
            + sizeof("\r\n");
 
+    /* If request body needs Content-Length (e.g., after chunked_transform),
+       calculate it from the buffered body. Empty chunked body (0\r\n\r\n
+       only) leaves r->body == NULL — still emit Content-Length: 0 for
+       backends that require it. */
+    content_length = -1;
+    if (r->chunked) {
+        if (r->body == NULL) {
+            content_length = 0;
+        } else {
+            nxt_buf_t  *b;
+
+            content_length = 0;
+
+            for (b = r->body; b != NULL; b = b->next) {
+                if (nxt_buf_is_file(b)) {
+                    content_length += b->file_end - b->file_pos;
+                } else {
+                    content_length += nxt_buf_mem_used_size(&b->mem);
+                }
+            }
+        }
+        /* Account for Content-Length header size (max off_t length + "Content-Length: \r\n"). */
+        size += nxt_length("Content-Length: ") + NXT_OFF_T_LEN + nxt_length("\r\n");
+    }
+
     nxt_list_each(field, r->fields) {
 
-        if (!field->hopbyhop) {
+        if (!field->hopbyhop && !field->skip) {
             size += field->name_length + field->value_length;
             size += nxt_length(": \r\n");
         }
@@ -2419,7 +2463,7 @@ nxt_h1p_peer_header_send(nxt_task_t *task, nxt_http_peer_t *peer)
 
     nxt_list_each(field, r->fields) {
 
-        if (!field->hopbyhop) {
+        if (!field->hopbyhop && !field->skip) {
             p = nxt_cpymem(p, field->name, field->name_length);
             *p++ = ':'; *p++ = ' ';
             p = nxt_cpymem(p, field->value, field->value_length);
@@ -2428,6 +2472,12 @@ nxt_h1p_peer_header_send(nxt_task_t *task, nxt_http_peer_t *peer)
 
     } nxt_list_loop;
 
+    if (content_length >= 0) {
+        p = nxt_cpymem(p, "Content-Length: ", nxt_length("Content-Length: "));
+        p = nxt_sprintf(p, header->mem.end, "%O", content_length);
+        *p++ = '\r'; *p++ = '\n';
+    }
+
     *p++ = '\r'; *p++ = '\n';
     header->mem.free = p;
     size = p - header->mem.pos;

src/nxt_main_process.c

@@ -1170,8 +1170,30 @@ nxt_main_port_socket_handler(nxt_task_t *task, nxt_port_recv_msg_t *msg)
         type = NXT_PORT_MSG_RPC_ERROR;
     }
 
-    nxt_port_socket_write(task, port, type, ls.socket, msg->port_msg.stream,
-                          0, out);
+    if (nxt_port_socket_write(task, port, type, ls.socket, msg->port_msg.stream,
+                              0, out)
+        != NXT_OK)
+    {
+        /*
+         * ls.socket is -1 unless nxt_main_listening_socket() succeeded.
+         * In that case the port layer did not take ownership, so close it
+         * explicitly.
+         */
+        if (ls.socket != -1) {
+            nxt_socket_close(task, ls.socket);
+        }
+
+        /*
+         * The buffer never reached the port queue, so the port layer will
+         * not run its completion.  Queue the completion to match normal
+         * port-layer cleanup semantics.
+         */
+        if (out != NULL) {
+            nxt_work_queue_add(&task->thread->engine->fast_work_queue,
+                               out->completion_handler, task, out,
+                               out->parent);
+        }
+    }
 }
 
 
@@ -1728,8 +1750,18 @@ nxt_main_port_access_log_handler(nxt_task_t *task, nxt_port_recv_msg_t *msg)
                                  msg->port_msg.reply_port);
 
     if (nxt_fast_path(port != NULL)) {
-        (void) nxt_port_socket_write(task, port, type, file.fd,
-                                     msg->port_msg.stream, 0, NULL);
+        if (nxt_port_socket_write(task, port, type, file.fd,
+                                  msg->port_msg.stream, 0, NULL)
+            != NXT_OK
+            && file.fd != -1)
+        {
+            /*
+             * Port layer never took ownership of the fd (e.g. malloc
+             * failure inside nxt_port_msg_alloc); close it explicitly to
+             * avoid leaking the open file in the main process.
+             */
+            nxt_file_close(task, &file);
+        }
 
     } else {
         nxt_file_close(task, &file);

src/nxt_port.h

@@ -333,6 +333,18 @@ void nxt_port_write_enable(nxt_task_t *task, nxt_port_t *port);
 void nxt_port_write_close(nxt_port_t *port);
 void nxt_port_read_enable(nxt_task_t *task, nxt_port_t *port);
 void nxt_port_read_close(nxt_port_t *port);
+
+/*
+ * Ownership contract:
+ *   On NXT_OK, ownership of fd, fd2, and b transfers to the port layer:
+ *   the port layer will close the descriptor(s) and run b's completion
+ *   handler.
+ *   On any other return, ownership remains with the caller, which is
+ *   responsible for closing fd/fd2 if owned and dispatching b's
+ *   completion handler.
+ *   The inline nxt_port_socket_write() wrapper below inherits this
+ *   contract.
+ */
 nxt_int_t nxt_port_socket_write2(nxt_task_t *task, nxt_port_t *port,
     nxt_uint_t type, nxt_fd_t fd, nxt_fd_t fd2, uint32_t stream,
     nxt_port_id_t reply_port, nxt_buf_t *b);