Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/foreign-connection-guard.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"slonik": minor
---

fix: `UnexpectedForeignConnectionError` is raised only while the transaction is open. Previously, async work that was started inside of a transaction (e.g. a deferred side effect, or a `commit` / `rollback` event listener) raised the error for any query that it ran on another connection, even after the transaction had ended. Add `runOutsideTransaction(routine)` to run queries on other connections while a transaction is open, e.g. `await runOutsideTransaction(() => pool.query(auditLogQuery))`.
5 changes: 5 additions & 0 deletions .changeset/foreign-connections-option.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"slonik": minor
---

add `foreignConnections` configuration (`"reject"` (default) or `"allow"`), which replaces `dangerouslyAllowForeignConnections`. `dangerouslyAllowForeignConnections: true` keeps working as a deprecated alias of `foreignConnections: "allow"`; `foreignConnections` takes precedence when both are provided.
23 changes: 20 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -342,7 +342,22 @@ This pattern ensures that the transaction is either committed or aborted the mom
> ```
>
> In this example, the query is executed using the `connection` that is not associated with the transaction. This is unsafe because the query is not part of the transaction and will not be rolled back if the transaction is aborted.
> This behaviour can be disabled by setting `dangerouslyAllowForeignConnections` to `true` in the `ClientConfiguration`.
>
> The check applies only while the transaction is open. Async work that is started inside of the transaction but runs after the transaction has been committed or rolled back (e.g. a deferred side effect or a [`commit` event](#transaction-events) listener) can use any connection.
>
> If a query is meant to run outside of the transaction (e.g. an audit log entry that must persist even if the transaction is rolled back), wrap it in `runOutsideTransaction`:
>
> ```ts
> import { runOutsideTransaction } from "slonik";
>
> pool.transaction(async (transactionConnection) => {
> await runOutsideTransaction(async () => {
> await pool.query(sql.typeAlias("void")`INSERT INTO audit_log (event) VALUES ('baz')`);
> });
> });
> ```
>
> The check can be disabled for all queries by setting `foreignConnections` to `"allow"` in the `ClientConfiguration`.

### Protecting against unsafe value interpolation

Expand Down Expand Up @@ -637,8 +652,9 @@ createPool(
* @property captureStackTrace Dictates whether to capture stack trace before executing query. Middlewares access stack trace through query execution context. (Default: false)
* @property connectionRetryLimit Number of times to retry establishing a new connection. (Default: 3)
* @property connectionTimeout Timeout (in milliseconds) after which an error is raised if connection cannot be established. (Default: 5000)
* @property dangerouslyAllowForeignConnections Allow using connections that are not associated with the transaction. (Default: false)
* @property dangerouslyAllowForeignConnections (Deprecated: use foreignConnections: "allow") Allow using connections that are not associated with the transaction. (Default: false)
* @property driverFactory Overrides the default DriverFactory. (Default: "pg" driver factory)
* @property foreignConnections What to do when a query that runs while a transaction is open uses a connection that is not associated with the transaction: "reject" throws `UnexpectedForeignConnectionError`, "allow" runs the query outside of the transaction. See [Protecting against unsafe transaction handling](#protecting-against-unsafe-transaction-handling). (Default: "reject")
* @property gracefulTerminationTimeout Timeout (in milliseconds) that kicks in after a connection with an active query is requested to end. This is the amount of time that is allowed for query to complete before terminating it. (Default: 5000)
* @property idleInTransactionSessionTimeout Timeout (in milliseconds) after which idle clients are closed. Use 'DISABLE_TIMEOUT' constant to disable the timeout. (Default: 60000)
* @property idleTimeout Timeout (in milliseconds) after which idle clients are closed. Use 'DISABLE_TIMEOUT' constant to disable the timeout. (Default: 5000)
Expand All @@ -662,8 +678,9 @@ type ClientConfigurationInput = {
captureStackTrace?: boolean,
connectionRetryLimit?: number,
connectionTimeout?: number | 'DISABLE_TIMEOUT',
dangerouslyAllowForeignConnections?: boolean,
dangerouslyAllowForeignConnections?: boolean, // deprecated, use foreignConnections: 'allow'
driverFactory?: DriverFactory,
foreignConnections?: 'allow' | 'reject',
gracefulTerminationTimeout?: number,
idleInTransactionSessionTimeout?: number | 'DISABLE_TIMEOUT',
idleTimeout?: number | 'DISABLE_TIMEOUT',
Expand Down
111 changes: 66 additions & 45 deletions packages/slonik/src/connectionMethods/transaction.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { bindTransactionConnection } from "../binders/bindTransactionConnection.js";
import { TRANSACTION_ROLLBACK_ERROR_PREFIX } from "../constants.js";
import { transactionContext } from "../contexts/transactionContext.js";
import type { TransactionContext } from "../contexts/transactionContext.js";
import { getPoolClientState } from "../state.js";
import type { DatabaseTransactionEventEmitter, InternalTransactionFunction } from "../types.js";
import { BackendTerminatedError, UnexpectedStateError } from "@slonik/errors";
Expand All @@ -19,8 +20,17 @@ const execTransaction: InternalTransactionFunction = async (
) => {
const poolClientState = getPoolClientState(connection);

// The context created by `transaction` for this transaction.
const context = transactionContext.getStore();

if (!context) {
throw new UnexpectedStateError("Transaction context is required for transaction execution.");
}

await connection.query("START TRANSACTION");

context.open = true;

if (typeof poolClientState.transactionDepth !== "number") {
throw new UnexpectedStateError(
"Cannot execute transaction without knowing the transaction depth.",
Expand Down Expand Up @@ -51,6 +61,9 @@ const execTransaction: InternalTransactionFunction = async (

await connection.query("COMMIT");

// Closed before emitting the event, so that listeners can use other connections.
context.open = false;

if (poolClientState.transactionDepth === 0) {
eventEmitter.emit("commit", {
transactionDepth: poolClientState.transactionDepth,
Expand All @@ -71,6 +84,10 @@ const execTransaction: InternalTransactionFunction = async (
);
}

// Closed before emitting the event, so that listeners can use other connections.
// A retry opens the transaction again (see above).
context.open = false;

eventEmitter.emit("rollback", {
error: error as Error,
transactionDepth: poolClientState.transactionDepth,
Expand Down Expand Up @@ -149,28 +166,49 @@ export const transaction: InternalTransactionFunction = async (
const transactionId = generateUid();
const eventEmitter = new EventEmitter() as DatabaseTransactionEventEmitter;

return transactionContext.run(
{
transactionId,
},
async () => {
const poolClientState = getPoolClientState(connection);
// Opened by `execTransaction` once the transaction starts.
const context: TransactionContext = {
open: false,
transactionId,
};

if (poolClientState.transactionDepth !== null) {
throw new UnexpectedStateError(
"Cannot use the same connection to start a new transaction before completing the last transaction.",
);
}
return transactionContext.run(context, async () => {
const poolClientState = getPoolClientState(connection);

poolClientState.transactionDepth = 0;
poolClientState.transactionId = transactionId;
if (poolClientState.transactionDepth !== null) {
throw new UnexpectedStateError(
"Cannot use the same connection to start a new transaction before completing the last transaction.",
);
}

const log = parentLog.child({
transactionId: poolClientState.transactionId,
});
poolClientState.transactionDepth = 0;
poolClientState.transactionId = transactionId;

try {
return await execTransaction(
const log = parentLog.child({
transactionId: poolClientState.transactionId,
});

try {
return await execTransaction(
log,
connection,
clientConfiguration,
handler,
transactionRetryLimit,
eventEmitter,
transactionId,
);
} catch (error) {
const transactionRetryLimitToUse =
transactionRetryLimit ?? clientConfiguration.transactionRetryLimit;

const shouldRetry =
typeof error.code === "string" &&
error.code.startsWith(TRANSACTION_ROLLBACK_ERROR_PREFIX) &&
transactionRetryLimitToUse > 0;

if (shouldRetry) {
return await retryTransaction(
log,
connection,
clientConfiguration,
Expand All @@ -179,32 +217,15 @@ export const transaction: InternalTransactionFunction = async (
eventEmitter,
transactionId,
);
} catch (error) {
const transactionRetryLimitToUse =
transactionRetryLimit ?? clientConfiguration.transactionRetryLimit;

const shouldRetry =
typeof error.code === "string" &&
error.code.startsWith(TRANSACTION_ROLLBACK_ERROR_PREFIX) &&
transactionRetryLimitToUse > 0;

if (shouldRetry) {
return await retryTransaction(
log,
connection,
clientConfiguration,
handler,
transactionRetryLimit,
eventEmitter,
transactionId,
);
} else {
throw error;
}
} finally {
poolClientState.transactionDepth = null;
poolClientState.transactionId = null;
} else {
throw error;
}
},
);
} finally {
poolClientState.transactionDepth = null;
poolClientState.transactionId = null;

// In case the transaction ended without COMMIT or ROLLBACK, e.g. the connection was terminated.
context.open = false;
}
});
};
25 changes: 22 additions & 3 deletions packages/slonik/src/contexts/transactionContext.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,24 @@
import { AsyncLocalStorage } from "node:async_hooks";

export const transactionContext = new AsyncLocalStorage<{
transactionId: string;
}>();
export type TransactionContext = {
/**
* Whether the transaction is open, i.e. it has started and has not yet been committed
* or rolled back. Async work that is started inside of a transaction inherits the
* transaction's context and can outlive the transaction.
*/
open: boolean;
readonly transactionId: string;
};

export const transactionContext = new AsyncLocalStorage<TransactionContext | undefined>();

/**
* Runs `routine` outside of the surrounding transaction (if any). Queries started by
* `routine` (including by async work that it starts) can use connections that are not
* associated with the transaction without raising `UnexpectedForeignConnectionError`.
* These queries are not part of the transaction: they are not rolled back if the
* transaction is rolled back.
*/
export const runOutsideTransaction = <T>(routine: () => T): T => {
return transactionContext.run(undefined, routine);
};
42 changes: 42 additions & 0 deletions packages/slonik/src/factories/createClientConfiguration.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
import { createClientConfiguration } from "./createClientConfiguration.js";
import { InvalidConfigurationError } from "@slonik/errors";
import test from "ava";

const baseUri = "postgres://user:password@localhost/db";

const makeConfig = (overrides: Record<string, unknown> = {}) =>
createClientConfiguration(baseUri, overrides as never);

test("foreignConnections defaults to reject", (t) => {
t.is(makeConfig().foreignConnections, "reject");
});

test("foreignConnections is passed through", (t) => {
t.is(makeConfig({ foreignConnections: "allow" }).foreignConnections, "allow");
t.is(makeConfig({ foreignConnections: "reject" }).foreignConnections, "reject");
});

test("dangerouslyAllowForeignConnections (deprecated) maps to foreignConnections", (t) => {
t.is(makeConfig({ dangerouslyAllowForeignConnections: true }).foreignConnections, "allow");
t.is(makeConfig({ dangerouslyAllowForeignConnections: false }).foreignConnections, "reject");
});

test("foreignConnections takes precedence over dangerouslyAllowForeignConnections", (t) => {
t.is(
makeConfig({ dangerouslyAllowForeignConnections: true, foreignConnections: "reject" })
.foreignConnections,
"reject",
);
t.is(
makeConfig({ dangerouslyAllowForeignConnections: false, foreignConnections: "allow" })
.foreignConnections,
"allow",
);
});

test("throws InvalidConfigurationError for an unknown foreignConnections value", (t) => {
const error = t.throws(() => makeConfig({ foreignConnections: "join" }));

t.true(error instanceof InvalidConfigurationError);
t.is(error?.message, 'foreignConnections must be "allow" or "reject".');
});
15 changes: 14 additions & 1 deletion packages/slonik/src/factories/createClientConfiguration.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,20 @@ export const createClientConfiguration = (
Object.entries(clientUserConfigurationInput ?? {}).filter(([, v]) => v !== undefined),
);

// `dangerouslyAllowForeignConnections: true` is a deprecated alias of
// `foreignConnections: "allow"`. `foreignConnections` takes precedence when both
// are provided.
const foreignConnections: ClientConfiguration["foreignConnections"] =
clientUserConfigurationInput?.foreignConnections ??
(clientUserConfigurationInput?.dangerouslyAllowForeignConnections === true
? "allow"
: "reject");

const configuration = {
captureStackTrace: false,
connectionRetryLimit: 3,
connectionTimeout: 5_000,
connectionUri,
dangerouslyAllowForeignConnections: false,
gracefulTerminationTimeout: 5_000,
idleInTransactionSessionTimeout: 60_000,
idleTimeout: 5_000,
Expand All @@ -32,6 +40,7 @@ export const createClientConfiguration = (
transactionRetryLimit: 5,
typeParsers,
...definedUserConfiguration,
foreignConnections,
// `maximumPoolSize` and `minimumPoolSize` are deprecated aliases of
// `maxPoolSize` and `minPoolSize`. The new names take precedence when both
// are provided; otherwise we fall back to the deprecated name, then the
Expand All @@ -46,6 +55,10 @@ export const createClientConfiguration = (
0,
};

if (!["allow", "reject"].includes(configuration.foreignConnections)) {
throw new InvalidConfigurationError('foreignConnections must be "allow" or "reject".');
}

if (configuration.maxPoolSize < 1) {
throw new InvalidConfigurationError("maxPoolSize must be equal to or greater than 1.");
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ export const createClientConfiguration = (): ClientConfiguration => {
connectionRetryLimit: 3,
connectionTimeout: 5_000,
connectionUri: "postgres://",
dangerouslyAllowForeignConnections: false,
foreignConnections: "reject",
gracefulTerminationTimeout: 5_000,
idleInTransactionSessionTimeout: 60_000,
idleTimeout: 5_000,
Expand Down
Loading
Loading