Skip to content

fix(cli-v3): ensure worker cleanup on SIGINT/SIGTERM (#2909) #2993

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
deepshekhardas wants to merge 5 commits into from
Closed

fix(cli-v3): ensure worker cleanup on SIGINT/SIGTERM (#2909) #2993

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
023c3fd
fix(cli-v3): ignore engine checks during deployment install to preven…
Feb 2, 2026
93aa053
fix(core): delegate to original console in ConsoleInterceptor to pres…
Feb 2, 2026
8b684e1
fix(cli-v3): authenticate to Docker Hub to prevent rate limits (#2911)
Feb 2, 2026
737ad56
fix(cli-v3): ensure worker cleanup on SIGINT/SIGTERM (#2909)
Feb 2, 2026
c97cbcc
verify: add reproduction scripts and PR details for all major fixes
Feb 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/fix-console-interceptor-2900.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---

Fix: ConsoleInterceptor now delegates to original console methods to preserve log chain when other interceptors (like Sentry) are present. (#2900)
5 changes: 5 additions & 0 deletions .changeset/fix-docker-hub-rate-limit-2911.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@trigger.dev/cli-v3": patch
---

Fix: Native build server failed with Docker Hub rate limits. Added support for checking checking `DOCKER_USERNAME` and `DOCKER_PASSWORD` in environment variables and logging into Docker Hub before building. (#2911)
Copy link
Contributor

@coderabbitai coderabbitai bot Feb 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Fix duplicated word in the changeset entry.

Minor typo (“checking checking”) in user-facing release notes.

📝 Suggested fix 
-Fix: Native build server failed with Docker Hub rate limits. Added support for checking checking `DOCKER_USERNAME` and `DOCKER_PASSWORD` in environment variables and logging into Docker Hub before building. (`#2911`)
+Fix: Native build server failed with Docker Hub rate limits. Added support for checking `DOCKER_USERNAME` and `DOCKER_PASSWORD` in environment variables and logging into Docker Hub before building. (`#2911`)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
Fix: Native build server failed with Docker Hub rate limits. Added support for checking checking `DOCKER_USERNAME` and `DOCKER_PASSWORD` in environment variables and logging into Docker Hub before building. (#2911)
Fix: Native build server failed with Docker Hub rate limits. Added support for checking `DOCKER_USERNAME` and `DOCKER_PASSWORD` in environment variables and logging into Docker Hub before building. (`#2911`)
🤖 Prompt for AI Agents 
In @.changeset/fix-docker-hub-rate-limit-2911.md at line 5, The changeset entry
contains a duplicated word "checking checking" in the release note string; edit
the changeset text to remove the duplicate so it reads "checking
`DOCKER_USERNAME` and `DOCKER_PASSWORD` in environment variables and logging
into Docker Hub before building." Make this single-word fix in the changeset
entry where the phrase appears.

5 changes: 5 additions & 0 deletions .changeset/fix-github-install-node-version-2913.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@trigger.dev/cli-v3": patch
---

Fix: Ignore engine checks during deployment install phase to prevent failure on build server when Node version mismatch exists. (#2913)
5 changes: 5 additions & 0 deletions .changeset/fix-orphaned-workers-2909.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@trigger.dev/cli-v3": patch
---

Fix: `trigger.dev dev` command left orphaned worker processes when exited via Ctrl+C (SIGINT). Added signal handlers to ensure proper cleanup of child processes and lockfiles. (#2909)
86 changes: 36 additions & 50 deletions packages/cli-v3/src/commands/deploy.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -252,7 +252,7 @@ async function _deployCommand(dir: string, options: DeployCommandOptions) {
}

if (!options.skipUpdateCheck) {
await updateTriggerPackages(dir, { ...options }, true, true);
await updateTriggerPackages(dir, { ...options, ignoreEngines: true }, true, true);
}

const cwd = process.cwd();
Expand Down Expand Up @@ -489,9 +489,8 @@ async function _deployCommand(dir: string, options: DeployCommandOptions) {
const version = deployment.version;

const rawDeploymentLink = `${authorization.dashboardUrl}/projects/v3/${resolvedConfig.project}/deployments/${deployment.shortCode}`;
const rawTestLink = `${authorization.dashboardUrl}/projects/v3/${
resolvedConfig.project
}/test?environment=${options.env === "prod" ? "prod" : "stg"}`;
const rawTestLink = `${authorization.dashboardUrl}/projects/v3/${resolvedConfig.project
}/test?environment=${options.env === "prod" ? "prod" : "stg"}`;

const deploymentLink = cliLink("View deployment", rawDeploymentLink);
const testLink = cliLink("Test tasks", rawTestLink);
Expand Down Expand Up @@ -708,8 +707,7 @@ async function _deployCommand(dir: string, options: DeployCommandOptions) {
}
} else {
outro(
`Version ${version} deployed with ${taskCount} detected task${taskCount === 1 ? "" : "s"} ${
isLinksSupported ? `| ${deploymentLink} | ${testLink}` : ""
`Version ${version} deployed with ${taskCount} detected task${taskCount === 1 ? "" : "s"} ${isLinksSupported ? `| ${deploymentLink} | ${testLink}` : ""
}`
);

Expand All @@ -733,18 +731,16 @@ async function _deployCommand(dir: string, options: DeployCommandOptions) {
TRIGGER_VERSION: version,
TRIGGER_DEPLOYMENT_SHORT_CODE: deployment.shortCode,
TRIGGER_DEPLOYMENT_URL: `${authorization.dashboardUrl}/projects/v3/${resolvedConfig.project}/deployments/${deployment.shortCode}`,
TRIGGER_TEST_URL: `${authorization.dashboardUrl}/projects/v3/${
resolvedConfig.project
}/test?environment=${options.env === "prod" ? "prod" : "stg"}`,
TRIGGER_TEST_URL: `${authorization.dashboardUrl}/projects/v3/${resolvedConfig.project
}/test?environment=${options.env === "prod" ? "prod" : "stg"}`,
},
outputs: {
deploymentVersion: version,
workerVersion: version,
deploymentShortCode: deployment.shortCode,
deploymentUrl: `${authorization.dashboardUrl}/projects/v3/${resolvedConfig.project}/deployments/${deployment.shortCode}`,
testUrl: `${authorization.dashboardUrl}/projects/v3/${
resolvedConfig.project
}/test?environment=${options.env === "prod" ? "prod" : "stg"}`,
testUrl: `${authorization.dashboardUrl}/projects/v3/${resolvedConfig.project
}/test?environment=${options.env === "prod" ? "prod" : "stg"}`,
needsPromotion: options.skipPromotion ? "true" : "false",
},
});
Expand Down Expand Up @@ -787,8 +783,7 @@ async function failDeploy(
checkLogsForErrors(logs);

outro(
`${chalkError(`${prefix}:`)} ${
error.message
`${chalkError(`${prefix}:`)} ${error.message
}. Full build logs have been saved to ${logPath}`
);

Expand Down Expand Up @@ -1088,9 +1083,8 @@ async function handleNativeBuildServerDeploy({
const deployment = initializeDeploymentResult.data;

const rawDeploymentLink = `${dashboardUrl}/projects/v3/${config.project}/deployments/${deployment.shortCode}`;
const rawTestLink = `${dashboardUrl}/projects/v3/${config.project}/test?environment=${
options.env === "prod" ? "prod" : "stg"
}`;
const rawTestLink = `${dashboardUrl}/projects/v3/${config.project}/test?environment=${options.env === "prod" ? "prod" : "stg"
}`;

const exposedDeploymentLink = isLinksSupported
? cliLink(chalk.bold(rawDeploymentLink), rawDeploymentLink)
Expand Down Expand Up @@ -1156,8 +1150,7 @@ async function handleNativeBuildServerDeploy({
log.warn(`Failed streaming build logs, open the deployment in the dashboard to view the logs`);

outro(
`Version ${deployment.version} is being deployed ${
isLinksSupported ? `| ${cliLink("View deployment", rawDeploymentLink)}` : ""
`Version ${deployment.version} is being deployed ${isLinksSupported ? `| ${cliLink("View deployment", rawDeploymentLink)}` : ""
}`
);

Expand Down Expand Up @@ -1204,10 +1197,10 @@ async function handleNativeBuildServerDeploy({
level === "error"
? chalk.bold(chalkError(message))
: level === "warn"
? chalkWarning(message)
: level === "debug"
? chalkGrey(message)
: message;
? chalkWarning(message)
: level === "debug"
? chalkGrey(message)
: message;

// We use console.log here instead of clack's logger as the current version does not support changing the line spacing.
// And the logs look verbose with the default spacing.
Expand Down Expand Up @@ -1240,8 +1233,7 @@ async function handleNativeBuildServerDeploy({
log.error("Failed dequeueing build, please try again shortly");

throw new OutroCommandError(
`Version ${deployment.version} ${
isLinksSupported ? `| ${cliLink("View deployment", rawDeploymentLink)}` : ""
`Version ${deployment.version} ${isLinksSupported ? `| ${cliLink("View deployment", rawDeploymentLink)}` : ""
}`
);
}
Expand All @@ -1256,8 +1248,7 @@ async function handleNativeBuildServerDeploy({
}

throw new OutroCommandError(
`Version ${deployment.version} ${
isLinksSupported ? `| ${cliLink("View deployment", rawDeploymentLink)}` : ""
`Version ${deployment.version} ${isLinksSupported ? `| ${cliLink("View deployment", rawDeploymentLink)}` : ""
}`
);
}
Expand All @@ -1283,13 +1274,12 @@ async function handleNativeBuildServerDeploy({
}

outro(
`Version ${deployment.version} was deployed ${
isLinksSupported
? `| ${cliLink("Test tasks", rawTestLink)} | ${cliLink(
"View deployment",
rawDeploymentLink
)}`
: ""
`Version ${deployment.version} was deployed ${isLinksSupported
? `| ${cliLink("Test tasks", rawTestLink)} | ${cliLink(
"View deployment",
rawDeploymentLink
)}`
: ""
}`
);
return process.exit(0);
Expand All @@ -1303,14 +1293,13 @@ async function handleNativeBuildServerDeploy({
chalk.bold(
chalkError(
"Deployment failed" +
(finalDeploymentEvent.message ? `: ${finalDeploymentEvent.message}` : "")
(finalDeploymentEvent.message ? `: ${finalDeploymentEvent.message}` : "")
)
)
);

throw new OutroCommandError(
`Version ${deployment.version} deployment failed ${
isLinksSupported ? `| ${cliLink("View deployment", rawDeploymentLink)}` : ""
`Version ${deployment.version} deployment failed ${isLinksSupported ? `| ${cliLink("View deployment", rawDeploymentLink)}` : ""
}`
);
}
Expand All @@ -1323,14 +1312,13 @@ async function handleNativeBuildServerDeploy({
chalk.bold(
chalkError(
"Deployment timed out" +
(finalDeploymentEvent.message ? `: ${finalDeploymentEvent.message}` : "")
(finalDeploymentEvent.message ? `: ${finalDeploymentEvent.message}` : "")
)
)
);

throw new OutroCommandError(
`Version ${deployment.version} deployment timed out ${
isLinksSupported ? `| ${cliLink("View deployment", rawDeploymentLink)}` : ""
`Version ${deployment.version} deployment timed out ${isLinksSupported ? `| ${cliLink("View deployment", rawDeploymentLink)}` : ""
}`
);
}
Expand All @@ -1343,14 +1331,13 @@ async function handleNativeBuildServerDeploy({
chalk.bold(
chalkError(
"Deployment was canceled" +
(finalDeploymentEvent.message ? `: ${finalDeploymentEvent.message}` : "")
(finalDeploymentEvent.message ? `: ${finalDeploymentEvent.message}` : "")
)
)
);

throw new OutroCommandError(
`Version ${deployment.version} deployment canceled ${
isLinksSupported ? `| ${cliLink("View deployment", rawDeploymentLink)}` : ""
`Version ${deployment.version} deployment canceled ${isLinksSupported ? `| ${cliLink("View deployment", rawDeploymentLink)}` : ""
}`
);
}
Expand All @@ -1369,13 +1356,12 @@ async function handleNativeBuildServerDeploy({
}

outro(
`Version ${deployment.version} ${
isLinksSupported
? `| ${cliLink("Test tasks", rawTestLink)} | ${cliLink(
"View deployment",
rawDeploymentLink
)}`
: ""
`Version ${deployment.version} ${isLinksSupported
? `| ${cliLink("Test tasks", rawTestLink)} | ${cliLink(
"View deployment",
rawDeploymentLink
)}`
: ""
}`
);
return process.exit(0);
Expand Down
30 changes: 23 additions & 7 deletions packages/cli-v3/src/commands/dev.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -171,22 +171,38 @@ export async function devCommand(options: DevCommandOptions) {
);
} else {
logger.log(
`${chalkError("X Error:")} You must login first. Use the \`login\` CLI command.\n\n${
authorization.error
`${chalkError("X Error:")} You must login first. Use the \`login\` CLI command.\n\n${authorization.error
}`
);
}
process.exitCode = 1;
return;
}

let watcher;
let devInstance: Awaited<ReturnType<typeof startDev>> | undefined;

const cleanup = async () => {
if (devInstance) {
await devInstance.stop();
}
};

const signalHandler = async (signal: string) => {
logger.debug(`Received ${signal}, cleaning up...`);
await cleanup();
process.exit(0);
};

try {
const devInstance = await startDev({ ...options, cwd: process.cwd(), login: authorization });
watcher = devInstance.watcher;
process.on("SIGINT", signalHandler);
process.on("SIGTERM", signalHandler);

devInstance = await startDev({ ...options, cwd: process.cwd(), login: authorization });
await devInstance.waitUntilExit();
} finally {
await watcher?.stop();
process.off("SIGINT", signalHandler);
process.off("SIGTERM", signalHandler);
await cleanup();
}
Comment on lines +182 to 206
Copy link
Contributor

@coderabbitai coderabbitai bot Feb 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

# First, let's examine the dev.ts file around the lines in question
cat -n packages/cli-v3/src/commands/dev.ts | sed -n '280,310p'

Repository: triggerdotdev/trigger.dev

Length of output: 857

🏁 Script executed:

# Let's also see the full context of the try-finally block and startDev
cat -n packages/cli-v3/src/commands/dev.ts | sed -n '170,210p'

Repository: triggerdotdev/trigger.dev

Length of output: 1442

🏁 Script executed:

# Search for startDev function definition to understand the full lifecycle
rg -n 'const startDev|function startDev' packages/cli-v3/src/commands/dev.ts -A 30

Repository: triggerdotdev/trigger.dev

Length of output: 1088

🏁 Script executed:

# Check if there's a watcher that might keep the process alive
rg -n 'watcher\|event loop\|setInterval\|setTimeout' packages/cli-v3/src/commands/dev.ts | head -20

Repository: triggerdotdev/trigger.dev

Length of output: 51

🏁 Script executed:

# Search for watchConfig implementation
rg -n 'export.*watchConfig|const watchConfig|function watchConfig' packages/cli-v3/src/ -A 5 --type=ts

Repository: triggerdotdev/trigger.dev

Length of output: 392

🏁 Script executed:

# Check what bootDevSession does
rg -n 'bootDevSession' packages/cli-v3/src/commands/dev.ts -B 2 -A 15

Repository: triggerdotdev/trigger.dev

Length of output: 1399

🏁 Script executed:

# See if there are any imports or setup that might explain the lifecycle
head -50 packages/cli-v3/src/commands/dev.ts

Repository: triggerdotdev/trigger.dev

Length of output: 2316

🏁 Script executed:

# Look for any existing test files or documentation that explains the expected behavior
fd -e test.ts -e spec.ts packages/cli-v3/src/commands/dev | head -5

Repository: triggerdotdev/trigger.dev

Length of output: 51

🏁 Script executed:

# Let's examine the watchConfig implementation in config.ts to see if it sets up file watchers that keep the process alive
cat -n packages/cli-v3/src/config.ts | sed -n '62,150p'

Repository: triggerdotdev/trigger.dev

Length of output: 3206

🏁 Script executed:

# Check the return type of watchConfig to understand what gets returned
rg -n 'return {' packages/cli-v3/src/config.ts | head -10

Repository: triggerdotdev/trigger.dev

Length of output: 148

🏁 Script executed:

# Let's see what the watcher setup looks like - search for chokidar or file watch setup
rg -n 'chokidar\|watch\|FSWatcher' packages/cli-v3/src/config.ts -B 2 -A 2

Repository: triggerdotdev/trigger.dev

Length of output: 51

🏁 Script executed:

# Check devSession to understand what startDevSession returns
rg -n 'export.*startDevSession|function startDevSession' packages/cli-v3/src/dev/devSession.ts -A 20 | head -40

Repository: triggerdotdev/trigger.dev

Length of output: 698

🏁 Script executed:

# Get the full startDevSession function to see what DevSessionInstance contains and what it does
cat -n packages/cli-v3/src/dev/devSession.ts | sed -n '50,150p'

Repository: triggerdotdev/trigger.dev

Length of output: 3562

🏁 Script executed:

# Check the DevSessionInstance type definition
rg -n 'type DevSessionInstance|interface DevSessionInstance' packages/cli-v3/src/dev/devSession.ts -A 10

Repository: triggerdotdev/trigger.dev

Length of output: 274

🏁 Script executed:

# Let's check what startWorkerRuntime returns and if it keeps the process alive
rg -n 'startWorkerRuntime|return {' packages/cli-v3/src/dev/devSession.ts | head -20

Repository: triggerdotdev/trigger.dev

Length of output: 186

🏁 Script executed:

# Look at the end of startDevSession to see what it returns
cat -n packages/cli-v3/src/dev/devSession.ts | tail -100

Repository: triggerdotdev/trigger.dev

Length of output: 3334

The waitUntilExit no-op creates a critical bug: the dev session terminates immediately after starting.

The bundler is started in watch mode (line 200 of devSession.ts, via bundleWorker with watch: true), which keeps the process alive by monitoring file changes. However:

  1. await devInstance.waitUntilExit() (line 291 of dev.ts) is an empty async function that returns immediately
  2. The finally block (lines 202-206) then executes and calls cleanup()
  3. cleanup() calls devInstance.stop(), which stops the bundler and file watchers
  4. The dev session terminates before any user interaction

In normal operation (without signals), the dev session would start and immediately stop, preventing the CLI from functioning. The signal handler flow (SIGINT/SIGTERM → cleanup() → process.exit(0)) would work correctly if a user manually terminates, but the command would not stay alive waiting for that signal.

The waitUntilExit function needs to block indefinitely while the dev session is running, not be a no-op.

🤖 Prompt for AI Agents 
In `@packages/cli-v3/src/commands/dev.ts` around lines 182 - 206, The dev session
is exiting immediately because devInstance.waitUntilExit is a no-op; change
waitUntilExit (the method returned by startDev / defined in devSession.ts) to
return a Promise that only resolves when the session stops (i.e., when
devInstance.stop() is called) or when an external termination signal occurs;
implement this by wiring stop() to resolve a stored Promise (or awaiting an
EventEmitter/AbortSignal) so that await devInstance.waitUntilExit() blocks until
cleanup/signalHandler triggers stop(), ensuring the bundler/watch stays alive
until explicit stop.

}

Expand Down Expand Up @@ -272,7 +288,7 @@ async function startDev(options: StartDevOptions) {

devInstance = await bootDevSession(watcher.config);

const waitUntilExit = async () => {};
const waitUntilExit = async () => { };
Copy link

@devin-ai-integration devin-ai-integration bot Feb 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 waitUntilExit is an empty function that immediately resolves, causing dev command to exit immediately

The waitUntilExit function is defined as an empty async function that immediately resolves.

Click to expand

Bug Mechanism

At packages/cli-v3/src/commands/dev.ts:291, the function is defined as:

const waitUntilExit = async () => { };

This function returns a Promise that immediately resolves to undefined. When called at line 201:

await devInstance.waitUntilExit();

The await completes instantly, then the finally block at lines 202-206 executes, which calls cleanup() that stops the dev instance.

Expected vs Actual Behavior

  • Expected: The waitUntilExit should block/wait indefinitely until the dev session is terminated by a signal or user action.
  • Actual: The function returns immediately, causing the dev command to start and then immediately stop itself.

Impact

The dev command (trigger.dev dev) will start up the worker runtime and then immediately shut it down, making the dev command unusable. Users will see the dev session start briefly and then exit.

Recommendation: The waitUntilExit function should return a Promise that never resolves (or resolves only when signaled to exit). For example:

const waitUntilExit = () => new Promise<void>(() => {});

Or better, create a mechanism to resolve the promise when the dev session should exit.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.


return {
watcher,
Expand Down
Loading