Skip to content

fix(memory): prune unbounded server-side caches causing heap growth #4651

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
waleedlatif1 wants to merge 2 commits into from
Closed

fix(memory): prune unbounded server-side caches causing heap growth #4651

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
08c64eb
fix(memory): prune toolSchemaCache and semaphores to prevent heap growth
waleedlatif1 May 18, 2026
6766bd2
fix(memory): prune effectiveEnvCache and instrument cache sizes in te…
waleedlatif1 May 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions apps/sim/lib/copilot/chat/payload.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { isPaid } from '@/lib/billing/plan-helpers'
import { getToolEntry } from '@/lib/copilot/tool-executor/router'
import { getCopilotToolDescription } from '@/lib/copilot/tools/descriptions'
import { isHosted } from '@/lib/core/config/feature-flags'
import { registerCache } from '@/lib/monitoring/cache-registry'
import { buildMothershipToolsForRequest } from '@/lib/mothership/settings/runtime'
import { trackChatUpload } from '@/lib/uploads/contexts/workspace/workspace-file-manager'
import { tools } from '@/tools/registry'
Expand All @@ -21,6 +22,15 @@ type ToolSchemaCacheEntry = {

const toolSchemaCache = new Map<string, ToolSchemaCacheEntry>()

setInterval(() => {
const now = Date.now()
for (const [key, entry] of toolSchemaCache) {
if (entry.expiresAt <= now) toolSchemaCache.delete(key)
}
}, TOOL_SCHEMA_CACHE_TTL_MS).unref()
Comment on lines +25 to +30
Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps Bot May 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 The toolSchemaCache sweep lacks the !entry.promise guard that the parallel effectiveEnvCache sweep in utils.ts correctly includes. When a schema-build request is in-flight and the 30s interval fires before the async work completes (e.g. cold DB connection, slow permission lookup), the promise-only cache entry is deleted. Any new caller arriving after that deletion creates a second parallel fetch for the same key, causing a thundering-herd on exactly the large, expensive computation this cache is meant to deduplicate.

Suggested change
setInterval(() => {
const now = Date.now()
for (const [key, entry] of toolSchemaCache) {
if (entry.expiresAt <= now) toolSchemaCache.delete(key)
}
}, TOOL_SCHEMA_CACHE_TTL_MS).unref()
setInterval(() => {
const now = Date.now()
for (const [key, entry] of toolSchemaCache) {
if (!entry.promise && entry.expiresAt <= now) toolSchemaCache.delete(key)
}
}, TOOL_SCHEMA_CACHE_TTL_MS).unref()

Copy link
Copy Markdown

@cursor cursor Bot May 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing promise guard in toolSchemaCache sweep causes deduplication bypass

Medium Severity

The toolSchemaCache sweep deletes entries solely based on expiresAt <= now, but the effectiveEnvCache sweep correctly guards with !entry.promise && entry.expiresAt <= now. The toolSchemaCache is missing this !entry.promise check. If the async tool-schema build takes longer than 30 seconds (possible under load with multiple DB queries and dynamic imports), the sweep will delete the in-flight promise entry, defeating request deduplication and causing redundant expensive computations.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 6766bd2. Configure here.


registerCache('toolSchemaCache', () => toolSchemaCache.size)

interface BuildPayloadParams {
message: string
workflowId?: string
Expand Down
6 changes: 5 additions & 1 deletion apps/sim/lib/core/async-jobs/backends/database.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ function rowToJob(row: AsyncJobRow): Job {
const inlineAbortControllers = new Map<string, AbortController>()

interface Semaphore {
limit: number
available: number
waiters: Array<() => void>
}
Expand All @@ -46,7 +47,7 @@ const semaphores = new Map<string, Semaphore>()
async function acquireSlot(key: string, limit: number): Promise<void> {
let s = semaphores.get(key)
if (!s) {
s = { available: limit, waiters: [] }
s = { limit, available: limit, waiters: [] }
semaphores.set(key, s)
}
if (s.available > 0) {
Expand All @@ -65,6 +66,9 @@ function releaseSlot(key: string): void {
return
}
s.available += 1
if (s.waiters.length === 0 && s.available === s.limit) {
semaphores.delete(key)
}
}

export class DatabaseJobQueue implements JobQueueBackend {
Expand Down
10 changes: 10 additions & 0 deletions apps/sim/lib/environment/utils.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import {
getAccessibleEnvCredentials,
syncPersonalEnvCredentialsForUser,
} from '@/lib/credentials/environment'
import { registerCache } from '@/lib/monitoring/cache-registry'
import { checkWorkspaceAccess } from '@/lib/workspaces/permissions/utils'

const logger = createLogger('EnvironmentUtils')
Expand All @@ -23,6 +24,15 @@ type EffectiveEnvCacheEntry = {

const effectiveEnvCache = new Map<string, EffectiveEnvCacheEntry>()

setInterval(() => {
const now = Date.now()
for (const [key, entry] of effectiveEnvCache) {
if (!entry.promise && entry.expiresAt <= now) effectiveEnvCache.delete(key)
}
}, EFFECTIVE_ENV_CACHE_TTL_MS).unref()

registerCache('effectiveEnvCache', () => effectiveEnvCache.size)

function getEffectiveEnvCacheKey(userId: string, workspaceId?: string) {
return `${userId}:${workspaceId ?? ''}`
}
Expand Down
13 changes: 13 additions & 0 deletions apps/sim/lib/monitoring/cache-registry.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
const registry = new Map<string, () => number>()

export function registerCache(name: string, getSize: () => number): void {
registry.set(name, getSize)
}

export function getCacheSizes(): Record<string, number> {
const sizes: Record<string, number> = {}
for (const [name, getSize] of registry) {
sizes[name] = getSize()
}
return sizes
}
Comment on lines +7 to +13
Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps Bot May 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 If any registered getSize callback throws (e.g. a future cache with a non-trivial getter), getCacheSizes() bubbles the error up to the logger.info call in memory-telemetry.ts, preventing the entire memory snapshot from being written. Wrapping each call in a try/catch keeps telemetry working even if one getter is broken.

Suggested change
export function getCacheSizes(): Record<string, number> {
const sizes: Record<string, number> = {}
for (const [name, getSize] of registry) {
sizes[name] = getSize()
}
return sizes
}
export function getCacheSizes(): Record<string, number> {
const sizes: Record<string, number> = {}
for (const [name, getSize] of registry) {
try {
sizes[name] = getSize()
} catch {
sizes[name] = -1
}
}
return sizes
}

2 changes: 2 additions & 0 deletions apps/sim/lib/monitoring/memory-telemetry.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

import v8 from 'node:v8'
import { createLogger } from '@sim/logger'
import { getCacheSizes } from '@/lib/monitoring/cache-registry'

const logger = createLogger('MemoryTelemetry', { logLevel: 'INFO' })

Expand Down Expand Up @@ -33,6 +34,7 @@ export function startMemoryTelemetry(intervalMs = 60_000) {
? process.getActiveResourcesInfo().length
: -1,
uptimeMin: Math.round(process.uptime() / 60),
cacheSizes: getCacheSizes(),
})
}, intervalMs)
timer.unref()
Expand Down
Loading