exports/benchmarks
Benchmarks barrel exports
Exposes the BenchmarkAdapter contract (#1960, #1961) and the existing memory/token/adapter-latency benchmark utilities to external consumers.
Classes
LatencySampler
Defined in: packages/nexus-agents/src/benchmarks/benchmark-runner.ts:29
Sample collector for latency measurements.
Constructors
Constructor
new LatencySampler(): LatencySampler;Returns
Methods
end()
end(id): number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-runner.ts:43
End timing and record the sample.
Parameters
id
string
Returns
number
getMetrics()
getMetrics(): LatencyMetrics;Defined in: packages/nexus-agents/src/benchmarks/benchmark-runner.ts:69
Calculate latency metrics from collected samples.
Returns
record()
record(durationMs): void;Defined in: packages/nexus-agents/src/benchmarks/benchmark-runner.ts:62
Record a sample directly.
Parameters
durationMs
number
Returns
void
reset()
reset(): void;Defined in: packages/nexus-agents/src/benchmarks/benchmark-runner.ts:100
Reset collected samples.
Returns
void
start()
start(id): void;Defined in: packages/nexus-agents/src/benchmarks/benchmark-runner.ts:36
Start timing an operation.
Parameters
id
string
Returns
void
MemoryError
Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:89
Error class for memory operations.
Extends
Constructors
Constructor
new MemoryError(message, options?): MemoryError;Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:90
Parameters
message
string
options?
Partial<Omit<{
cause?: Error;
code: ErrorCode;
context?: Record<string, unknown>;
}, "code">>
Returns
Overrides
Properties
cause
readonly cause: Error | undefined;Defined in: packages/nexus-agents/src/core/errors.ts:105
Inherited from
code
readonly code: ErrorCode;Defined in: packages/nexus-agents/src/core/errors.ts:103
Inherited from
context
readonly context: Record<string, unknown> | undefined;Defined in: packages/nexus-agents/src/core/errors.ts:104
Inherited from
message
message: string;Defined in: node_modules/.pnpm/typescript@6.0.3/node_modules/typescript/lib/lib.es5.d.ts:1075
Inherited from
name
name: string;Defined in: node_modules/.pnpm/typescript@6.0.3/node_modules/typescript/lib/lib.es5.d.ts:1074
Inherited from
stack?
optional stack?: string;Defined in: node_modules/.pnpm/typescript@6.0.3/node_modules/typescript/lib/lib.es5.d.ts:1076
Inherited from
stackTraceLimit
static stackTraceLimit: number;Defined in: node_modules/.pnpm/@types+node@25.9.8/node_modules/@types/node/globals.d.ts:67
The Error.stackTraceLimit property specifies the number of stack frames
collected by a stack trace (whether generated by new Error().stack or
Error.captureStackTrace(obj)).
The default value is 10 but may be set to any valid JavaScript number. Changes
will affect any stack trace captured after the value has been changed.
If set to a non-number value, or set to a negative number, stack traces will not capture any frames.
Inherited from
Methods
toJSON()
toJSON(): SerializedError;Defined in: packages/nexus-agents/src/core/errors.ts:121
Serializes the error to a JSON-safe object.
Returns
Inherited from
captureStackTrace()
static captureStackTrace(targetObject, constructorOpt?): void;Defined in: node_modules/.pnpm/@types+node@25.9.8/node_modules/@types/node/globals.d.ts:51
Creates a .stack property on targetObject, which when accessed returns
a string representing the location in the code at which
Error.captureStackTrace() was called.
const myObject = {};
Error.captureStackTrace(myObject);
myObject.stack; // Similar to `new Error().stack`The first line of the trace will be prefixed with
${myObject.name}: ${myObject.message}.
The optional constructorOpt argument accepts a function. If given, all frames
above constructorOpt, including constructorOpt, will be omitted from the
generated stack trace.
The constructorOpt argument is useful for hiding implementation
details of error generation from the user. For instance:
function a() {
b();
}
function b() {
c();
}
function c() {
// Create an error without stack trace to avoid calculating the stack trace twice.
const { stackTraceLimit } = Error;
Error.stackTraceLimit = 0;
const error = new Error();
Error.stackTraceLimit = stackTraceLimit;
// Capture the stack trace above function b
Error.captureStackTrace(error, b); // Neither function c, nor b is included in the stack trace
throw error;
}
a();Parameters
targetObject
object
constructorOpt?
Function
Returns
void
Inherited from
prepareStackTrace()
static prepareStackTrace(err, stackTraces): any;Defined in: node_modules/.pnpm/@types+node@25.9.8/node_modules/@types/node/globals.d.ts:55
Parameters
err
Error
stackTraces
CallSite[]
Returns
any
See
https://v8.dev/docs/stack-trace-api#customizing-stack-traces
Inherited from
Interfaces
AdapterLatencyConfig
Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:30
Configuration for adapter latency benchmarks.
Properties
measurementIterations
readonly measurementIterations: number;Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:34
Number of measured iterations per scenario.
timeoutMs
readonly timeoutMs: number;Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:36
Timeout per operation in milliseconds.
warmupIterations
readonly warmupIterations: number;Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:32
Number of warmup iterations (not measured).
AdapterLatencyResult
Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:110
Complete adapter latency benchmark result.
Properties
environment
readonly environment: BenchmarkEnvironment;Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:114
Environment information.
results
readonly results: readonly AdapterScenarioResult[];Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:116
Per-adapter, per-scenario results.
timestamp
readonly timestamp: string;Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:112
Timestamp of the benchmark run.
totalDurationMs
readonly totalDurationMs: number;Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:118
Total benchmark duration in milliseconds.
AdapterScenarioResult
Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:90
Result for a single adapter + scenario combination.
Properties
adapterName
readonly adapterName: "claude" | "gemini" | "codex" | "opencode";Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:92
CLI adapter name.
errors
readonly errors: readonly string[];Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:104
Error messages from failures.
failureCount
readonly failureCount: number;Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:102
Number of failed iterations.
latency
readonly latency: LatencyMetrics;Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:98
Latency metrics from measured iterations.
scenario
readonly scenario: string;Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:96
Scenario name.
successCount
readonly successCount: number;Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:100
Number of successful iterations.
transport
readonly transport: CliTransport;Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:94
Transport type used.
BenchmarkAdapter
Defined in: packages/nexus-agents/src/benchmarks/adapter.ts:72
Contract every benchmark implementation fulfills.
Type parameters:
TInstance: one task / problem in the benchmark (e.g., a SWE-bench issue)TPrediction: the solver’s output (e.g., a proposed patch)TEvalResult: the evaluator’s verdict (e.g., patch applied + tests passed)
A correct implementation composes as:
loadInstances -> runInstance(each) -> evaluate(each) -> summarize
Example
class SweBenchAdapter implements BenchmarkAdapter<SweIssue, SwePatch, SweEval> {
readonly name = 'swe-bench';
readonly variant = 'lite';
async loadInstances(config) { ... }
async runInstance(inst, ctx) { ... }
async evaluate(inst, pred) { ... }
summarize(results) { ... }
}Type Parameters
TInstance
TInstance
TPrediction
TPrediction
TEvalResult
TEvalResult
Properties
name
readonly name: string;Defined in: packages/nexus-agents/src/benchmarks/adapter.ts:74
Stable identifier (e.g., ‘swe-bench’, ‘humaneval’). Used in CLI routing and reporting.
variant?
readonly optional variant?: string;Defined in: packages/nexus-agents/src/benchmarks/adapter.ts:76
Optional variant within a benchmark family (e.g., ‘lite’ vs ‘verified’).
Methods
evaluate()
evaluate(instance, prediction): Promise<TEvalResult>;Defined in: packages/nexus-agents/src/benchmarks/adapter.ts:85
Evaluate a prediction against ground truth. Returns a benchmark-specific verdict.
Parameters
instance
TInstance
prediction
TPrediction
Returns
Promise<TEvalResult>
isPass()
isPass(result): boolean;Defined in: packages/nexus-agents/src/benchmarks/adapter.ts:88
Determine whether a verdict counts as pass. Keeps pass/fail semantics localized.
Parameters
result
TEvalResult
Returns
boolean
loadInstances()
loadInstances(config): Promise<readonly TInstance[]>;Defined in: packages/nexus-agents/src/benchmarks/adapter.ts:79
Load the benchmark task set from disk/remote. Runs once per invocation.
Parameters
config
Record<string, unknown>
Returns
Promise<readonly TInstance[]>
runInstance()
runInstance(instance, ctx): Promise<TPrediction>;Defined in: packages/nexus-agents/src/benchmarks/adapter.ts:82
Execute the solver on one instance. No evaluation here — just generate the prediction.
Parameters
instance
TInstance
ctx
Returns
Promise<TPrediction>
summarize()
summarize(results, runTimeMs): BenchmarkRunSummary;Defined in: packages/nexus-agents/src/benchmarks/adapter.ts:91
Aggregate instance results into a summary. Should be pure + deterministic.
Parameters
results
readonly TEvalResult[]
runTimeMs
number
Returns
BenchmarkComparison
Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks-helpers.ts:58
Benchmark comparison result.
Properties
baseline
readonly baseline: string;Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks-helpers.ts:59
comparisons
readonly comparisons: readonly OperationComparison[];Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks-helpers.ts:61
current
readonly current: string;Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks-helpers.ts:60
meetsMemZeroTarget
readonly meetsMemZeroTarget: boolean;Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks-helpers.ts:63
overallLatencyChangePercent
readonly overallLatencyChangePercent: number;Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks-helpers.ts:62
BenchmarkConfig
Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:174
Configuration for running benchmarks.
Extended by
Properties
datasetSizes
readonly datasetSizes: readonly number[];Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:176
Dataset sizes to test.
measurementIterations
readonly measurementIterations: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:180
Number of measurement iterations per size.
thresholds
readonly thresholds: BenchmarkThresholds;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:184
Thresholds for pass/fail.
timeoutMs
readonly timeoutMs: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:182
Timeout per operation in milliseconds.
warmupIterations
readonly warmupIterations: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:178
Number of warmup iterations.
BenchmarkEnvironment
Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:138
Benchmark environment information.
Properties
arch
readonly arch: string;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:144
Architecture.
cpuCores
readonly cpuCores: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:148
CPU cores.
cpuModel
readonly cpuModel: string;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:146
CPU model.
nodeVersion
readonly nodeVersion: string;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:140
Node.js version.
platform
readonly platform: string;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:142
Platform.
totalMemory
readonly totalMemory: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:150
Total memory in bytes.
BenchmarkOrchestratorOptions
Defined in: packages/nexus-agents/src/benchmarks/orchestrator.ts:12
Properties
concurrency?
readonly optional concurrency?: number;Defined in: packages/nexus-agents/src/benchmarks/orchestrator.ts:14
Max parallel runInstance calls. Default 1 (serial).
instanceTimeoutMs?
readonly optional instanceTimeoutMs?: number;Defined in: packages/nexus-agents/src/benchmarks/orchestrator.ts:16
Per-instance timeout in ms. Default 300_000 (5 min).
limit?
readonly optional limit?: number;Defined in: packages/nexus-agents/src/benchmarks/orchestrator.ts:18
Limit instances evaluated (useful for smoke runs).
onProgress?
readonly optional onProgress?: (completed, total, label?) => void;Defined in: packages/nexus-agents/src/benchmarks/orchestrator.ts:20
Progress callback.
Parameters
completed
number
total
number
label?
string
Returns
void
signal?
readonly optional signal?: AbortSignal;Defined in: packages/nexus-agents/src/benchmarks/orchestrator.ts:22
Abort the whole run.
BenchmarkReport
Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:40
Complete benchmark report.
Properties
comparison
readonly comparison: BenchmarkComparison | null;Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:44
consolidation
readonly consolidation: ConsolidationBenchmarkResult | null;Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:46
mem0Validation
readonly mem0Validation: readonly ClaimValidation[];Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:47
overallPass
readonly overallPass: boolean;Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:48
suite
readonly suite: BenchmarkSuiteResult | null;Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:43
timestamp
readonly timestamp: string;Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:42
tokenResults
readonly tokenResults: readonly TokenBenchmarkResult[];Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:45
version
readonly version: string;Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:41
BenchmarkRunContext
Defined in: packages/nexus-agents/src/benchmarks/adapter.ts:40
Execution context handed to a runner.
Keep this interface narrow — benchmarks that need more (e.g. access to specific adapters) should take those as constructor args, not widen this.
Properties
onProgress?
readonly optional onProgress?: (completed, total, label?) => void;Defined in: packages/nexus-agents/src/benchmarks/adapter.ts:44
Emit progress updates for long-running benchmarks.
Parameters
completed
number
total
number
label?
string
Returns
void
signal?
readonly optional signal?: AbortSignal;Defined in: packages/nexus-agents/src/benchmarks/adapter.ts:46
Optional abort signal for cancellation.
timeoutMs
readonly timeoutMs: number;Defined in: packages/nexus-agents/src/benchmarks/adapter.ts:42
Per-instance timeout budget in milliseconds.
BenchmarkRunSummary
Defined in: packages/nexus-agents/src/benchmarks/adapter.ts:17
High-level summary of a benchmark run, CLI-printable and JSON-serializable.
Benchmarks that need extra dimensions attach them via metadata.
Properties
metadata
readonly metadata: Record<string, unknown>;Defined in: packages/nexus-agents/src/benchmarks/adapter.ts:31
Benchmark-specific extras (dataset hash, model IDs, etc.).
name
readonly name: string;Defined in: packages/nexus-agents/src/benchmarks/adapter.ts:19
Benchmark name (e.g., ‘swe-bench’).
passed
readonly passed: number;Defined in: packages/nexus-agents/src/benchmarks/adapter.ts:25
Instances whose evaluation reported pass.
passRate
readonly passRate: number;Defined in: packages/nexus-agents/src/benchmarks/adapter.ts:27
passed / total, in [0, 1].
runTimeMs
readonly runTimeMs: number;Defined in: packages/nexus-agents/src/benchmarks/adapter.ts:29
Wall-clock runtime in milliseconds.
total
readonly total: number;Defined in: packages/nexus-agents/src/benchmarks/adapter.ts:23
Total instances attempted.
variant
readonly variant: string | undefined;Defined in: packages/nexus-agents/src/benchmarks/adapter.ts:21
Variant, if applicable (e.g., ‘lite’, ‘verified’).
BenchmarkSuiteResult
Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:120
Complete benchmark suite result.
Properties
component
readonly component: string;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:124
Component being benchmarked.
environment
readonly environment: BenchmarkEnvironment;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:130
Environment information.
name
readonly name: string;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:122
Suite name.
operations
readonly operations: readonly OperationBenchmark[];Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:128
Individual operation benchmarks.
summary
readonly summary: BenchmarkSummary;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:132
Overall summary.
version
readonly version: string;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:126
Version of the component.
BenchmarkSummary
Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:156
Benchmark summary.
Properties
avgP95Latency
readonly avgP95Latency: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:164
Average p95 latency across operations.
failures
readonly failures: readonly string[];Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:168
Failures if any.
overallThroughput
readonly overallThroughput: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:162
Overall throughput.
passed
readonly passed: boolean;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:166
Pass/fail status based on thresholds.
totalDurationMs
readonly totalDurationMs: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:158
Total benchmark duration in milliseconds.
totalOperations
readonly totalOperations: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:160
Total operations run.
BenchmarkThresholds
Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:190
Pass/fail thresholds.
Properties
maxMemoryBytes
readonly maxMemoryBytes: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:196
Maximum acceptable memory usage in bytes.
maxP95LatencyMs
readonly maxP95LatencyMs: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:192
Maximum acceptable p95 latency in milliseconds.
minPrecision?
readonly optional minPrecision?: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:198
Minimum precision for retrieval (0-1).
minRecall?
readonly optional minRecall?: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:200
Minimum recall for retrieval (0-1).
minThroughput
readonly minThroughput: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:194
Minimum acceptable throughput (ops/sec).
ClaimValidation
Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:29
Validation result for a single Mem0 claim.
Properties
actualPercent
readonly actualPercent: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:32
claim
readonly claim: string;Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:30
delta
readonly delta: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:34
met
readonly met: boolean;Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:33
targetPercent
readonly targetPercent: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:31
ConsolidationBenchmarkResult
Defined in: packages/nexus-agents/src/benchmarks/consolidation-benchmark.ts:30
Consolidation benchmark result.
Properties
operations
readonly operations: readonly OperationBenchmark[];Defined in: packages/nexus-agents/src/benchmarks/consolidation-benchmark.ts:31
timestamp
readonly timestamp: string;Defined in: packages/nexus-agents/src/benchmarks/consolidation-benchmark.ts:32
ConsolidationOperation
Defined in: packages/nexus-agents/src/benchmarks/consolidation-benchmark.ts:22
Consolidation operation that can be benchmarked.
Properties
name
readonly name: string;Defined in: packages/nexus-agents/src/benchmarks/consolidation-benchmark.ts:23
run
readonly run: () => Promise<void>;Defined in: packages/nexus-agents/src/benchmarks/consolidation-benchmark.ts:24
Returns
Promise<void>
IContextMemoryBackend
Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:104
Interface for memory backend implementations.
Extended by
Methods
prune()
prune(olderThan): Promise<Result<number, MemoryError>>;Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:132
Remove memories older than the specified date.
Parameters
olderThan
Date
Cutoff date for pruning
Returns
Promise<Result<number, MemoryError>>
Number of entries pruned
retrieve()
retrieve(key): Promise<Result<unknown, MemoryError>>;Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:118
Retrieve a value by key.
Parameters
key
string
The key to look up
Returns
Promise<Result<unknown, MemoryError>>
The value or null if not found
search()
search(query, limit): Promise<Result<MemoryEntry[], MemoryError>>;Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:125
Search memories using full-text search.
Parameters
query
string
Search query string
limit
number
Maximum number of results
Returns
Promise<Result<MemoryEntry[], MemoryError>>
store()
store(
key,
value,
metadata
): Promise<Result<void, MemoryError>>;Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:111
Store a value with associated metadata.
Parameters
key
string
Unique key for the memory
value
unknown
The value to store (must be JSON-serializable)
metadata
Associated metadata
Returns
Promise<Result<void, MemoryError>>
IMemoryBackend
Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:154
The previous name of IContextMemoryBackend.
Deprecated
Renamed in #5142: this package’s context-store contract shared
the name IMemoryBackend with nexus-memory’s unrelated registry contract
(zero overlapping members), so implements IMemoryBackend meant one of two
things depending on the import line. Use IContextMemoryBackend.
An empty EXTENDING interface rather than a type alias, deliberately: a
type alias cannot participate in declaration merging, so a consumer
augmenting IMemoryBackend would have broken. This form is the same
structural type, assignable both ways, and still augmentable — which is
what makes the rename non-breaking (the majority-bar vote on #5142).
Removal is tracked and blocked on the next major; see the issue linked
from the #5142 record. Internal code must not import this name —
no-restricted-imports in eslint.config.js enforces it.
Extends
Methods
prune()
prune(olderThan): Promise<Result<number, MemoryError>>;Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:132
Remove memories older than the specified date.
Parameters
olderThan
Date
Cutoff date for pruning
Returns
Promise<Result<number, MemoryError>>
Number of entries pruned
Inherited from
retrieve()
retrieve(key): Promise<Result<unknown, MemoryError>>;Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:118
Retrieve a value by key.
Parameters
key
string
The key to look up
Returns
Promise<Result<unknown, MemoryError>>
The value or null if not found
Inherited from
IContextMemoryBackend.retrieve
search()
search(query, limit): Promise<Result<MemoryEntry[], MemoryError>>;Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:125
Search memories using full-text search.
Parameters
query
string
Search query string
limit
number
Maximum number of results
Returns
Promise<Result<MemoryEntry[], MemoryError>>
Inherited from
store()
store(
key,
value,
metadata
): Promise<Result<void, MemoryError>>;Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:111
Store a value with associated metadata.
Parameters
key
string
Unique key for the memory
value
unknown
The value to store (must be JSON-serializable)
metadata
Associated metadata
Returns
Promise<Result<void, MemoryError>>
Inherited from
LatencyMetrics
Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:13
Latency percentile metrics.
Properties
max
readonly max: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:17
Maximum latency in milliseconds.
mean
readonly mean: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:19
Mean latency in milliseconds.
min
readonly min: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:15
Minimum latency in milliseconds.
p50
readonly p50: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:21
50th percentile (median) in milliseconds.
p75
readonly p75: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:23
75th percentile in milliseconds.
p90
readonly p90: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:25
90th percentile in milliseconds.
p95
readonly p95: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:27
95th percentile in milliseconds.
p99
readonly p99: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:29
99th percentile in milliseconds.
sampleCount
readonly sampleCount: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:33
Total number of samples.
stdDev
readonly stdDev: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:31
Standard deviation in milliseconds.
LatencyScenario
Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:51
A single scenario to benchmark.
Properties
content
readonly content: string;Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:55
Input prompt content.
maxTokens?
readonly optional maxTokens?: number;Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:59
Max tokens for generation.
name
readonly name: string;Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:53
Scenario name (e.g., ‘simple-prompt’, ‘complex-prompt’).
systemPrompt?
readonly optional systemPrompt?: string;Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:57
Optional system prompt.
MemoryBenchmarkConfig
Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks-helpers.ts:69
Memory benchmark configuration extending base benchmark config.
Extends
Properties
contentSizeBytes
readonly contentSizeBytes: number;Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks-helpers.ts:71
Size of content in bytes.
datasetSizes
readonly datasetSizes: readonly number[];Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:176
Dataset sizes to test.
Inherited from
measurementIterations
readonly measurementIterations: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:180
Number of measurement iterations per size.
Inherited from
BenchmarkConfig.measurementIterations
searchPatterns
readonly searchPatterns: readonly string[];Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks-helpers.ts:75
Search query patterns.
tagsPerEntry
readonly tagsPerEntry: number;Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks-helpers.ts:73
Number of tags per entry.
thresholds
readonly thresholds: BenchmarkThresholds;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:184
Thresholds for pass/fail.
Inherited from
timeoutMs
readonly timeoutMs: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:182
Timeout per operation in milliseconds.
Inherited from
warmupIterations
readonly warmupIterations: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:178
Number of warmup iterations.
Inherited from
BenchmarkConfig.warmupIterations
MemoryEntry
Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:62
A complete memory entry with all fields.
Properties
accessedAt
accessedAt: Date;Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:72
When the entry was last accessed
createdAt
createdAt: Date;Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:70
When the entry was created
key
key: string;Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:64
Unique key for the memory
metadata
metadata: MemoryMetadata;Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:68
Associated metadata
value
value: unknown;Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:66
The stored value (JSON-serializable)
MemoryMetadata
Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:41
Metadata associated with a memory entry.
Properties
importance
importance: MemoryImportance;Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:43
Importance level determining storage strategy
tags?
optional tags?: string[];Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:45
Optional tags for categorization
ttl?
optional ttl?: number;Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:47
Time-to-live in milliseconds (optional)
OperationBenchmark
Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:100
Benchmark result for a single operation type.
Properties
datasetSize
readonly datasetSize: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:104
Dataset size used.
latency
readonly latency: LatencyMetrics;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:106
Latency metrics.
operation
readonly operation: string;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:102
Operation name.
quality?
readonly optional quality?: QualityMetrics;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:112
Quality metrics (for retrieval operations).
resources
readonly resources: ResourceMetrics;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:110
Resource metrics.
throughput
readonly throughput: ThroughputMetrics;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:108
Throughput metrics.
timestamp
readonly timestamp: string;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:114
Timestamp when benchmark was run.
OperationComparison
Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks-helpers.ts:43
Operation comparison result.
Properties
baselineP95
readonly baselineP95: number;Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks-helpers.ts:46
baselineThroughput
readonly baselineThroughput: number;Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks-helpers.ts:49
currentP95
readonly currentP95: number;Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks-helpers.ts:47
currentThroughput
readonly currentThroughput: number;Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks-helpers.ts:50
datasetSize
readonly datasetSize: number;Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks-helpers.ts:45
improved
readonly improved: boolean;Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks-helpers.ts:52
latencyChangePercent
readonly latencyChangePercent: number;Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks-helpers.ts:48
operation
readonly operation: string;Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks-helpers.ts:44
throughputChangePercent
readonly throughputChangePercent: number;Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks-helpers.ts:51
QualityMetrics
Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:65
Quality metrics for retrieval operations.
Properties
f1Score
readonly f1Score: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:71
F1 score: harmonic mean of precision and recall.
failedQueries?
readonly optional failedQueries?: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:80
Queries whose search call returned an error (#5689). Each is scored as zero precision/recall/MRR rather than dropped from the average.
mrr
readonly mrr: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:73
Mean reciprocal rank.
ndcgAtK
readonly ndcgAtK: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:75
Normalized discounted cumulative gain at k.
precision
readonly precision: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:67
Precision: relevant retrieved / total retrieved.
recall
readonly recall: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:69
Recall: relevant retrieved / total relevant.
ReportOptions
Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:54
Options for generating a benchmark report.
Properties
comparison?
readonly optional comparison?: BenchmarkComparison;Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:56
consolidation?
readonly optional consolidation?: ConsolidationBenchmarkResult;Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:58
suite?
readonly optional suite?: BenchmarkSuiteResult;Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:55
tokenResults?
readonly optional tokenResults?: readonly TokenBenchmarkResult[];Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:57
ResourceMetrics
Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:86
Resource usage metrics.
Properties
avgMemoryBytes
readonly avgMemoryBytes: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:90
Average memory usage in bytes.
cpuTimeMs
readonly cpuTimeMs: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:92
CPU time in milliseconds.
dbSizeBytes?
readonly optional dbSizeBytes?: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:94
Database file size in bytes (if applicable).
peakMemoryBytes
readonly peakMemoryBytes: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:88
Peak memory usage in bytes.
ThroughputMetrics
Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:39
Throughput metrics.
Properties
durationMs
readonly durationMs: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:45
Total duration in milliseconds.
opsPerSecond
readonly opsPerSecond: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:41
Operations per second.
totalOps
readonly totalOps: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:43
Total operations completed.
TokenBenchmarkResult
Defined in: packages/nexus-agents/src/benchmarks/token-benchmark.ts:25
Token benchmark result comparing baseline vs memory-optimized retrieval.
Properties
baseline
readonly baseline: TokenMetrics;Defined in: packages/nexus-agents/src/benchmarks/token-benchmark.ts:27
datasetSize
readonly datasetSize: number;Defined in: packages/nexus-agents/src/benchmarks/token-benchmark.ts:26
meetsMemZeroTarget
readonly meetsMemZeroTarget: boolean;Defined in: packages/nexus-agents/src/benchmarks/token-benchmark.ts:30
optimized
readonly optimized: TokenMetrics;Defined in: packages/nexus-agents/src/benchmarks/token-benchmark.ts:28
savingsPercent
readonly savingsPercent: number;Defined in: packages/nexus-agents/src/benchmarks/token-benchmark.ts:29
searchesFailed
readonly searchesFailed: number;Defined in: packages/nexus-agents/src/benchmarks/token-benchmark.ts:36
Search calls that returned an error (#5689). When every search failed the
“optimized” context is empty for the wrong reason, so savingsPercent is
reported as 0 and the target as not met — not as a 100% saving.
TokenMetrics
Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:51
Token usage metrics.
Properties
avgTokensPerOp
readonly avgTokensPerOp: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:59
Average tokens per operation.
inputTokens
readonly inputTokens: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:53
Total input tokens.
outputTokens
readonly outputTokens: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:55
Total output tokens.
totalTokens
readonly totalTokens: number;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:57
Total tokens (input + output).
Type Aliases
BenchmarkOperation
type BenchmarkOperation = () => Promise<void> | void;Defined in: packages/nexus-agents/src/benchmarks/benchmark-runner.ts:145
Benchmark operation function type.
Returns
Promise<void> | void
MemoryImportance
type MemoryImportance = {
HIGH: "high";
LOW: "low";
MEDIUM: "medium";
};Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:25
Importance levels for memory entries.
Properties
HIGH
readonly HIGH: "high" = 'high';Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:28
LOW
readonly LOW: "low" = 'low';Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:26
MEDIUM
readonly MEDIUM: "medium" = 'medium';Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:27
MemoryImportance
type MemoryImportance = typeof MemoryImportance[keyof typeof MemoryImportance];Defined in: packages/nexus-agents/src/context/memory-backend-types.ts:25
Importance levels for memory entries.
Variables
DEFAULT_ADAPTER_LATENCY_CONFIG
const DEFAULT_ADAPTER_LATENCY_CONFIG: AdapterLatencyConfig;Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:42
Default adapter latency benchmark configuration.
DEFAULT_BENCHMARK_CONFIG
const DEFAULT_BENCHMARK_CONFIG: BenchmarkConfig;Defined in: packages/nexus-agents/src/benchmarks/benchmark-types.ts:206
Default benchmark configuration.
DEFAULT_MEMORY_BENCHMARK_CONFIG
const DEFAULT_MEMORY_BENCHMARK_CONFIG: MemoryBenchmarkConfig;Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks.ts:46
Default memory benchmark configuration.
DEFAULT_SCENARIOS
const DEFAULT_SCENARIOS: readonly LatencyScenario[];Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:65
Default scenarios matching issue #694 requirements.
MEM0_TARGETS
const MEM0_TARGETS: {
latencyReductionPercent: 91;
qualityImprovementPercent: 26;
tokenSavingsPercent: 90;
};Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:20
Mem0 claimed targets from arXiv:2504.19413.
Type Declaration
latencyReductionPercent
readonly latencyReductionPercent: 91 = 91;qualityImprovementPercent
readonly qualityImprovementPercent: 26 = 26;tokenSavingsPercent
readonly tokenSavingsPercent: 90 = 90;NOOP_PROGRESS
const NOOP_PROGRESS: BenchmarkRunContext["onProgress"];Defined in: packages/nexus-agents/src/benchmarks/adapter.ts:95
Default no-op progress handler.
Functions
calculateTokenMetrics()
function calculateTokenMetrics(entries, queryCount): TokenMetrics;Defined in: packages/nexus-agents/src/benchmarks/token-benchmark.ts:49
Calculate token metrics for a set of entries.
Parameters
entries
readonly {
content: string;
}[]
queryCount
number
Returns
compareBenchmarks()
function compareBenchmarks(baseline, current): BenchmarkComparison;Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks.ts:281
Compare benchmarks between two backends.
Parameters
baseline
current
Returns
createBenchmarkSummary()
function createBenchmarkSummary(operations, config?): BenchmarkSummary;Defined in: packages/nexus-agents/src/benchmarks/benchmark-runner.ts:227
Create benchmark summary from operations.
Parameters
operations
readonly OperationBenchmark[]
config?
Partial<BenchmarkConfig> = {}
Returns
createDecayOp()
function createDecayOp(name, decayFn): ConsolidationOperation;Defined in: packages/nexus-agents/src/benchmarks/consolidation-benchmark.ts:84
Create a decay operation from a callback.
Parameters
name
string
decayFn
() => Promise<void>
Returns
createPromotionOp()
function createPromotionOp(name, promoteFn): ConsolidationOperation;Defined in: packages/nexus-agents/src/benchmarks/consolidation-benchmark.ts:74
Create a promotion operation from a callback.
Parameters
name
string
promoteFn
() => Promise<void>
Returns
estimateBenchmarkTokens()
function estimateBenchmarkTokens(text): number;Defined in: packages/nexus-agents/src/benchmarks/token-benchmark.ts:42
Estimate token count from text content.
Parameters
text
string
Returns
number
formatAdapterLatencyReport()
function formatAdapterLatencyReport(result): string;Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:289
Format adapter latency results as a markdown report.
Parameters
result
Returns
string
formatBenchmarkReport()
function formatBenchmarkReport(report): string;Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:144
Format a benchmark report as a human-readable string.
Parameters
report
Returns
string
formatBenchmarkResults()
function formatBenchmarkResults(result): string;Defined in: packages/nexus-agents/src/benchmarks/benchmark-runner.ts:292
Format benchmark results for console output.
Parameters
result
Returns
string
formatComparisonResults()
function formatComparisonResults(comparison): string;Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks-helpers.ts:217
Format comparison results as a human-readable string.
Parameters
comparison
Returns
string
generateBenchmarkReport()
function generateBenchmarkReport(options): BenchmarkReport;Defined in: packages/nexus-agents/src/benchmarks/benchmark-report.ts:122
Generate a complete benchmark report.
Parameters
options
Returns
getBenchmarkEnvironment()
function getBenchmarkEnvironment(): BenchmarkEnvironment;Defined in: packages/nexus-agents/src/benchmarks/benchmark-runner.ts:212
Get benchmark environment information.
Returns
runAdapterLatencyBenchmark()
function runAdapterLatencyBenchmark(
adapters,
scenarios?,
config?
): Promise<AdapterLatencyResult>;Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:124
Run latency benchmarks across adapters and scenarios.
Parameters
adapters
readonly ICliAdapter[]
scenarios?
readonly LatencyScenario[] = DEFAULT_SCENARIOS
config?
Partial<AdapterLatencyConfig> = {}
Returns
Promise<AdapterLatencyResult>
runBenchmark()
function runBenchmark<TInstance, TPrediction, TEvalResult>(
adapter,
config,
options?
): Promise<BenchmarkRunSummary>;Defined in: packages/nexus-agents/src/benchmarks/orchestrator.ts:100
Execute one adapter end-to-end. Returns the adapter-produced summary.
Behavioral notes:
- An instance failure (either runInstance or evaluate throws) is captured as a failure count in summary metadata; the run continues.
- Timeouts cancel via AbortController; adapters should honor
ctx.signal.
Type Parameters
TInstance
TInstance
TPrediction
TPrediction
TEvalResult
TEvalResult
Parameters
adapter
BenchmarkAdapter<TInstance, TPrediction, TEvalResult>
config
Record<string, unknown>
options?
BenchmarkOrchestratorOptions = {}
Returns
Promise<BenchmarkRunSummary>
runConsolidationBenchmark()
function runConsolidationBenchmark(operations, config?): Promise<ConsolidationBenchmarkResult>;Defined in: packages/nexus-agents/src/benchmarks/consolidation-benchmark.ts:41
Run consolidation benchmarks on a set of operations.
Measures latency and throughput of promotion, decay, and eviction operations that maintain memory health over time.
Parameters
operations
readonly ConsolidationOperation[]
config?
Partial<MemoryBenchmarkConfig> = {}
Returns
Promise<ConsolidationBenchmarkResult>
runMemoryBenchmarks()
function runMemoryBenchmarks(
backend,
name,
config?
): Promise<BenchmarkSuiteResult>;Defined in: packages/nexus-agents/src/benchmarks/memory-benchmarks.ts:226
Run all memory backend benchmarks.
Parameters
backend
name
string
config?
Partial<MemoryBenchmarkConfig> = {}
Returns
Promise<BenchmarkSuiteResult>
runOperationBenchmark()
function runOperationBenchmark(
operation,
datasetSize,
fn,
config?
): Promise<OperationBenchmark>;Defined in: packages/nexus-agents/src/benchmarks/benchmark-runner.ts:150
Run a single operation benchmark.
Parameters
operation
string
datasetSize
number
fn
config?
Partial<BenchmarkConfig> = {}
Returns
Promise<OperationBenchmark>
runTokenBenchmark()
function runTokenBenchmark(backend, config?): Promise<readonly TokenBenchmarkResult[]>;Defined in: packages/nexus-agents/src/benchmarks/token-benchmark.ts:69
Run token savings benchmark.
Compares baseline (all entries in context) vs optimized (only relevant entries from search) token usage.
Parameters
backend
config?
Partial<MemoryBenchmarkConfig> = {}
Returns
Promise<readonly TokenBenchmarkResult[]>
toSuiteResult()
function toSuiteResult(result): BenchmarkSuiteResult;Defined in: packages/nexus-agents/src/benchmarks/adapter-latency-benchmark.ts:332
Convert adapter latency results to BenchmarkSuiteResult for compatibility with the generic formatBenchmarkResults() function.