Lab

Choosing embedding infrastructure through comparable measurements

Short answer

Dedicated processors are not automatically the fastest choice for an embedding engine. The Nawame Lab compared several machines on a defined workload. The lesson depends as much on the measurement conditions as on the resulting ranking.

A queue separates document-chunk processing from a short query’s waiting time.

An embedding represents text as a vector so that similarity can be compared. Generating vectors takes computation, especially when indexing long documents.

When a short query waits behind document chunks already being processed, the user’s experience depends on the queue. Idle processing time is therefore insufficient to describe the service while people are writing or updating memories.

Separating heavy computation from the database makes that competition easier to examine and avoids imposing the same load fluctuations on every function. The separation must still be assessed alongside operating costs and trust boundaries.

Measure a defined workload

The benchmark run on 20 October 2026 used the same embedding engine, model weights and parameters across the compared machines. Its text was synthetic. Several lengths distinguished short queries from more expensive document chunks.

The CPX42 had eight shared virtual CPUs on AMD EPYC Genoa hardware. The CCX23 had four dedicated virtual CPUs on Milan hardware. Resource counts, processor generation and available instructions differed, so the benchmark compares complete configurations rather than isolating a single hardware feature.

For chunks of 2,900 characters, the measured sequential median was 914 milliseconds on the CPX42 and 2,166 milliseconds on the CCX23. Those figures describe one computational operation in this benchmark. They do not measure the total time of a Nawame search.

Keep measured and projected results separate: scope and limits

The candidate machine could be placed under heavier load. Machines serving production were protected through light sequential measurements and pauses between requests. Their published concurrency figures were extrapolations, rather than equivalent concurrent trials.

Presenting every figure as a comparison under identical conditions would erase that important distinction. Sequential measurements can inform a decision without supporting a fixed advantage under every workload.

The benchmark also showed why concurrency does not automatically multiply throughput. When computation already uses most available resources, additional requests can mainly make the queue longer. A short query that is fast at rest may then wait substantially longer before it is processed.

Understand what “shared” implies

No CPU steal was observed on the CPX42 during the measured load window. That observation is not a permanent guarantee. Shared instances depend on their environment, and the hardware conditions may change.

A machine choice therefore needs monitoring for consistency, throughput, errors and relevant hardware changes. The word “dedicated” cannot replace measurement. A strong result from a shared instance cannot replace observation over time either.

The comparison also checked numerical differences between vectors. Representation consistency matters before attributing an improvement to faster processing alone. A faster engine that silently changes the search space would raise a different evaluation question.

A common engine for Nawame

The work concerns the common engine supporting Nawame’s memory and context functions, including Embedded. It does not describe infrastructure intended exclusively for Embedded.

Sometimes the most valuable computation is the work avoided altogether. Reusing unchanged chunks reduces pressure on the queue without making each individual calculation faster. Hardware selection and avoiding redundant work therefore address complementary problems.

The discussion of resilience and full costs considers the service beyond this benchmark. Memory embedded in business software illustrates one category of use that depends on this shared engine. Neither an isolated speed result nor a server label is a complete account of product performance.

Related reading: Limits of retrieval at scale and Why we publish Nawame’s limitations too.

Your next prompt should not have to contain your whole story.

Start building a memory your AI tools can find again when it matters.

Start free