Skip to content
VENKAIRequest Early Access

SME Audit Interface — live prototype

What the Semantic Merge Engine sees, measures, and changes.

This is not a mockup of a future product — it is the same engine and the same benchmark corpus published on the home page, reassembled into the interface we run in front of a client. Every number below names the command that reproduces it.

1 · System analysis

The semantic graph, built on this codebase.

5,188files indexed
88,663symbols resolved
300execution flows mapped

$ npx gitnexus analyze · measured 2026-08-04 · GitNexus index of the monorepo Venkai is developed and tested in. Snapshot was 12 commits behind HEAD at read time — re-running the command refreshes it.

2 · Token economics

Classic agent vs. SME, on the same 25 edits.

A classic agent has no name for the part of a file that changed, so it re-emits the whole symbol. SME resolves the anchor by AST and emits the operation instead.

Silent damage rate — autonomous AI agent execution

0.5% vs 53.6%

Silent damage rate (Venkai governed vs direct agent)

Direct un-governed agent execution(silent damage rate)53.6%
Venkai governed execution channel(verified & recorded)0.5%

$ python benchmarks/run_governance_eval.py --cases 100

Measured 2026-08-04. Measures unverified modifications, unauthorized data accesses, and policy violations that execute without raising runtime exceptions.

Secondary metric — token payload efficiency per edit

−95.9%

token reduction per execution

Model rewrites whole file / symbol(raw output tokens)17,553
Venkai governed semantic call(governed delta)713

$ python benchmarks/see_bench.py --limit 25

Measured 2026-08-03. Measured token output reduction when passing governed semantic diffs versus full-file re-generation.

3 · Before / after

One instruction. What each approach actually writes.

Instruction

“The timeout in fetch_manifest is too short for large manifests — make it 30 seconds.”

Classic agenttext search, no discernment
def fetch_manifest(url, retries=3): """Fetch the remote manifest and validate it.""" session = _session_for(url) for attempt in range(retries): try:- resp = session.get(url, timeout=10)+ resp = session.get(url, timeout=30) ...entire symbol re-emitted around it
17 lines · 147 tokens
SMEanchor resolved by AST
try:- resp = session.get(url, timeout=10)+ resp = session.get(url, timeout=30)what the model actually emits:edit("core/manifest.py", "timeout=10", "timeout=30")→ resolved by AST, refused if the anchor is ambiguous, rolled back if it no longer compiles.
1 character · 19 tokens

4 · SME Score

A measured score, not a marketing number.

Computed from the same figures published above — change the evidence, the score changes with it. Two categories are not yet measured; the score is the average of the three that are, not an inflated blend.

97

/ 100

3 of 5 categories measured

  • Change minimality

    Lines touched vs. lines a full-symbol rewrite would emit.

    Measured

  • Token efficiency

    python benchmarks/see_bench.py --limit 25

    Measured

  • Validation success

    25-task corpus, AST re-parse + rollback on failure.

    Measured

  • Regression risk

    The rollback guard exists and runs; no adversarial regression corpus has been scored yet.

    Not yet measured

  • Intent accuracy

    No labeled intent-classification benchmark yet — today the operation is described, not inferred from prose.

    Not yet measured

Category weights and the full formula are documented for the pitch and for a future IP filing — not published here.