A decision and its reason
The heuristic layer returns both a model tier and the rule behind the decision. A continuation can inherit previous context.
05 / Archive · Local AI research
Local Agent RouterThe right task. The right model.
Before moving to Hermes, I explored how to choose a local AI model for a task and work within the computer’s memory limits. These are the results of those experiments.
EARLIER EXPERIMENT
First, find out what works.
Before my current Hermes workflow, I explored how to choose a local model for a task and the memory available. The project separated request routing, agent execution and model serving so that each could be measured independently.
This page preserves the experiment's results. I now handle current tasks in a customised Hermes with Kanban. These router figures therefore do not measure the efficiency of my current Hermes setup.
- 01
Decide
Which model tier fits the request, and why?
- 02
Do the work
Can the selected model use the tools provided by that runtime?
- 03
Check the result
Does the modified workspace pass a check that could also have failed?
SAVED MEASUREMENTS & EXPLANATIONS
Explore the preserved measurements.
Historical records and a current recalculation.
No model calls.
LOCAL AGENT ROUTER / INTERACTIVE DEMOSAVED MEASUREMENTS · NO SIGN-IN
Loading the demo… A text overview is available below.
What you can try in this demo
- Models and runtimes — The same tasks, two agent runtimes and historical timings.
- Router decisions — A fresh classifier calculation alongside a simple baseline strategy.
- Example decision — Prepared queries with a decision and reason from the actual router function.
TECHNICAL DECISIONS
The classifier, model and runtime are three separate questions.
Memory as a launch condition
Geometry from the GGUF header feeds into the VRAM budget. The calculated estimate is kept separate from actual measured memory use.
My own evaluation procedure
The classifier, model with its runtime, and work produced in the test workspace are evaluated separately.
A result needs a comparison
I show classifier accuracy alongside a strategy that always chooses the most common class.
Errors have different consequences
Demanding work sent to a small model and simple work sent to a larger model are reported separately.
One series does not decide a winner
The historical tests have a single run per task, a fixed order and known limitations in their checks. They do not form a statistical ranking.
My contribution
Design of the local experiment, connecting the model serving stack, routing and verification through the same synthetic tasks.
Project and demo status
The original result JSONL files were aggregated again. The classifier itself was recalculated on both preserved datasets. Each view states its scope, baseline strategy and limits on what the result can tell us.
EVIDENCE AND LIMITS
The sources are local code, experiment logs and preserved result files. Historical GPU runs were not repeated. The fresh classifier calculation does not measure answer quality, model speed or savings in Hermes.