Lesson 30. Release the command-line model and check our claims

Why this matters

A first-aid kit is useful when each item has a name, expiry, and instructions. Our release is also a complete kit: program, rules, exercise cards, a run command, and checkable boundaries. It answers only two kinds of short question about a fictional club.

Before the code

Copy all five files from step 30: main.py, engine.py, checks.py, facts.json, and examples.json. The command takes a date and question; quotes keep a spaced question as one argument. Put optional --trace before the date to print the decision trail. sys.argv stores arguments; len selects one of two accepted command forms. sys.exit(2) stops bad usage with an error code. try attempts to parse the date, while except ValueError catches a bad format and prints a clear message. The date is the day being asked about, not a claim that an exercise record is true now.

Files in this step: examples.json, checks.py, engine.py, facts.json, main.py.

From the project root, enter the step folder and run the program:

cd course/kazakh-ai/step-30
python3 main.py 2026-09-24 "Шахмат қашан?"

On Windows, replace python3 with py.

import sys
from datetime import date

from engine import answer

if len(sys.argv) == 3:
    show_trace = False
    raw_date = sys.argv[1]
    question = sys.argv[2]
elif len(sys.argv) == 4 and sys.argv[1] == "--trace":
    show_trace = True
    raw_date = sys.argv[2]
    question = sys.argv[3]
else:
    print('Қолдану: python3 main.py [--trace] YYYY-MM-DD "Сұрақ"')
    sys.exit(2)
try:
    on_date = date.fromisoformat(raw_date)
except ValueError:
    print("Қате күн: YYYY-MM-DD түрінде жазыңыз")
    sys.exit(2)
result = answer(question, on_date)
print(result["text"])
if show_trace:
    for line in result["trace"]:
        print("Із:", line)

Step files.

How the program works

From the step folder, run python3 main.py 2026-09-24 "Шахмат қашан?". It prints шахмат үйірмесінің уақыты: бейсенбі, 15:00 | 2026-09-24 күнгі дерек | дереккөз: club-sheet-01. Сурет қайда? yields білмеймін: дерек жоқ; Шахмат қашан қайда? asks for one type; date 2027-01-01 refuses an expired record. Add --trace to see five source-check steps. The small held-out set in lesson 25 tested question types only; it does not establish real service quality. We built a narrow local model, not universal AI. No paid API call in exercise code does not mean zero total cost; avoiding free-form generation does not eliminate source errors.

The finished model’s specification sheet

A model specification is like the plate on an electrical appliance: units and test conditions replace words such as “fast” and “small.” The step folder has two more files. quality_cases.json contains 16 checks written before the run; benchmark.py runs them and measures the finished system. Run:

python3 benchmark.py --json

A cold start begins in a new Python process. A warm request runs inside an existing process. p95 is the time within which 95 percent of requests finished. RSS is the peak physical memory of the entire process; tracemalloc counts only observed Python allocations. A trainable parameter is a number changed while a neural network learns. This system has none: it builds intent counts from 13 cards at startup.

Throughput is the number of requests completed per second. One worker process is one running copy of Python; in this experiment it handles requests in sequence and uses no more than one core at once. The program then warms 1, 2, 4, and 8 processes, gives each process 10,000 requests three times, and selects the middle total rate. One KiB is 1,024 bytes, and one MiB is 1,024 KiB. ARM is the processor family of the computer used for the control run.

A control run on 25 September 2026 used an ARM computer with macOS 26.5.2 and Python 3.14.5:

On a phone, swipe the following wide tables left and right.

What additional processes delivered

Speedup compares total throughput with one worker. Parallel efficiency divides speedup by the worker count: 100% would mean that every new worker added its full share of speed. Summed RSS below adds the worker peaks; it excludes the coordinator process.

Two and four workers raised total throughput. Eight were slightly slower than four and used roughly twice the worker memory. The experiment reveals a limit of this computer and program; it does not by itself identify the cause. Core types, shared files, memory, or operating-system scheduling may contribute. A different server needs a new measurement.

The last quality row gives “hallucination” an operational meaning for this experiment. An error occurs when the reference requires refusal but the program confidently produces another answer. 0 of 10 applies only to the published checks. It does not establish zero errors on arbitrary text and does not protect against a wrong fact in facts.json.

Comparison with large LLMs

The makers of closed GPT, Claude, and Gemini models do not publish their weight counts or server memory. The claim that every parameter of every major LLM is known is therefore false. Open model scale is published; memory still depends on weight precision and deployment.

A stored parameter value is called a weight. FP8 and BF16 are two ways to store a weight, in about 1 or 2 bytes. Where the parameter count is known, multiplication therefore gives a reproducible lower estimate. These figures cover weights only; working memory, context cache, and serving data are additional. A token is a word or word part into which an LLM divides text; context is the set of tokens available to it in one request. The ≈ sign below marks a calculated figure, not one measured on our computer.

The “one thousand times” threshold is exceeded for the size of these stated representations. That factor cannot be transferred to intelligence, task range, speed, or quality: an LLM handles free text and many tasks, while our model knows two question types and two facts. We also do not substitute someone else’s number for LLM speed. A valid ratio requires running a named version on the same hardware, or through a fixed API, with the same 16 requests.

Scale sources: Meta on Llama 3.1 405B and 128K context, DeepSeek-V3 model card, NVIDIA requirements for DeepSeek-V3, Qwen3 description, official GPT-5.6 specifications, official Gemini 3.7 Flash specifications.

Where our model is stronger and where it falls short

Comparison helps us choose a tool. A pocket calculator multiplies faster than a person but cannot summarize a book; a librarian understands an open question but uses more resources to search. Our model and an LLM differ in the same way.

There is a third option: a hybrid system. An LLM can interpret open wording, while a deterministic component checks allowed fields, date, and source before returning the fact. The combination is more complex than either component alone, but joins a broad input with a controlled fact.

Reconstruct the selection rule from three questions:

1. Is the fact set small and approved, with a high cost for invention? Start with a deterministic model.
2. Does the user need open text, with too many possible phrasings to list? Consider an LLM plus a separate fact check.
3. Are both properties required? Measure the hybrid on one sealed suite instead of adding together the promises of two technologies.

Support map

Command + date + question → word guard → type → one card → valid date → sourced answer; otherwise an explained refusal.

Recall and check

Hide the code and reconstruct the support map. Try four commands: a known question, unknown club, expired date, and --trace. Hint: 2026-09-24 is between checked_on and valid_until, while 2027-01-01 is not. Expected for the expired date: білмеймін: бұл күнге жарамды дерек жоқ; for an unknown club: білмеймін: таныс емес сөз. Common mistake: presenting an exercise date as a real club’s current schedule.

Gate 6: final defense

Show how an answer depends on a card and date; obtain the same answer for Шахмат қашан? and the known form Шахматтың уақыты қашан?; name two justified refusals; run the specification and explain the difference between RSS and Python allocations. Then explain why 16/16 is not perfection and why a size factor cannot be transferred to intelligence. Pass when you can guide a new learner through the chain without the lesson text. If you get stuck, return to the matching one of the six support signals and defend the project again.