Nightly research and grooming · Layer 4
A knowledge base goes stale when nobody feeds it. Norm runs overnight, looks for new material on the topics you configured, and files it so your answers are current by morning.
Runs on a nightly schedule or on demand through the API. Only one run is in flight at a time.
Each topic can switch either source on, with its own search terms, arXiv categories and direct URLs.
Adding a topic means editing config: seeds, sources, on or off. No code change.
New papers are chunked, embedded and added to the collections that Annie retrieves from.
Every run is recorded as a job in a local database, so you can see what ran, when, and how it ended.
The same nightly pipeline also grooms the ticket backlog.
Each run follows the same five phases, in order.
Norm is a layer 4 worker. It writes into the collections that layer 3 retrieves from, and its runs show up as jobs you can inspect in the web app.
Copyright (c) 1998-2026 spw1.com and steven p williams All Rights Reserved