Supporting engineering · Ongoing experiments
Local AI infrastructure
What it takes to keep the models useful.
Running a model is one part of building an AI application. Keeping several applications useful introduces another set of problems: available memory, model selection, context limits, routing, streaming behavior, and the consequences of a service becoming unavailable.
The local AI work is where I deal with those constraints directly. It includes model serving, quantization experiments, application-specific configurations, and the operational work needed to connect inference to the products people actually use.
This infrastructure gives the rest of the portfolio somewhere to run and somewhere to experiment. It also keeps the tradeoffs concrete. A larger model, a longer context window, or another layer of orchestration has a cost that needs to justify itself in the application.
Availability is part of the interface
Application-specific model configurations, context budgets, and explicit fallback paths connect serving to product behavior. FRIDAY’s optional specialist route illustrates the tradeoff: an unavailable specialist can fall back to the primary model while failures remain inspectable.
Quantization and serving experiments are operational work, not a promise that every model or application is always available. No hardware utilization, cost-saving, or comparative benchmark claim is implied here.