AI Engineering Lab
A self-hosted engineering platform for experimenting with AI-assisted software development,
local inference, agentic workflows, and the integration of AI into modern software engineering practices.
Overview
The AI Engineering Lab is a self-hosted development environment designed to explore how artificial intelligence can be integrated into professional software engineering workflows. Rather than treating AI as a standalone service or development tool, the environment provides the infrastructure needed to experiment with AI-assisted development as part of a complete software engineering platform.
The lab combines virtualization, GPU-accelerated local inference, source control, continuous integration, containerized services, and isolated development environments. Individual services are separated into virtual machines so that components can evolve independently while still participating in a common development workflow.
The initial environment is intentionally modest enough to support experimentation on a single workstation while being designed around concepts that can extend to a distributed environment. This allows architectural ideas to be validated before committing additional hardware and operational complexity.
Problem
AI-assisted software development is increasingly centered around highly capable frontier models delivered as hosted services. These models provide tremendous capability, but relying exclusively on them introduces tradeoffs involving cost, latency, privacy, availability, and control over how development workloads are processed.
At the same time, increasingly capable smaller models can execute many software engineering tasks locally. The challenge is not simply determining whether a local model can generate code, but understanding which development tasks can be performed effectively by local models, which require frontier-model capabilities, and how both can participate in a reliable engineering workflow.
Exploring those questions requires more than a GPU and a model server. It requires an environment where AI inference can be integrated with source control, automated testing, continuous integration, development tooling, and eventually observability and workload routing.
Solution
The AI Engineering Lab provides a controlled platform for experimenting with these architectural questions. The environment separates development infrastructure, AI inference, CI/CD services, and developer workloads into independently managed components running on a Proxmox virtualization host.
A dedicated Forge environment provides Git-based source control through Gitea and automated build and testing workflows through Woodpecker CI. A separate AI Gateway provides GPU-backed local model inference through Ollama, with model data stored independently from the operating system. Additional virtual machines provide isolated environments for development roles and experiments without coupling those workloads directly to the underlying infrastructure.
This separation allows individual components to be replaced, upgraded, or scaled without redesigning the entire environment. More importantly, it creates a platform where AI-assisted development can be evaluated as part of a complete software delivery process rather than as an isolated coding demonstration.
System Architecture
Every major architectural decision in the AI Engineering Lab traces back to four guiding design principles:
- Separation of concerns — development infrastructure, inference, CI/CD, and experimental workloads should evolve independently.
- Local control — models, development artifacts, and engineering workflows should be capable of operating on infrastructure under direct control.
- Replaceable components — individual tools and models should be replaceable without redesigning the platform.
- Design for expansion — the initial single-host implementation should provide a foundation for distributed inference and additional compute resources.
The architecture deliberately separates the software delivery platform from AI inference. Local models are therefore a service consumed by development environments rather than being embedded directly into individual development workstations or applications. This provides a natural boundary for future routing, monitoring, model replacement, and distributed inference.
Gitea • Woodpecker CI"] GATEWAY["AI Gateway
Ollama • Model Services"] ENV["Development Environments
Isolated Linux VMs"] OBS["Observability
Metrics • Logs • Monitoring
(Planned)"] end subgraph COMPUTE["Compute & Storage"] GPU["GPU-Accelerated Inference"] MODELS["Persistent Model Storage"] end HOST["Proxmox Virtualization Platform"] DEV --> ENV ENV --> FORGE ENV --> GATEWAY FORGE -->|"Build • Test • Validate"| ENV GATEWAY --> GPU GATEWAY --> MODELS PLATFORM --> HOST COMPUTE --> HOST OBS -.->|"Planned"| FORGE OBS -.->|"Planned"| GATEWAY OBS -.->|"Planned"| ENV
Design Tradeoffs
Every architectural decision represents a tradeoff between competing priorities.
Virtual Machines vs. Consolidated Services
Running each major responsibility in a separate virtual machine introduces additional resource and administrative overhead compared with hosting every service on a single operating system. The isolation was intentional: source control and CI/CD, AI inference, and development environments have different resource requirements and upgrade cycles. Separating them creates stable service boundaries and allows individual components to evolve without destabilizing the rest of the platform.
Self-Hosted vs. Managed Services
Managed source control, CI/CD, and AI services could significantly reduce the operational effort required to build the environment. Self-hosting was selected because the purpose of the lab is to understand and experiment with the complete engineering platform. Direct control over infrastructure also makes it possible to evaluate local inference, networking, storage, security, and resource allocation in ways that managed services would conceal.
Local Models vs. Frontier Models
Local models provide control over data, predictable infrastructure costs, low-latency access, and the ability to execute workloads without depending on an external service. Frontier models provide substantially greater capability for difficult reasoning and complex software engineering tasks.
The lab is therefore not designed around the assumption that one approach should replace the other. Its longer-term architecture is intended to explore a hybrid model in which development workloads can be directed to an appropriate inference resource based on factors such as task complexity, quality requirements, privacy, latency, and cost.
Prototype Scale vs. Production Scale
The initial environment runs on a single workstation with a dedicated high-memory GPU rather than beginning as a distributed compute cluster. This limits aggregate inference capacity but substantially reduces the complexity required to validate the architecture. The platform can therefore establish working service boundaries, storage strategies, development workflows, and routing concepts before those decisions are extended across additional GPU hosts.
Persistent Model Storage vs. VM Storage
AI models represent large, relatively static assets with different lifecycle and capacity requirements from operating-system and application data. Model storage is therefore separated from the AI Gateway's system disk, allowing models to persist independently and making future storage expansion or migration less disruptive.
Lessons Learned
Infrastructure boundaries matter as much as model capability
Early experimentation reinforced that running a model is only a small part of building a useful AI engineering environment. Storage, GPU access, networking, authentication, source control, CI/CD, and development isolation all become part of the architecture once AI is incorporated into a repeatable software engineering workflow.
Build the platform incrementally
Establishing each service independently before integrating the complete workflow made infrastructure problems significantly easier to isolate. Source control, CI/CD, webhook integration, GPU access, persistent storage, and model inference could each be validated before becoming dependencies of higher-level workflows.
Self-hosting exposes architectural assumptions
Managed platforms make many infrastructure decisions invisible. Building equivalent capabilities from individual components exposed assumptions about networking, persistence, service discovery, authentication, resource allocation, and failure boundaries that would otherwise remain hidden. Those constraints influenced the architecture of the platform as much as the choice of AI model.
AI architecture should optimize the system, not the model
The most capable model is not necessarily the appropriate model for every development task. A useful AI engineering platform must consider quality alongside latency, privacy, compute requirements, and cost. This shifts the architectural question from "Which model should we use?" to "How should the system select and validate the appropriate capability for each workload?"
Current Status and Next Phase
The core AI Engineering Lab is operational. The virtualization environment, isolated development systems, self-hosted source control, CI/CD pipeline, persistent model storage, and GPU-accelerated local inference have been implemented and validated.
The remaining infrastructure component is the observability layer, which will provide centralized visibility into service health, model workloads, resource utilization, logs, and eventually inference performance.
Once observability is in place, the next phase will extend the platform beyond infrastructure validation and into quantitative experimentation. Representative software engineering workloads will be evaluated across local models, frontier models, and hybrid routing strategies.
Cost / Quality Evaluation
The longer-term goal is to measure whether a hybrid AI architecture can preserve software quality while reducing dependence on frontier-model inference. Experiments will compare factors such as:
- Task completion quality
- Execution time and latency
- Local compute utilization
- Frontier-model token consumption
- Approximate cost
- Success of automated validation through builds, tests, and CI/CD
Rather than assuming that local or frontier models are universally preferable, the experiment is intended to identify the workloads for which each provides the strongest engineering tradeoff.
Conclusion
The AI Engineering Lab began as an effort to create a controlled environment for experimenting with local AI-assisted software development, but its broader purpose is to explore how AI should fit into the architecture of modern software engineering.
The platform deliberately treats models as one component of a larger engineering system. Source control, continuous integration, automated validation, development environments, storage, compute, and eventually observability all participate in determining whether AI-generated work can be used reliably and economically.
The longer-term architecture does not assume that local models will replace frontier systems. Instead, it anticipates a heterogeneous environment in which different levels of AI capability are applied where they provide the greatest value, with conventional software engineering practices providing validation and control.
As the platform evolves toward distributed inference and workload routing, its central question remains consistent: How can AI capability be integrated into the software engineering process to improve development speed and quality while using compute, cost, and human attention efficiently?