Network security for an AI home lab

I run an AI home lab from my spare room. I want models and agents to do useful things, not leak data or sit behind black-box services I do not trust. The basic rule is simple: treat every AI process as if it wants to phone home.

Why AI changes the network picture

AI workloads change the attack surface. Local models can still pull in large files and use GPUs. Cloud models want API keys. Either way, data, telemetry or credentials can go out if the network is loose enough. That is where the mess starts.

A typical AI home lab has a host for virtualisation, a GPU or two, a NAS for data, a router with VLAN support, and automation tools such as Home Assistant. I run a hypervisor on a small server, a NAS for datasets, and a Pi-class device for DNS and light services. The physical layout matters. Put more sensitive systems on separate subnets.

Hypervisor: Proxmox, KVM or similar for virtualisation of services. Use VMs for isolation that must touch hardware; use containers for stateless services.
GPU host: pass through the GPU to a VM for model training or inference.
Router/switch: VLAN-capable, with a configurable firewall.
DNS filter: Pi-hole or Unbound to control DNS requests and stop accidental data leaks.
VPN: WireGuard for remote access; do not expose admin consoles to the open internet.
Automation engine: Home Assistant or scripts to trigger security scenes when something odd happens.

Automation that actually helps

Automation scenes make security practical. I use automations to quarantine odd endpoints, rotate ephemeral keys, and turn on deeper logging when something looks off. When a non-trusted device appears on the network, an automation can move it into a quarantine VLAN and notify me. That cuts response time and keeps the lab usable.

Keep the setup boring where you can. Separate services by function, minimise privileges, keep secrets out of container images, and centralise logs. Back up model weights and configs. Rotate keys and use per-service credentials. Test restores. If the data is sensitive, local inference is usually the cleaner option.

Privacy settings and outbound control

Privacy settings live at two layers: the app and the network. Switch models to offline mode where you can and turn off telemetry. For cloud services, create least-privilege API keys and bind them to specific IPs or referrers. Store secrets in a vault such as pass, Vault, or a simple encrypted file with tight access controls.

Network controls that help:

Use DNS filtering to block known telemetry domains. Pi-hole with a curated blocklist stops a lot of calls.
Block outbound ports except the ones you expect. For example, allow HTTPS for specific hosts only.
Create an allowlist for external endpoints your model legitimately needs.
Run local instances of services the model expects where feasible. A local embedding service or a self-hosted vector DB cuts outbound traffic.

Example: to block a VM from making arbitrary outbound connections with nftables:

sudo nft add table inet filter
sudo nft add chain inet filter output { type filter hook output priority 0 ; }
sudo nft add rule inet filter output oifname != lo ip protocol tcp tcp dport 443 ct state established,related accept
sudo nft add rule inet filter output oifname != lo counter reject

That keeps the VM from talking to the web except where rules permit.

Virtualisation and isolation

Virtualisation isolates workloads. I run untrusted models in dedicated VMs and use a lightweight management VM for orchestration. GPU passthrough gives performance without losing isolation.

Use separate VLANs per trust level: trusted, untrusted, IoT, guests.
Host a jump VM for admin access. Admin into that VM, then SSH into internal services; do not expose SSH to the wider network.
Use snapshots and immutable images for inference VMs. Rebuild them routinely to remove persisted compromise.
Apply AppArmor or SELinux policies within VMs and containers for process-level restrictions.

If you pass a GPU to a VM, remember PCI passthrough removes some hypervisor-level inspection. Compensate with tighter network filters and closer monitoring of that VM.

Basic network rules

Keep rules small, explicit and logged. Logs show what tried to get out. Keep them long enough to investigate. I forward firewall and host logs to a central syslog or an ELK-like stack on a separate server.

Use strong SSH controls: key-based auth only, different keys per host, and short-lived agent keys for remote sessions. Disable password login. Use multi-factor authentication for any web UI that supports it.

If you use a VPN, keep the configuration in version control and rotate keys. For remote access, I prefer WireGuard because it is lightweight and auditable. Lock administrative interfaces to specific internal IP ranges.

Security scenes for common problems

Automation scenes are useful when incidents happen. Examples I use:

Quarantine scene: on detection of an unknown MAC address on the main VLAN, move it to a quarantine VLAN and trigger a notification.
Lockdown scene: if a high rate of outbound connections is detected from a host, block that host for a fixed period and start packet capture.
Key-rotation scene: weekly rotate ephemeral API keys used by auxiliary services and reload them via a secrets store.

Build these with Home Assistant, Ansible or simple scripts tied into the router API. Keep them auditable and reversible.

What bites most AI setups

Large models use large data. That raises storage and leak risks. Encrypt model storage at rest. Limit who and what can read model directories. Run discovery scans for sensitive files before exposing them to inference services.

Models and tool APIs can run arbitrary code if misconfigured. Turn off exec hooks in agent frameworks unless you really need them. Audit third-party model packages. Treat downloaded models as untrusted binary blobs until you have scanned them.

Watch the simple signs: spikes in outbound connections, DNS queries to new domains, or sudden CPU or GPU load from an unusual VM. That often gives you time to act before data leaves the lab.

Where this is going

More AI components will phone home by default. Local privacy controls matter more for that reason. Design the lab so you can swap in local services quickly. Keep an eye on model supply chains for trojans and on telemetry channels for new endpoints.

Practical takeaways

Segment the network with VLANs and explicit firewall rules.
Run risky models inside dedicated VMs, with GPU passthrough when needed.
Use DNS filtering and an allowlist for outbound connections.
Automate quarantine and key rotation to cut reaction time.
Encrypt model storage and limit privileges to reduce leak risk.

I build AI labs in layers. Start with a VLAN and Pi-hole, add a VM for models, then add automation scenes. Each step reduces exposure and keeps the lab usable.

Network security for an AI home lab