Complete Network Architecture — Security Review Document
Classification: Internal — Infrastructure Architecture Review Prepared by: Anshin Health Engineering Version: 1.0 — Draft for Review, 2026-03-22 Compliance scope: HIPAA Security Rule, SOC 2 Type II design target
This document is a living design specification.
1. Document Purpose and Scope
This document provides a complete, device-by-device, port-by-port specification of the Anshin Health network architecture. It covers:
- Every physical device and its role
- Every cable connection, including which specific port on which device
- All software installed on each network device
- VLAN design and security zone boundaries
- Firewall rule intent (not exact OPNsense syntax — implementation detail for the SRE deploying this)
- VPN design for 4–6 remote staff users
- 12–18 public URL routing from internet to internal services
- Application-level failover to AWS or a partner-hosted standby environment
- HIPAA-relevant security controls mapped to infrastructure decisions
This document is suitable for printing to PDF and circulating for security review.
2. Architecture Philosophy
2.1 Zero-License-Fee Security Commitment
All security and network functions use open-source, perpetually free software. There are no per-device license fees, annual subscription fees, or feature-unlock charges — now or ever.
| Function | Software | License |
|---|---|---|
| Edge firewall | OPNsense CE | BSD 2-Clause (free forever) |
| IDS/IPS | Suricata (OPNsense plugin) | GPLv2 (free forever) |
| VPN server | WireGuard + OpenVPN (built into OPNsense) | GPLv2/GPL (free forever) |
| Load balancer / reverse proxy | HAProxy (OPNsense plugin) | GPLv2 (free forever) |
| Application proxy | Caddy 2 (on rp-01) | Apache 2.0 (free forever) |
| Switch management | TP-Link Omada SDN (self-hosted controller) | Proprietary but free (no license fees) |
| DNS (internal) | Unbound (built into OPNsense) | BSD (free forever) |
| Monitoring | kube-prometheus-stack (Prometheus/Grafana) | Apache 2.0 (free forever) |
No security vendor will ever come back to Anshin Health requesting license renewals.
2.2 Defense in Depth
Security is implemented at every layer:
Internet
└── Layer 1: ISP Modem (NAT, port filtering)
└── Layer 2: OPNsense CARP HA pair (stateful firewall, IDS/IPS, VPN termination)
└── Layer 3: VLAN segmentation (storage isolated, OOB isolated, homestead isolated)
└── Layer 4: Kubernetes NetworkPolicy (pod-to-pod firewall)
└── Layer 5: TLS everywhere (HTTPS only, mTLS between services)
└── Layer 6: FreeIPA / RBAC (identity and access control)
└── Layer 7: SaaS Guard (per-tenant data isolation at application layer)
No single failure at any layer exposes PHI or allows unauthorized access.
2.3 HIPAA Security Rule Alignment
| HIPAA Control | Implementation |
|---|---|
| Access Control (§164.312(a)(1)) | FreeIPA RBAC, VPN required for admin access, OPNsense firewall zones |
| Audit Controls (§164.312(b)) | OPNsense firewall logging → syslog, Prometheus audit metrics, BFF Audit Log DocType |
| Integrity (§164.312(c)(1)) | TLS 1.2+ on all services, certificate pinning at ingress, Frappe document checksums |
| Transmission Security (§164.312(e)(1)) | TLS everywhere externally, WireGuard VPN for remote access, VLAN isolation internally |
| Automatic Logoff | Session timeout configured in Frappe + OPNsense admin (15 min idle) |
| Emergency Access | Break-glass procedures documented: physical OPNsense console, iLO remote access |
| Encryption (addressable) | AES-256 at rest (Ansible Vault for secrets, LUKS for storage volumes when deployed) |
3. Hardware Inventory
3.1 Edge Firewall — Beelink EQ12 Pro × 2
| Attribute | opnsense-01 (Primary) | opnsense-02 (Standby) |
|---|---|---|
| Device | Beelink EQ12 Pro | Beelink EQ12 Pro |
| CPU | Intel N100 (4 cores, 3.4GHz boost) | Intel N100 |
| RAM | 16GB DDR5 | 16GB DDR5 |
| Storage | 500GB NVMe | 500GB NVMe |
| NIC built-in | 2× Intel i226-V (2.5GbE RJ45) | 2× Intel i226-V (2.5GbE RJ45) |
| NIC added | 1× USB 3.2 to Gigabit (pfsync only) | 1× USB 3.2 to Gigabit (pfsync only) |
| Power | DC 12V barrel jack (~15W idle, 25W peak) | DC 12V barrel jack |
| OS | OPNsense CE 24.x (latest stable) | OPNsense CE 24.x |
| Role | CARP primary — handles all traffic when healthy | CARP standby — takes over on opnsense-01 failure |
Why two units? OPNsense CARP (Common Address Redundancy Protocol) provides automatic failover with state synchronization (pfsync). If opnsense-01 loses power or crashes, opnsense-02 takes over the CARP VIPs within ~3 seconds — no manual intervention, no dropped connections.
Why Beelink EQ12 Pro specifically? The Intel i226-V NIC is specifically chosen for OPNsense/FreeBSD compatibility. It is well-supported, does not have the i225-V erratum issues, and handles the routing throughput for their fiber connection without hardware offload issues. Estimated cost: ~$189 each × 2 = $378 total for both firewalls.
3.2 Compute Servers
| Unit | Hostname | IP (future VLAN 200) | iLO IP | Status | CPU | RAM |
|---|---|---|---|---|---|---|
| HP #1 | pmx-01 | 10.10.200.5 | 10.10.100.237 | ✅ ACTIVE | 2× E5-2697 v4 (36c/72t) | 512GB |
| HP #2 | (pending) | 10.10.200.6 | 10.10.100.235 | 🔵 PENDING | 2× E5-2697 v4 | 512GB |
| HP #3 | (pending) | 10.10.200.7 | 10.10.100.236 | 🔵 PENDING | 2× E5-2697 v4 | 512GB |
| HP #4 | gpu-farm-01 | 10.10.200.8 | (reconnect) | 🟡 COMING SOON | 2× E5-2683 v4 (32c/64t) | 128GB |
Each server has:
- Built-in NIC: Intel I350 Quad Port 1GbE (ports eno1–eno4)
- Add-on NIC: 10Gtek dual SFP+ (Intel 82599ES, ports eth0/eth1 or sfp0/sfp1)
3.3 Network Switches
| Device | Hostname | Model | Ports | Speed | Role | Status |
|---|---|---|---|---|---|---|
| 10G Network Switch | sx3008-01 | TL-SX3008F | 8× SFP+ | 10Gbps | Network plane spine | 📦 Ordered |
| 10G Storage Switch | sx3008-02 | TL-SX3008F | 8× SFP+ | 10Gbps | Storage plane (isolated) | 📦 Ordered |
| 1G Network Switch | sg3428-01 | TL-SG3428 | 24×RJ45 + 4×SFP | 1Gbps | Network 1G access, OPNsense LAN | 📦 Ordered |
| 1G Storage Switch | sg3428-02 | TL-SG3428 | 24×RJ45 + 4×SFP | 1Gbps | Storage 1G fallback (isolated) | 📦 Ordered |
| Homestead Switch | sg3452-01 | TL-SG3452 | 48×RJ45 + 4×SFP | 1Gbps | Residential VLAN (isolated) | 📦 Ordered |
| SDN Controller | oc200-01 | Omada OC200 | 1×RJ45 | 1Gbps | Omada SDN management plane | 📦 Ordered |
| OOB Switch | ex2200p-01 | Juniper EX2200-24P | 24×PoE + 4×SFP | 1Gbps | Out-of-band management (repurposed) | ✅ On-site |
3.4 Other Infrastructure
| Device | Hostname | IP | Role |
|---|---|---|---|
| QNAP NAS | qnap-01 | 10.10.200.31 | NFS storage for K8s PVs, backups |
| Beelink Mini PC | blnk-01 | 10.10.200.3 | Dev workstation, monitoring, mgmt access |
| UPS × 4 | ups-01–04 | 10.10.100.41–44 | Power management (OOB network) |
4. Physical Connectivity — Complete Port Matrix
4.1 OPNsense-01 (Primary Firewall) — Beelink EQ12 Pro
opnsense-01
├── i226-V Port 1 (re0) ──── WAN ──── ISP Modem (Fiber, RJ45)
│ └── IP: [ISP-assigned public IP] / CARP WAN VIP
├── i226-V Port 2 (re1) ──── LAN ──── sg3428-01, Port 1 (RJ45, 1G, trunk all VLANs)
│ └── IP: 10.10.200.2 / CARP LAN VIP: 10.10.96.1
└── USB NIC (ue0) ─────── pfsync ──── opnsense-02 USB NIC (crossover or dedicated switch)
└── IP: 10.10.250.1/30 (dedicated pfsync subnet)
4.2 OPNsense-02 (Standby Firewall) — Beelink EQ12 Pro
opnsense-02
├── i226-V Port 1 (re0) ──── WAN ──── ISP Modem (via unmanaged splitter or modem 2nd port)
│ └── IP: [ISP-assigned, secondary] / shares WAN CARP VIP
├── i226-V Port 2 (re1) ──── LAN ──── sg3428-01, Port 2 (RJ45, 1G, trunk all VLANs)
│ └── IP: 10.10.200.3 / CARP LAN VIP: 10.10.96.1 (shared)
└── USB NIC (ue0) ─────── pfsync ──── opnsense-01 USB NIC (crossover cable)
└── IP: 10.10.250.2/30
With one fiber IP, both Beelinks share a single WAN IP via CARP. The ISP modem's LAN port connects to a small unmanaged switch or splitter, with both Beelinks' WAN ports connected. Only the CARP primary (opnsense-01) uses the IP — the standby listens. When the ISP provides a second static IP (second fiber or cable), opnsense-02 gets its own dedicated WAN IP.
4.3 sx3008-01 — 10G Network Plane Switch (8 ports)
sx3008-01 — TL-SX3008F (all ports SFP+ 10G)
Port 1: ── 1.5m DAC ──── HP #1 (pmx-01) sfp0 (bond-net, VLAN 200 trunk)
Port 2: ── 1.5m DAC ──── HP #2 sfp0 (bond-net, VLAN 200 trunk) [future]
Port 3: ── 1.5m DAC ──── HP #3 sfp0 (bond-net, VLAN 200 trunk) [future]
Port 4: ── 1.5m DAC ──── HP #4 (gpu-farm) sfp0 (bond-net, VLAN 200 trunk)
Port 5: ── 1.5m DAC ──── HP #5 (reserved) [planned]
Port 6: ── 1G SFP ────── sg3428-01 SFP1 (uplink trunk — all VLANs, 1G)
Port 7: ── (reserved) ── Future OPNsense 10G uplink (if 10G SFP transceiver added)
Port 8: ── 0.3m DAC ──── sx3008-02 Port 8 (inter-switch — management VLAN only)
VLAN tagging on sx3008-01: All server ports (1–5) carry tagged VLANs 200 (compute), 600 (K8s nodes), 610 (MetalLB). Port 6 (sg3428-01 uplink) is a tagged trunk carrying VLANs 100, 200, 500, 600, 610.
4.4 sx3008-02 — 10G Storage Plane Switch (8 ports) ⚠️ ISOLATED
sx3008-02 — TL-SX3008F (all ports SFP+ 10G)
⚠️ THIS SWITCH HAS NO UPLINK TO THE ROUTER. STORAGE TRAFFIC NEVER CROSSES THE FIREWALL.
Port 1: ── 1.5m DAC ──── HP #1 (pmx-01) sfp1 (bond-stor, VLAN 300)
Port 2: ── 1.5m DAC ──── HP #2 sfp1 (bond-stor, VLAN 300) [future]
Port 3: ── 1.5m DAC ──── HP #3 sfp1 (bond-stor, VLAN 300) [future]
Port 4: ── 1.5m DAC ──── HP #4 (gpu-farm) sfp1 (bond-stor, VLAN 300)
Port 5: ── 1.5m DAC ──── QNAP qnap-01 (10G SFP+ if NIC installed, else see sg3428-02)
Port 6: ── (reserved) ── HP #5 [planned]
Port 7: ── 1G SFP ────── sg3428-02 SFP1 (storage 1G fallback uplink)
Port 8: ── 0.3m DAC ──── sx3008-01 Port 8 (inter-switch management — VLAN 100 only)
Why is storage isolated? Ceph storage traffic (OSD replication, client I/O) is high-bandwidth and low-latency-sensitive. Mixing it with compute/internet traffic degrades both. Isolating it also means a compromised network VLAN cannot see storage replication traffic — an important security boundary for encrypted-at-rest data.
4.5 sg3428-01 — 1G Network Access Switch (24 ports + 4 SFP)
sg3428-01 — TL-SG3428
RJ45 Ports (1G):
Port 1: ── opnsense-01 LAN (re1) — tagged trunk, VLANs 100/200/500/600/610
Port 2: ── opnsense-02 LAN (re1) — tagged trunk (same VLANs)
Port 3: ── HP #1 (pmx-01) eno1 — VLAN 200 (compute, native) + VLAN 100 (tagged)
Port 4: ── HP #1 (pmx-01) eno2 — (LACP bond with eno1, or standby)
Port 5: ── HP #2 eno1 — VLAN 200 [future]
Port 6: ── HP #3 eno1 — VLAN 200 [future]
Port 7: ── HP #4 eno1 — VLAN 200
Port 8: ── rp-01 VM NIC — VLAN 200 (Caddy reverse proxy)
Port 9: ── gitlab-01 VM NIC — VLAN 200
Port 10: ── dc-01 VM NIC — VLAN 500 (services)
Port 11: ── dc-02 VM NIC — VLAN 500
Port 12: ── blnk-01 (workstation) — VLAN 200
Port 13: ── deepseek-01 VM NIC — VLAN 500
Port 14: ── nb-01 (NetBird VPN) — VLAN 500
Ports 15-20: — Workstations / expansion
Ports 21-22: — Reserved
Ports 23-24: — Access switch uplinks (homestead bridge if needed)
SFP Ports (1G SFP):
SFP1: ── sx3008-01 Port 6 (uplink trunk to 10G network spine)
SFP2: ── oc200-01 management port (Omada controller 1G)
SFP3: ── (reserved)
SFP4: ── (reserved)
4.6 sg3428-02 — 1G Storage Fallback Switch ⚠️ ISOLATED
sg3428-02 — TL-SG3428 (storage plane only, no router connection)
⚠️ ISOLATED — no connectivity to sg3428-01 or OPNsense
RJ45 Ports (1G):
Port 1: ── HP #1 (pmx-01) eno3 — VLAN 300 (storage native)
Port 2: ── HP #1 (pmx-01) eno4 — LACP bond with eno3
Port 3: ── HP #2 eno3 — VLAN 300 [future]
Port 4: ── HP #2 eno4 — bond [future]
Port 5: ── HP #3 eno3 — VLAN 300 [future]
Port 6: ── HP #3 eno4 — bond [future]
Port 7: ── HP #4 eno3 — VLAN 300
Port 8: ── HP #4 eno4 — bond
Port 9: ── QNAP qnap-01 1G RJ45 (fallback path if 10G not installed)
Ports 10-24: — Reserved (Ceph OSDs added with HP #2/#3)
SFP Ports:
SFP1: ── sx3008-02 Port 7 (uplink to 10G storage spine)
SFP2-4: — Reserved
4.7 ex2200p-01 — Out-of-Band Management Switch (ISOLATED)
ex2200p-01 — Juniper EX2200-24P (PoE)
⚠️ MANAGEMENT ONLY — iLO, UPS, OC200 (PoE), OPNsense console only
RJ45 PoE Ports:
Port 1: ── oc200-01 management NIC (PoE powered — no external power brick needed)
Port 2: ── HP #1 iLO 4 (10.10.100.237)
Port 3: ── HP #2 iLO 4 (10.10.100.235)
Port 4: ── HP #3 iLO 4 (10.10.100.236)
Port 5: ── HP #4 iLO 4 (reconnect before power-on)
Port 6: ── UPS #1 management (10.10.100.41)
Port 7: ── UPS #2 management (10.10.100.42)
Port 8: ── UPS #3 management (10.10.100.43)
Port 9: ── UPS #4 management (10.10.100.44)
Port 10: ── opnsense-01 serial console adapter (USB-to-serial, always-on access)
Port 11: ── opnsense-02 serial console adapter
Port 12: ── blnk-01 management NIC (dedicated OOB access machine)
Ports 13-24: — Reserved
SFP Uplinks:
SFP1: ── OPNsense-01 LAN port 3 (if USB NIC on opnsense not used for pfsync)
OR direct RJ45 management VLAN 100 path from sg3428-01
SFP2: ── Reserved
OOB access principle: Management traffic (iLO, UPS, OPNsense console) NEVER crosses the main network switches. It travels exclusively on the EX2200-24P. Even during a complete network switch failure, iLO access to all servers remains intact.
4.8 sg3452-01 — Homestead Residential Switch
sg3452-01 — TL-SG3452
Connected to: OPNsense via a dedicated WAN/DMZ-style interface or as a separate VLAN trunk
Isolation: Homestead devices can reach the internet but CANNOT reach lab VLANs
Ports 1-48: Residential devices (PCs, phones, smart home, printers)
SFP: Uplink to OPNsense homestead interface (dedicated VLAN 1000+)
4.9 Physical DAC Cable Summary
| Cable | Length | From | To | Purpose |
|---|---|---|---|---|
| DAC-01 | 1.5m | sx3008-01 Port 1 | HP#1 sfp0 | Network plane 10G |
| DAC-02 | 1.5m | sx3008-01 Port 2 | HP#2 sfp0 | Network plane 10G [future] |
| DAC-03 | 1.5m | sx3008-01 Port 3 | HP#3 sfp0 | Network plane 10G [future] |
| DAC-04 | 1.5m | sx3008-01 Port 4 | HP#4 sfp0 | Network plane 10G |
| DAC-05 | 1.5m | sx3008-02 Port 1 | HP#1 sfp1 | Storage plane 10G |
| DAC-06 | 1.5m | sx3008-02 Port 2 | HP#2 sfp1 | Storage plane 10G [future] |
| DAC-07 | 1.5m | sx3008-02 Port 3 | HP#3 sfp1 | Storage plane 10G [future] |
| DAC-08 | 1.5m | sx3008-02 Port 4 | HP#4 sfp1 | Storage plane 10G |
| DAC-09 | 0.3m | sx3008-01 Port 8 | sx3008-02 Port 8 | Inter-switch management |
| RJ45-01 | Cat6A | opnsense-01 re0 | ISP Modem LAN port | WAN primary |
| RJ45-02 | Cat6A | opnsense-02 re0 | ISP Modem LAN port | WAN standby (shared IP) |
| RJ45-03 | Cat6A | opnsense-01 re1 | sg3428-01 Port 1 | LAN trunk |
| RJ45-04 | Cat6A | opnsense-02 re1 | sg3428-01 Port 2 | LAN trunk standby |
| XOVER-01 | Cat6A crossover | opnsense-01 USB NIC | opnsense-02 USB NIC | pfsync dedicated |
Additional DAC cables to order: 2× 0.3m SFP DAC for switch-to-switch trunks (sx3008-01 Port 6 ↔ sg3428-01 SFP1 requires SFP module, not DAC — order appropriate 1G SFP transceivers for both ends).
5. VLAN Design and IP Addressing
5.1 VLAN Table
| VLAN ID | Name | Subnet | Purpose | Routed? | Internet? |
|---|---|---|---|---|---|
| 100 | Management/OOB | 10.10.100.0/24 | iLO, UPS, OC200, OPNsense admin | Yes, restricted | ❌ No |
| 200 | Compute | 10.10.200.0/24 | Servers, VMs, application nodes | Yes | Egress only |
| 300 | Storage | 10.10.300.0/24 | Ceph OSD, NFS, iSCSI | No router | ❌ No |
| 400 | Storage-Alt | 10.10.400.0/24 | Ceph public network | No router | ❌ No |
| 500 | Services | 10.10.500.0/24 | FreeIPA, monitoring, supporting services | Yes | Egress only |
| 600 | K8s Nodes | 10.10.600.0/24 | Kubernetes node NICs | Yes | Egress only |
| 610 | MetalLB | 10.10.98.0/24 | K8s LoadBalancer VIPs (ingress) | Yes | Ingress from FW |
| 700 | VPN Clients | 10.10.700.0/24 | WireGuard / OpenVPN allocated IPs | Yes | Full access |
| 1000 | Homestead | 192.168.125.0/24 | Residential devices | Yes | Internet only |
VLAN subnets use the VLAN ID in the third octet (10.10.100.x = VLAN 100) for easy readability and troubleshooting. Storage VLANs (300, 400) have no gateway — OPNsense does not have a leg on these VLANs; they are Layer 2 isolated.
5.2 Static IP Assignments — Critical Devices
| Device | Hostname | VLAN | IP Address | Role |
|---|---|---|---|---|
| OPNsense CARP VIP (gateway) | — | 200 | 10.10.200.1 | Default gateway for all compute |
| OPNsense CARP VIP (gateway) | — | 500 | 10.10.500.1 | Default gateway for services |
| OPNsense CARP VIP (gateway) | — | 600 | 10.10.600.1 | Default gateway for K8s nodes |
| OPNsense CARP VIP (OOB) | — | 100 | 10.10.100.1 | OOB management gateway |
| opnsense-01 | opnsense-01 | 200 | 10.10.200.2 | Primary node own IP |
| opnsense-02 | opnsense-02 | 200 | 10.10.200.3 | Standby node own IP |
| HP #1 (Proxmox) | pmx-01 | 200 | 10.10.200.5 | Proxmox host |
| HP #2 (OpenStack) | os-01 | 200 | 10.10.200.6 | OpenStack node [future] |
| HP #3 (OpenStack) | os-02 | 200 | 10.10.200.7 | OpenStack node [future] |
| HP #4 (GPU Farm) | gpu-farm-01 | 200 | 10.10.200.8 | GPU Proxmox host |
| FreeIPA DC-01 | dc-01 | 500 | 10.10.500.11 | Primary domain controller |
| FreeIPA DC-02 | dc-02 | 500 | 10.10.500.12 | Replica domain controller |
| Caddy Reverse Proxy | rp-01 | 200 | 10.10.200.22 | Public URL routing |
| GitLab | gitlab-01 | 200 | 10.10.200.41 | Source control + CI/CD |
| NetBird VPN | nb-01 | 500 | 10.10.500.25 | Mesh VPN gateway |
| QNAP NAS | qnap-01 | 200 | 10.10.200.31 | NFS storage (1G) |
| K8s MetalLB VIP | — | 610 | 10.10.98.40 | All K8s ingress |
| OC200 Controller | oc200-01 | 100 | 10.10.100.10 | Omada SDN controller |
| HP#1 iLO | — | 100 | 10.10.100.237 | Out-of-band management |
| HP#2 iLO | — | 100 | 10.10.100.235 | Out-of-band management |
| HP#3 iLO | — | 100 | 10.10.100.236 | Out-of-band management |
| UPS #1–4 | ups-01–04 | 100 | 10.10.100.41–44 | Power management |
| BeeLink workstation | blnk-01 | 200 | 10.10.200.3 | Dev/management workstation |
6. OPNsense Software Stack — Complete Feature List
Both Beelink units run OPNsense CE (Community Edition). The following features are active or planned. All are included with OPNsense CE at no cost.
6.1 Core Features (Built-In, Always Free)
| Feature | Configuration |
|---|---|
| Stateful Firewall | Zone-based: WAN / LAN / OOB / Homestead / VPN |
| CARP HA | opnsense-01 primary, opnsense-02 standby, VIP preemption enabled |
| pfsync | State table sync on dedicated crossover link (10.10.250.0/30) |
| VLAN (802.1Q) | VLANs 100, 200, 500, 600, 610, 700, 1000 on re1 (LAN) |
| Multi-WAN | WAN1: fiber (primary), WAN2: ready for 2nd fiber or cable (future) |
| WireGuard VPN | Staff remote access, 4–6 users, 10.10.700.0/24 pool |
| OpenVPN | Fallback VPN for devices that cannot run WireGuard |
| IPsec IKEv2 | Mobile device VPN (iOS/Android native clients) |
| Unbound DNS | Internal resolver with DNS-over-TLS upstream (Cloudflare 1.1.1.1, Quad9) |
| DHCP | Per-VLAN DHCP pools with static leases for infrastructure devices |
| NAT/PAT | Outbound masquerade on WAN, port forwards for inbound HTTPS and VPN |
| Traffic Shaping / QoS | Prioritize VPN and K8s API traffic over bulk storage replication |
| Syslog | All firewall events → Prometheus syslog receiver → Grafana |
| 2FA Admin UI | TOTP required for OPNsense web interface (all admin accounts) |
| Automatic Logoff | Admin session timeout: 15 minutes idle |
6.2 OPNsense Plugins (Free, Install via UI)
| Plugin | Purpose |
|---|---|
| os-suricata | IDS/IPS — Suricata with ET Open ruleset (Emerging Threats, free). Monitors WAN for known malicious traffic patterns, drops on IPS mode |
| os-haproxy | Load balancer — routes 12–18 public URLs to rp-01 (Caddy) with health checks; provides application-level failover to AWS or partner standby |
| os-acme-client | ACME certificates — though Anshin uses acme.sh + Ansible for wildcard certs, this is available for OPNsense's own management certificate |
| os-zeek | Network traffic analysis — logs all connection metadata, files transferred, DNS queries to a local index |
| os-nginx | Optional: serve OPNsense admin UI on a non-standard port with mutual TLS |
| os-netdata | Host-level metrics from OPNsense boxes → Prometheus |
| os-freeradius | RADIUS for future Wi-Fi 802.1X authentication |
6.3 Suricata IDS/IPS Configuration
Suricata runs in IPS mode (inline, drop mode) on the WAN interface:
| Rule Source | Update Frequency | Focus |
|---|---|---|
| Emerging Threats Open | Daily (free) | Known malicious IPs, C2, exploit signatures |
| Abuse.ch / ThreatFox | Daily (free) | Malware URLs, botnet C2 |
| SANS Internet Storm Center | Daily (free) | High-confidence threat intel |
Suricata will alert (not drop) on the following to avoid false positives initially:
- Port scanning originating from internet
- SQL injection patterns against web ports
- TLS certificate anomalies
Suricata will drop immediately:
- Known botnet command-and-control IPs
- Exploit kit landing page signatures
- Log4Shell / ProxyShell exploit patterns
All Suricata alerts feed into: OPNsense → syslog → Prometheus → Grafana #alerts channel in Zoho Cliq.
7. Firewall Rules — Intent
The tables below define the intent of each firewall rule. The deploying engineer will translate these into exact OPNsense rule syntax. Rule order matters — OPNsense processes rules top-to-bottom, first match wins.
7.1 WAN Interface — Inbound Rules
| Priority | Action | Protocol | Source | Destination | Port | Justification |
|---|---|---|---|---|---|---|
| 1 | BLOCK | Any | Known bogon/RFC1918 | Any | Any | Anti-spoofing — inbound private IPs are always malicious |
| 2 | PASS | TCP | Any | WAN VIP | 443 | HTTPS for all 12–18 public URLs (→ HAProxy → rp-01) |
| 3 | PASS | UDP | Any | WAN VIP | 51820 | WireGuard VPN |
| 4 | PASS | TCP/UDP | Any | WAN VIP | 1194 | OpenVPN fallback |
| 5 | PASS | UDP | Any | WAN VIP | 500, 4500 | IPsec IKEv2 |
| 6 | BLOCK | TCP | Any | WAN VIP | 22 | SSH is VPN-only — never directly from internet |
| 7 | BLOCK | TCP | Any | WAN VIP | 80 | HTTP redirected at DNS level — not open here |
| 8 | BLOCK | Any | Any | Any | Any | Default deny — log all |
7.2 LAN (Compute VLAN 200) — Inter-VLAN and Outbound Rules
| Priority | Action | Protocol | Source | Destination | Port | Justification |
|---|---|---|---|---|---|---|
| 1 | PASS | Any | VLAN 200 | FreeIPA (VLAN 500) | 88, 389, 636, 464 | Kerberos + LDAP/S + kpasswd |
| 2 | PASS | TCP/UDP | VLAN 200 | VLAN 500 (dc-01/02) | 53 | Internal DNS resolution |
| 3 | PASS | TCP | VLAN 200 | VLAN 610 (MetalLB) | 443, 80 | K8s ingress access |
| 4 | PASS | Any | VLAN 600 (K8s) | VLAN 200 | Any | K8s pods reaching compute APIs |
| 5 | PASS | TCP | VLAN 200 | Internet | 443, 80 | Software updates, Docker pulls, external APIs |
| 6 | BLOCK | Any | VLAN 200 | VLAN 100 (OOB) | Any | Compute cannot reach iLO/UPS management |
| 7 | PASS | Any | VLAN 200 | Any | Any | General egress (restrictable per service later) |
7.3 OOB Management VLAN 100 — Strict Access Rules
| Priority | Action | Protocol | Source | Destination | Port | Justification |
|---|---|---|---|---|---|---|
| 1 | PASS | Any | VPN Clients (VLAN 700) | VLAN 100 | Any | Admin access via VPN only |
| 2 | PASS | TCP | VLAN 100 | Internet | 443 | iLO firmware updates |
| 3 | BLOCK | Any | Any | VLAN 100 | Any | Everything else blocked |
7.4 Storage VLANs 300/400 — Not Routed (Layer 2 Only)
OPNsense has no interface on VLANs 300 or 400. These VLANs exist only on sx3008-02 and sg3428-02, which have no router uplink. Storage traffic cannot reach the internet or any other VLAN by design.
7.5 VPN Clients VLAN 700 — Remote Access Rules
| Priority | Action | Protocol | Source | Destination | Justification |
|---|---|---|---|---|---|
| 1 | PASS | Any | VLAN 700 | VLAN 100 | VPN users can reach OOB (iLO, UPS) |
| 2 | PASS | Any | VLAN 700 | VLAN 200 | VPN users can reach compute services |
| 3 | PASS | Any | VLAN 700 | VLAN 500 | VPN users can reach FreeIPA, monitoring |
| 4 | PASS | TCP | VLAN 700 | Internet | 443 |
| 5 | BLOCK | Any | VLAN 700 | Homestead (1000) | VPN cannot reach residential devices |
8. VPN Design — Remote Access for 4–6 Staff
8.1 WireGuard (Primary — Recommended for All Staff)
WireGuard is the default VPN protocol. It is built into OPNsense (no plugin needed), uses the WireGuard kernel module for high performance, and the client software is free on all platforms (iOS, Android, macOS, Windows, Linux).
Architecture:
Staff Device (WireGuard client)
│
│ UDP/51820 — WireGuard tunnel (ChaCha20-Poly1305 encryption)
│
▼
OPNsense CARP VIP (10.10.96.1 external / WAN VIP)
│
▼
WireGuard interface (wg0) — VLAN 700 (10.10.700.0/24)
│
├── Route to VLAN 100 (OOB management)
├── Route to VLAN 200 (compute)
├── Route to VLAN 500 (services, monitoring)
└── Route to VLAN 610 (MetalLB, K8s services)
Per-user configuration:
| User | WireGuard IP | Access Level |
|---|---|---|
| SRE Admin | 10.10.700.10 | Full — all VLANs including OOB |
| Engineering Director | 10.10.700.11 | Full — all VLANs including OOB |
| Developer 1 | 10.10.700.20 | VLAN 200 + 610 only |
| Developer 2 | 10.10.700.21 | VLAN 200 + 610 only |
| Developer 3 | 10.10.700.22 | VLAN 200 + 610 only |
| Emergency/Spare | 10.10.700.30 | Restricted — temporary access |
Each user gets a WireGuard config file (or QR code for mobile) generated from OPNsense. The config file is never reused — if a device is lost or an employee leaves, their peer key is removed from OPNsense and they immediately lose access.
8.2 OpenVPN (Fallback — for Devices That Cannot Run WireGuard)
Available on TCP/1194 for corporate firewalls that block UDP, or for devices with WireGuard client issues.
8.3 IPsec IKEv2 (Mobile Devices — iOS/Android Native)
iOS and Android have IPsec IKEv2 clients built in (no app install required). For staff who need mobile access without installing a VPN app.
8.4 Two-Factor Authentication for VPN
VPN access requires certificate + 2FA TOTP (authenticator app):
- Step 1: WireGuard keypair (cryptographic authentication)
- Step 2: TOTP code (FreeIPA OTP integrated with OPNsense via RADIUS)
This means a stolen WireGuard config file alone is not enough to gain access.
9. Public URL Routing — 12–18 Services
9.1 Traffic Flow for All Public URLs
User Browser
│
│ HTTPS (port 443)
▼
OPNsense WAN VIP (public IP: 65.182.226.114)
│
│ Port forward: 443 → 10.10.200.22:443 (rp-01 Caddy)
│ OR: HAProxy on OPNsense terminates TLS, routes by Host header
▼
HAProxy on OPNsense (health-checked routing)
│
├── Primary backend: rp-01 (10.10.200.22) ─── Caddy ─── K8s services
└── Fallback backend: [AWS ELB or Partner Standby IP] (if rp-01 health check fails)
9.2 Known Public URLs and Their Internal Routes
The following 14 known services are currently configured in Caddy on rp-01. All use wildcard TLS certificates issued by ZeroSSL via acme.sh.
| Public URL | Internal Upstream | Service |
|---|---|---|
| orchestrate.anshin.us | 10.10.98.40 (MetalLB, NGINX Ingress) | Orchestrate main app |
| dev.orchestrate.anshin.us | 10.10.98.40 | Dev environment |
| api.orchestrate.anshin.us | 10.10.98.40 | API gateway |
| core-dev.orchestrate.anshin.us | 10.10.98.40 | Frappe/ERPNext core |
| erp.anshinhealth.net | 10.10.98.40 | ERPNext portal |
| knowledge.anshin.us | 10.10.98.40 | This knowledge base |
| gitlab.anshinhealth.net | 10.10.200.41 (direct) | GitLab |
| grafana.mon.anshinhealth.net | 10.10.98.40 (VPN-only) | Grafana monitoring |
| infisical.svcs.anshinhealth.net | 10.10.98.40 | Secrets management |
| accelerate.onnex.app | 10.10.98.40 | Onnex Accelerate |
| sweetpealinens.com | 10.10.98.40 | Sweet Pea Linens |
| (4–8 additional URLs) | 10.10.98.40 | Expansion services |
9.3 HAProxy Health Check and Failover Config (Intent)
HAProxy on OPNsense monitors rp-01 with HTTP health checks every 5 seconds:
Frontend: anshin_https
Bind: 0.0.0.0:443 (WAN)
SSL: terminate with wildcard certs (loaded from Ansible-deployed K8s secrets)
Default backend: anshin_primary
Backend: anshin_primary (rp-01)
Server: rp-01 10.10.200.22:443 check inter 5s fall 2 rise 2
Health check: GET /health → expect HTTP 200
Backend: anshin_failover (AWS or Partner Standby)
Server: aws-lb [AWS ELB DNS or static IP]:443 check inter 10s
OR
Server: partner-standby [Partner external IP]:443 check inter 10s
Failover behavior: If 2 consecutive health checks to rp-01 fail (10 seconds), HAProxy routes all traffic to the failover backend automatically. When rp-01 recovers (2 consecutive successful checks), traffic shifts back. No DNS change required — failover is sub-second at the HAProxy level.
10. Application Failover Design
10.1 Failover Tiers
| Tier | Target | Trigger | Recovery |
|---|---|---|---|
| Tier 1 — Primary | Anshin Lab (rp-01 → K8s) | — | Always preferred |
| Tier 2 — AWS | AWS ELB (Anshin-managed AWS deployment) | rp-01 health check fails for 10s | Auto by HAProxy |
| Tier 3 — Partner Standby | Partner-hosted infrastructure | AWS also unavailable | Manual DNS switch or HAProxy Tier 3 backend |
10.2 What Stays in Sync for Failover to Work
For a failover backend to serve users correctly, it must have:
| Requirement | Current Status | Implementation |
|---|---|---|
| Identical application code | 🔵 Needs CI/CD pipeline | GitLab CI pushes to AWS + lab on every main branch deploy |
| Database replication | 🔵 Needs design | MariaDB → AWS RDS read replica or periodic snapshot restore |
| Session state | 🔵 Needs design | Redis/Valkey session state → AWS ElastiCache mirror |
| TLS certificates | ✅ acme.sh wildcard | Same certs deployed to AWS and partner standby via Ansible |
| DNS TTL low | 🔵 Needs action | All public URLs must be at TTL 60s to enable fast DNS failover as backup |
Application failover to a partner lab requires a formal agreement covering:
- Network path (private WireGuard tunnel between Anshin Lab and partner infrastructure)
- Data replication (no PHI until HIPAA BAA is in place with the partner)
- Capacity (can the partner environment serve Anshin's traffic load?)
- Monitoring (escalation path when Anshin failover triggers to the partner)
10.3 WAN Failover (Future — When Second IP Is Added)
When a second fiber or cable IP is available:
OPNsense Multi-WAN (built-in, no plugins needed):
WAN1: Fiber primary (current) ─── Gateway Group: ANSHIN_WAN, Tier 1
WAN2: Fiber-2 or Cable ────────── Gateway Group: ANSHIN_WAN, Tier 2
Policy-based routing:
- All outbound traffic → ANSHIN_WAN gateway group
- If WAN1 drops → OPNsense automatically routes via WAN2 (no user impact)
- If WAN1 recovers → traffic returns to WAN1 (preempt enabled)
Load balancing option:
- Weight WAN1:WAN2 = 2:1 (fiber 2x preference over cable)
- Both active simultaneously (bandwidth aggregation)
11. Security Controls — HIPAA Mapping
11.1 Network Security
| Control | Implementation | HIPAA §164.312 |
|---|---|---|
| Perimeter firewall | OPNsense CE, stateful inspection, default-deny | (e)(2)(i) |
| IDS/IPS | Suricata with ET Open + Abuse.ch rules | (e)(2)(i) |
| Network segmentation | 8 VLANs, storage fully isolated, OOB isolated | (e)(2)(i) |
| Encrypted transport | TLS 1.2+ enforced, TLS 1.0/1.1 disabled at OPNsense | (e)(2)(ii) |
| VPN for remote access | WireGuard (ChaCha20) + 2FA TOTP required | (e)(1) |
| DNS security | Unbound with DNSSEC validation, DNS-over-TLS upstream | Supportive |
11.2 Access Control
| Control | Implementation | HIPAA §164.312 |
|---|---|---|
| Unique user identification | FreeIPA (LDAP/Kerberos) — every service authenticates against IPA | (a)(2)(i) |
| Emergency access | iLO out-of-band on isolated OOB VLAN, break-glass procedure documented | (a)(2)(ii) |
| Automatic logoff | OPNsense: 15 min, Frappe: 30 min, Grafana: 60 min | (a)(2)(iii) |
| Encryption/decryption (addressable) | TLS everywhere; LUKS full-disk encryption on storage VMs when deployed | (a)(2)(iv) |
11.3 Audit Controls
| What Is Logged | Where | Retention |
|---|---|---|
| All firewall allow/deny events | OPNsense → syslog → Prometheus → Grafana | 90 days hot, archive |
| All VPN connections (who, when, from where) | OPNsense WireGuard log | 90 days |
| All IDS/IPS alerts | Suricata → syslog | 90 days |
| All K8s API calls | kube-apiserver audit log → Prometheus | 90 days |
| All Frappe/ERPNext API calls | BFF Audit Log DocType | 90 days (purged weekly) |
| All admin logins to OPNsense | OPNsense auth log | 90 days |
11.4 Incident Response — Network Events
| Event | Detection | Response |
|---|---|---|
| Suricata critical alert | Zoho Cliq #alerts within 30s | SRE on-call investigates → block IP at OPNsense |
| VPN brute force (5 failed auths) | OPNsense auth log → alert | Automatic block of source IP for 24h |
| Unexpected port scan | Suricata alert | Alert + add to OPNsense blocklist |
| iLO/OOB access from non-VPN | Firewall deny log → alert | Immediate investigation |
| DDoS on WAN | Suricata + OPNsense rate limiting | Null route at ISP + haproxy 429 |
12. Omada SDN — Switch Management
12.1 What Omada Manages
The OC200 controller manages only the TP-Link switch fabric (sx3008-01, sx3008-02, sg3428-01, sg3428-02, sg3452-01). It does not manage OPNsense — OPNsense has its own web interface and API.
12.2 OC200 Placement and Access
| Item | Value |
|---|---|
| Device | TP-Link Omada OC200 hardware controller |
| Power source | PoE from ex2200p-01 port 1 (isolated OOB switch) |
| Management IP | 10.10.100.10 (OOB VLAN 100 only) |
| Web UI | https://10.10.100.10:8043 (VPN required) |
| REST API | https://10.10.100.10:8043/api/v2/ |
12.3 Omada REST API — Claude Automation Potential
The Omada controller exposes a full REST API. A mcp-omada MCP server would allow Claude Code to:
| Capability | API Endpoint |
|---|---|
| List all switches and health | GET /sites/{id}/devices |
| Show per-port traffic stats | GET /sites/{id}/stat/port |
| Change port VLAN assignment | PATCH /sites/{id}/ports/{portId} |
| Show all connected clients | GET /sites/{id}/clients |
| View active alerts | GET /sites/{id}/alerts |
| Reboot a switch | POST /sites/{id}/cmd/devices |
Combined with the OPNsense REST API (via mcp-opnsense), the entire network stack becomes manageable through Claude — Bryan can ask plain-English questions and get actionable answers.
13. Physical Network Topology Diagrams
13.1 Internet Edge — WAN Design
══════════════════════════ INTERNET ══════════════════════════
ISP Fiber (1× static public IP: 65.182.226.114)
│
┌─────────────────┐
│ ISP Modem │
│ (passthrough │
│ / bridge mode)│
└────────┬────────┘
│
┌──────────────┴──────────────┐
│ (splitter / modem 2nd port) │
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ opnsense-01 │ │ opnsense-02 │
│ (PRIMARY) │◄──────── │ (STANDBY) │
│ Beelink EQ12 │ pfsync │ Beelink EQ12 │
│ re0: WAN │ crossover│ re0: WAN │
│ re1: LAN ──────┤ ├── re1: LAN │
└────────┬────────┘ └────────┬────────┘
│ │
└──────────────┬─────────────┘
│ CARP VIP: 10.10.96.1
│ (both connect to sg3428-01)
▼
══════════════════════════ INTERNAL ══════════════════════════
13.2 Internal Network Plane — 10G + 1G
sg3428-01 (1G Network Access)
┌─────────────────────────────┐
opnsense-01 ──┤ P1 SFP1────────────────────┼──── sx3008-01 Port 6 (1G uplink)
opnsense-02 ──┤ P2 │
pmx-01 eno1 ──┤ P3 │
pmx-01 eno2 ──┤ P4 │
hp4 eno1 ──┤ P7 │
rp-01 ──┤ P8 │
gitlab-01 ──┤ P9 │
dc-01 ──┤ P10 │
dc-02 ──┤ P11 │
blnk-01 ──┤ P12 │
└─────────────────────────────-┘
sx3008-01 (10G Network Spine)
┌─────────────────────────────┐
pmx-01 sfp0 ──┤ P1 │
hp2 sfp0 ──┤ P2 [future] │
hp3 sfp0 ──┤ P3 [future] │
hp4 sfp0 ──┤ P4 │
hp5 sfp0 ──┤ P5 [planned] │
(10G uplink)──┤ P6 [OPNsense 10G if added] │
sg3428-01 ──┤ P7 (1G SFP uplink trunk) │
sx3008-02 ──┤ P8 (inter-switch, mgmt only)│
└─────────────────────────────┘
│
│ VLANs 200, 600, 610 distributed
│ to all server 10G ports at line rate
13.3 Storage Plane — 10G ISOLATED
sx3008-02 (10G Storage Spine) ⚠️ NO WAN UPLINK
┌─────────────────────────────┐
pmx-01 sfp1 ──┤ P1 │
hp2 sfp1 ──┤ P2 [future] │
hp3 sfp1 ──┤ P3 [future] │
hp4 sfp1 ──┤ P4 │
QNAP 10G ──┤ P5 (if SFP+ NIC installed) │
hp5 sfp1 ──┤ P6 [planned] │
sg3428-02 ──┤ P7 (1G storage fallback) │
sx3008-01 ──┤ P8 (inter-switch, mgmt only)│
└─────────────────────────────┘
sg3428-02 (1G Storage Fallback) ⚠️ NO WAN UPLINK
┌─────────────────────────────┐
pmx-01 eno3 ──┤ P1 │
pmx-01 eno4 ──┤ P2 │
hp4 eno3 ──┤ P7 │
hp4 eno4 ──┤ P8 │
QNAP RJ45 ──┤ P9 │
sx3008-02 ──┤ SFP1 │
└─────────────────────────────┘
13.4 Out-of-Band Management Plane — ISOLATED
ex2200p-01 (OOB Management) ⚠️ ISOLATED
┌─────────────────────────────┐
oc200-01 ──┤ P1 (PoE powered) │
hp1 iLO ──┤ P2 │
hp2 iLO ──┤ P3 │
hp3 iLO ──┤ P4 │
hp4 iLO ──┤ P5 │
ups-01 ──┤ P6 │
ups-02 ──┤ P7 │
ups-03 ──┤ P8 │
ups-04 ──┤ P9 │
opns-01 con ──┤ P10 (USB-serial adapter) │
opns-02 con ──┤ P11 (USB-serial adapter) │
blnk-01 ──┤ P12 │
└─────────────────────────────┘
Access: VPN required (VLAN 700 → VLAN 100 allowed by firewall)
14. Open Technical Questions for Reviewer Validation
| # | Question | Notes |
|---|---|---|
| 1 | VLAN numbering — does this match the sre-iac IP scheme? | sre-iac uses 10.20.0.0/24 (VLAN 200), 10.50.0.0/24 (VLAN 500) — see comparison doc |
| 2 | pfsync: dedicated crossover cable vs. VLAN on LAN interface? | Crossover is cleaner; VLAN is simpler. Recommend crossover for HIPAA isolation |
| 3 | OPNsense WAN: does the ISP modem support 2-port LAN, or is a WAN splitter needed? | Determines physical WAN setup for CARP standby |
| 4 | QNAP storage NIC: does qnap-01 have an SFP+ port installed? | If yes → sx3008-02 Port 5 at 10G. If no → sg3428-02 at 1G |
| 5 | Suricata mode for first deployment: IPS (drop) or IDS (alert only)? | Recommend IDS first; promote to IPS after 2-week baseline review |
| 6 | 1G SFP modules: sx3008-01 Port 6 ↔ sg3428-01 SFP1 — are compatible 1G SFP modules available? | Need 2× 1G SFP multimode or LC fiber, or 1× 0.3m DAC if same chassis |
| 7 | VLAN subnets vs. existing FreeIPA DNS zones — do FreeIPA reverse zones need updating? | FreeIPA zones may need PTR zone additions for 10.10.200.0/24 etc. |
| 8 | Partner failover: static external IP confirmed? WireGuard peer config needed? | Required before Tier 3 failover can be activated |
| 9 | HIPAA BAA status with any partner hosting failover traffic? | No PHI in failover path until BAA is signed and verified |
15. Budget Summary (Network Hardware Only)
| Item | Qty | Est. Unit | Est. Total |
|---|---|---|---|
| Beelink EQ12 Pro | 2 | $189 | $378 |
| TL-SX3008F (10G switch) | 2 | $240 | $480 |
| TL-SG3428 (1G switch) | 2 | $200 | $400 |
| TL-SG3452 (48-port homestead) | 1 | $350 | $350 |
| Omada OC200 controller | 1 | $80 | $80 |
| 10Gtek dual SFP+ NIC (servers) | 4 | $49 | $196 |
| H!Fiber 1.5m DAC cables | 8 | $9.49 | $76 |
| H!Fiber 0.3m DAC cables | 3 | $9.49 | $28 |
| USB 3.0 to Gigabit (pfsync) | 2 | $15 | $30 |
| 1G SFP transceivers (inter-switch) | 4 | $8 | $32 |
| Network hardware total | ~$2,050 |
Note: Juniper EX2200-24P (OOB switch), HP servers, QNAP NAS, and UPS units already on-site — not included above.
16. Document Control
| Rev | Date | Author | Change |
|---|---|---|---|
| 1.0 | 2026-03-22 | Anshin Engineering | Initial release — complete network and security architecture |
| 1.1 | 2026-03-22 | Anshin Engineering | Removed named reviewers; generalized reviewer questions |
Deployment sequence:
- Deploy OPNsense CARP pair (opnsense-01, opnsense-02) — replace Cisco bridge
- Deploy TP-Link 10G switch fabric (sx3008-01, sx3008-02, sg3428-01, sg3428-02)
- Install 10Gtek NICs in all HP servers; connect DAC cables
- Configure VLANs — migrate from flat /20 to segmented scheme
- Enable WireGuard VPN — provision staff peer configs
- Deploy HAProxy — configure primary and failover backends
- Enable Suricata IDS (alert mode first, promote to IPS after baseline)
- Decommission Cisco bridge
- Validate all 14+ public URLs, VPN access, OOB management path