Skip to main content

Complete Network Architecture — Security Review Document

Classification: Internal — Infrastructure Architecture Review Prepared by: Anshin Health Engineering Version: 1.0 — Draft for Review, 2026-03-22 Compliance scope: HIPAA Security Rule, SOC 2 Type II design target

Draft for Review

This document is a living design specification.


1. Document Purpose and Scope

This document provides a complete, device-by-device, port-by-port specification of the Anshin Health network architecture. It covers:

  • Every physical device and its role
  • Every cable connection, including which specific port on which device
  • All software installed on each network device
  • VLAN design and security zone boundaries
  • Firewall rule intent (not exact OPNsense syntax — implementation detail for the SRE deploying this)
  • VPN design for 4–6 remote staff users
  • 12–18 public URL routing from internet to internal services
  • Application-level failover to AWS or a partner-hosted standby environment
  • HIPAA-relevant security controls mapped to infrastructure decisions

This document is suitable for printing to PDF and circulating for security review.


2. Architecture Philosophy

2.1 Zero-License-Fee Security Commitment

All security and network functions use open-source, perpetually free software. There are no per-device license fees, annual subscription fees, or feature-unlock charges — now or ever.

FunctionSoftwareLicense
Edge firewallOPNsense CEBSD 2-Clause (free forever)
IDS/IPSSuricata (OPNsense plugin)GPLv2 (free forever)
VPN serverWireGuard + OpenVPN (built into OPNsense)GPLv2/GPL (free forever)
Load balancer / reverse proxyHAProxy (OPNsense plugin)GPLv2 (free forever)
Application proxyCaddy 2 (on rp-01)Apache 2.0 (free forever)
Switch managementTP-Link Omada SDN (self-hosted controller)Proprietary but free (no license fees)
DNS (internal)Unbound (built into OPNsense)BSD (free forever)
Monitoringkube-prometheus-stack (Prometheus/Grafana)Apache 2.0 (free forever)

No security vendor will ever come back to Anshin Health requesting license renewals.

2.2 Defense in Depth

Security is implemented at every layer:

Internet
└── Layer 1: ISP Modem (NAT, port filtering)
└── Layer 2: OPNsense CARP HA pair (stateful firewall, IDS/IPS, VPN termination)
└── Layer 3: VLAN segmentation (storage isolated, OOB isolated, homestead isolated)
└── Layer 4: Kubernetes NetworkPolicy (pod-to-pod firewall)
└── Layer 5: TLS everywhere (HTTPS only, mTLS between services)
└── Layer 6: FreeIPA / RBAC (identity and access control)
└── Layer 7: SaaS Guard (per-tenant data isolation at application layer)

No single failure at any layer exposes PHI or allows unauthorized access.

2.3 HIPAA Security Rule Alignment

HIPAA ControlImplementation
Access Control (§164.312(a)(1))FreeIPA RBAC, VPN required for admin access, OPNsense firewall zones
Audit Controls (§164.312(b))OPNsense firewall logging → syslog, Prometheus audit metrics, BFF Audit Log DocType
Integrity (§164.312(c)(1))TLS 1.2+ on all services, certificate pinning at ingress, Frappe document checksums
Transmission Security (§164.312(e)(1))TLS everywhere externally, WireGuard VPN for remote access, VLAN isolation internally
Automatic LogoffSession timeout configured in Frappe + OPNsense admin (15 min idle)
Emergency AccessBreak-glass procedures documented: physical OPNsense console, iLO remote access
Encryption (addressable)AES-256 at rest (Ansible Vault for secrets, LUKS for storage volumes when deployed)

3. Hardware Inventory

Attributeopnsense-01 (Primary)opnsense-02 (Standby)
DeviceBeelink EQ12 ProBeelink EQ12 Pro
CPUIntel N100 (4 cores, 3.4GHz boost)Intel N100
RAM16GB DDR516GB DDR5
Storage500GB NVMe500GB NVMe
NIC built-in2× Intel i226-V (2.5GbE RJ45)2× Intel i226-V (2.5GbE RJ45)
NIC added1× USB 3.2 to Gigabit (pfsync only)1× USB 3.2 to Gigabit (pfsync only)
PowerDC 12V barrel jack (~15W idle, 25W peak)DC 12V barrel jack
OSOPNsense CE 24.x (latest stable)OPNsense CE 24.x
RoleCARP primary — handles all traffic when healthyCARP standby — takes over on opnsense-01 failure

Why two units? OPNsense CARP (Common Address Redundancy Protocol) provides automatic failover with state synchronization (pfsync). If opnsense-01 loses power or crashes, opnsense-02 takes over the CARP VIPs within ~3 seconds — no manual intervention, no dropped connections.

Why Beelink EQ12 Pro specifically? The Intel i226-V NIC is specifically chosen for OPNsense/FreeBSD compatibility. It is well-supported, does not have the i225-V erratum issues, and handles the routing throughput for their fiber connection without hardware offload issues. Estimated cost: ~$189 each × 2 = $378 total for both firewalls.

3.2 Compute Servers

UnitHostnameIP (future VLAN 200)iLO IPStatusCPURAM
HP #1pmx-0110.10.200.510.10.100.237✅ ACTIVE2× E5-2697 v4 (36c/72t)512GB
HP #2(pending)10.10.200.610.10.100.235🔵 PENDING2× E5-2697 v4512GB
HP #3(pending)10.10.200.710.10.100.236🔵 PENDING2× E5-2697 v4512GB
HP #4gpu-farm-0110.10.200.8(reconnect)🟡 COMING SOON2× E5-2683 v4 (32c/64t)128GB

Each server has:

  • Built-in NIC: Intel I350 Quad Port 1GbE (ports eno1–eno4)
  • Add-on NIC: 10Gtek dual SFP+ (Intel 82599ES, ports eth0/eth1 or sfp0/sfp1)

3.3 Network Switches

DeviceHostnameModelPortsSpeedRoleStatus
10G Network Switchsx3008-01TL-SX3008F8× SFP+10GbpsNetwork plane spine📦 Ordered
10G Storage Switchsx3008-02TL-SX3008F8× SFP+10GbpsStorage plane (isolated)📦 Ordered
1G Network Switchsg3428-01TL-SG342824×RJ45 + 4×SFP1GbpsNetwork 1G access, OPNsense LAN📦 Ordered
1G Storage Switchsg3428-02TL-SG342824×RJ45 + 4×SFP1GbpsStorage 1G fallback (isolated)📦 Ordered
Homestead Switchsg3452-01TL-SG345248×RJ45 + 4×SFP1GbpsResidential VLAN (isolated)📦 Ordered
SDN Controlleroc200-01Omada OC2001×RJ451GbpsOmada SDN management plane📦 Ordered
OOB Switchex2200p-01Juniper EX2200-24P24×PoE + 4×SFP1GbpsOut-of-band management (repurposed)✅ On-site

3.4 Other Infrastructure

DeviceHostnameIPRole
QNAP NASqnap-0110.10.200.31NFS storage for K8s PVs, backups
Beelink Mini PCblnk-0110.10.200.3Dev workstation, monitoring, mgmt access
UPS × 4ups-01–0410.10.100.41–44Power management (OOB network)

4. Physical Connectivity — Complete Port Matrix

opnsense-01
├── i226-V Port 1 (re0) ──── WAN ──── ISP Modem (Fiber, RJ45)
│ └── IP: [ISP-assigned public IP] / CARP WAN VIP
├── i226-V Port 2 (re1) ──── LAN ──── sg3428-01, Port 1 (RJ45, 1G, trunk all VLANs)
│ └── IP: 10.10.200.2 / CARP LAN VIP: 10.10.96.1
└── USB NIC (ue0) ─────── pfsync ──── opnsense-02 USB NIC (crossover or dedicated switch)
└── IP: 10.10.250.1/30 (dedicated pfsync subnet)
opnsense-02
├── i226-V Port 1 (re0) ──── WAN ──── ISP Modem (via unmanaged splitter or modem 2nd port)
│ └── IP: [ISP-assigned, secondary] / shares WAN CARP VIP
├── i226-V Port 2 (re1) ──── LAN ──── sg3428-01, Port 2 (RJ45, 1G, trunk all VLANs)
│ └── IP: 10.10.200.3 / CARP LAN VIP: 10.10.96.1 (shared)
└── USB NIC (ue0) ─────── pfsync ──── opnsense-01 USB NIC (crossover cable)
└── IP: 10.10.250.2/30
WAN failover with one fiber IP

With one fiber IP, both Beelinks share a single WAN IP via CARP. The ISP modem's LAN port connects to a small unmanaged switch or splitter, with both Beelinks' WAN ports connected. Only the CARP primary (opnsense-01) uses the IP — the standby listens. When the ISP provides a second static IP (second fiber or cable), opnsense-02 gets its own dedicated WAN IP.

4.3 sx3008-01 — 10G Network Plane Switch (8 ports)

sx3008-01 — TL-SX3008F (all ports SFP+ 10G)

Port 1: ── 1.5m DAC ──── HP #1 (pmx-01) sfp0 (bond-net, VLAN 200 trunk)
Port 2: ── 1.5m DAC ──── HP #2 sfp0 (bond-net, VLAN 200 trunk) [future]
Port 3: ── 1.5m DAC ──── HP #3 sfp0 (bond-net, VLAN 200 trunk) [future]
Port 4: ── 1.5m DAC ──── HP #4 (gpu-farm) sfp0 (bond-net, VLAN 200 trunk)
Port 5: ── 1.5m DAC ──── HP #5 (reserved) [planned]
Port 6: ── 1G SFP ────── sg3428-01 SFP1 (uplink trunk — all VLANs, 1G)
Port 7: ── (reserved) ── Future OPNsense 10G uplink (if 10G SFP transceiver added)
Port 8: ── 0.3m DAC ──── sx3008-02 Port 8 (inter-switch — management VLAN only)

VLAN tagging on sx3008-01: All server ports (1–5) carry tagged VLANs 200 (compute), 600 (K8s nodes), 610 (MetalLB). Port 6 (sg3428-01 uplink) is a tagged trunk carrying VLANs 100, 200, 500, 600, 610.

4.4 sx3008-02 — 10G Storage Plane Switch (8 ports) ⚠️ ISOLATED

sx3008-02 — TL-SX3008F (all ports SFP+ 10G)
⚠️ THIS SWITCH HAS NO UPLINK TO THE ROUTER. STORAGE TRAFFIC NEVER CROSSES THE FIREWALL.

Port 1: ── 1.5m DAC ──── HP #1 (pmx-01) sfp1 (bond-stor, VLAN 300)
Port 2: ── 1.5m DAC ──── HP #2 sfp1 (bond-stor, VLAN 300) [future]
Port 3: ── 1.5m DAC ──── HP #3 sfp1 (bond-stor, VLAN 300) [future]
Port 4: ── 1.5m DAC ──── HP #4 (gpu-farm) sfp1 (bond-stor, VLAN 300)
Port 5: ── 1.5m DAC ──── QNAP qnap-01 (10G SFP+ if NIC installed, else see sg3428-02)
Port 6: ── (reserved) ── HP #5 [planned]
Port 7: ── 1G SFP ────── sg3428-02 SFP1 (storage 1G fallback uplink)
Port 8: ── 0.3m DAC ──── sx3008-01 Port 8 (inter-switch management — VLAN 100 only)

Why is storage isolated? Ceph storage traffic (OSD replication, client I/O) is high-bandwidth and low-latency-sensitive. Mixing it with compute/internet traffic degrades both. Isolating it also means a compromised network VLAN cannot see storage replication traffic — an important security boundary for encrypted-at-rest data.

4.5 sg3428-01 — 1G Network Access Switch (24 ports + 4 SFP)

sg3428-01 — TL-SG3428

RJ45 Ports (1G):
Port 1: ── opnsense-01 LAN (re1) — tagged trunk, VLANs 100/200/500/600/610
Port 2: ── opnsense-02 LAN (re1) — tagged trunk (same VLANs)
Port 3: ── HP #1 (pmx-01) eno1 — VLAN 200 (compute, native) + VLAN 100 (tagged)
Port 4: ── HP #1 (pmx-01) eno2 — (LACP bond with eno1, or standby)
Port 5: ── HP #2 eno1 — VLAN 200 [future]
Port 6: ── HP #3 eno1 — VLAN 200 [future]
Port 7: ── HP #4 eno1 — VLAN 200
Port 8: ── rp-01 VM NIC — VLAN 200 (Caddy reverse proxy)
Port 9: ── gitlab-01 VM NIC — VLAN 200
Port 10: ── dc-01 VM NIC — VLAN 500 (services)
Port 11: ── dc-02 VM NIC — VLAN 500
Port 12: ── blnk-01 (workstation) — VLAN 200
Port 13: ── deepseek-01 VM NIC — VLAN 500
Port 14: ── nb-01 (NetBird VPN) — VLAN 500
Ports 15-20: — Workstations / expansion
Ports 21-22: — Reserved
Ports 23-24: — Access switch uplinks (homestead bridge if needed)

SFP Ports (1G SFP):
SFP1: ── sx3008-01 Port 6 (uplink trunk to 10G network spine)
SFP2: ── oc200-01 management port (Omada controller 1G)
SFP3: ── (reserved)
SFP4: ── (reserved)

4.6 sg3428-02 — 1G Storage Fallback Switch ⚠️ ISOLATED

sg3428-02 — TL-SG3428 (storage plane only, no router connection)
⚠️ ISOLATED — no connectivity to sg3428-01 or OPNsense

RJ45 Ports (1G):
Port 1: ── HP #1 (pmx-01) eno3 — VLAN 300 (storage native)
Port 2: ── HP #1 (pmx-01) eno4 — LACP bond with eno3
Port 3: ── HP #2 eno3 — VLAN 300 [future]
Port 4: ── HP #2 eno4 — bond [future]
Port 5: ── HP #3 eno3 — VLAN 300 [future]
Port 6: ── HP #3 eno4 — bond [future]
Port 7: ── HP #4 eno3 — VLAN 300
Port 8: ── HP #4 eno4 — bond
Port 9: ── QNAP qnap-01 1G RJ45 (fallback path if 10G not installed)
Ports 10-24: — Reserved (Ceph OSDs added with HP #2/#3)

SFP Ports:
SFP1: ── sx3008-02 Port 7 (uplink to 10G storage spine)
SFP2-4: — Reserved

4.7 ex2200p-01 — Out-of-Band Management Switch (ISOLATED)

ex2200p-01 — Juniper EX2200-24P (PoE)
⚠️ MANAGEMENT ONLY — iLO, UPS, OC200 (PoE), OPNsense console only

RJ45 PoE Ports:
Port 1: ── oc200-01 management NIC (PoE powered — no external power brick needed)
Port 2: ── HP #1 iLO 4 (10.10.100.237)
Port 3: ── HP #2 iLO 4 (10.10.100.235)
Port 4: ── HP #3 iLO 4 (10.10.100.236)
Port 5: ── HP #4 iLO 4 (reconnect before power-on)
Port 6: ── UPS #1 management (10.10.100.41)
Port 7: ── UPS #2 management (10.10.100.42)
Port 8: ── UPS #3 management (10.10.100.43)
Port 9: ── UPS #4 management (10.10.100.44)
Port 10: ── opnsense-01 serial console adapter (USB-to-serial, always-on access)
Port 11: ── opnsense-02 serial console adapter
Port 12: ── blnk-01 management NIC (dedicated OOB access machine)
Ports 13-24: — Reserved

SFP Uplinks:
SFP1: ── OPNsense-01 LAN port 3 (if USB NIC on opnsense not used for pfsync)
OR direct RJ45 management VLAN 100 path from sg3428-01
SFP2: ── Reserved

OOB access principle: Management traffic (iLO, UPS, OPNsense console) NEVER crosses the main network switches. It travels exclusively on the EX2200-24P. Even during a complete network switch failure, iLO access to all servers remains intact.

4.8 sg3452-01 — Homestead Residential Switch

sg3452-01 — TL-SG3452
Connected to: OPNsense via a dedicated WAN/DMZ-style interface or as a separate VLAN trunk
Isolation: Homestead devices can reach the internet but CANNOT reach lab VLANs

Ports 1-48: Residential devices (PCs, phones, smart home, printers)
SFP: Uplink to OPNsense homestead interface (dedicated VLAN 1000+)

4.9 Physical DAC Cable Summary

CableLengthFromToPurpose
DAC-011.5msx3008-01 Port 1HP#1 sfp0Network plane 10G
DAC-021.5msx3008-01 Port 2HP#2 sfp0Network plane 10G [future]
DAC-031.5msx3008-01 Port 3HP#3 sfp0Network plane 10G [future]
DAC-041.5msx3008-01 Port 4HP#4 sfp0Network plane 10G
DAC-051.5msx3008-02 Port 1HP#1 sfp1Storage plane 10G
DAC-061.5msx3008-02 Port 2HP#2 sfp1Storage plane 10G [future]
DAC-071.5msx3008-02 Port 3HP#3 sfp1Storage plane 10G [future]
DAC-081.5msx3008-02 Port 4HP#4 sfp1Storage plane 10G
DAC-090.3msx3008-01 Port 8sx3008-02 Port 8Inter-switch management
RJ45-01Cat6Aopnsense-01 re0ISP Modem LAN portWAN primary
RJ45-02Cat6Aopnsense-02 re0ISP Modem LAN portWAN standby (shared IP)
RJ45-03Cat6Aopnsense-01 re1sg3428-01 Port 1LAN trunk
RJ45-04Cat6Aopnsense-02 re1sg3428-01 Port 2LAN trunk standby
XOVER-01Cat6A crossoveropnsense-01 USB NICopnsense-02 USB NICpfsync dedicated

Additional DAC cables to order: 2× 0.3m SFP DAC for switch-to-switch trunks (sx3008-01 Port 6 ↔ sg3428-01 SFP1 requires SFP module, not DAC — order appropriate 1G SFP transceivers for both ends).


5. VLAN Design and IP Addressing

5.1 VLAN Table

VLAN IDNameSubnetPurposeRouted?Internet?
100Management/OOB10.10.100.0/24iLO, UPS, OC200, OPNsense adminYes, restricted❌ No
200Compute10.10.200.0/24Servers, VMs, application nodesYesEgress only
300Storage10.10.300.0/24Ceph OSD, NFS, iSCSINo router❌ No
400Storage-Alt10.10.400.0/24Ceph public networkNo router❌ No
500Services10.10.500.0/24FreeIPA, monitoring, supporting servicesYesEgress only
600K8s Nodes10.10.600.0/24Kubernetes node NICsYesEgress only
610MetalLB10.10.98.0/24K8s LoadBalancer VIPs (ingress)YesIngress from FW
700VPN Clients10.10.700.0/24WireGuard / OpenVPN allocated IPsYesFull access
1000Homestead192.168.125.0/24Residential devicesYesInternet only
IP addressing

VLAN subnets use the VLAN ID in the third octet (10.10.100.x = VLAN 100) for easy readability and troubleshooting. Storage VLANs (300, 400) have no gateway — OPNsense does not have a leg on these VLANs; they are Layer 2 isolated.

5.2 Static IP Assignments — Critical Devices

DeviceHostnameVLANIP AddressRole
OPNsense CARP VIP (gateway)20010.10.200.1Default gateway for all compute
OPNsense CARP VIP (gateway)50010.10.500.1Default gateway for services
OPNsense CARP VIP (gateway)60010.10.600.1Default gateway for K8s nodes
OPNsense CARP VIP (OOB)10010.10.100.1OOB management gateway
opnsense-01opnsense-0120010.10.200.2Primary node own IP
opnsense-02opnsense-0220010.10.200.3Standby node own IP
HP #1 (Proxmox)pmx-0120010.10.200.5Proxmox host
HP #2 (OpenStack)os-0120010.10.200.6OpenStack node [future]
HP #3 (OpenStack)os-0220010.10.200.7OpenStack node [future]
HP #4 (GPU Farm)gpu-farm-0120010.10.200.8GPU Proxmox host
FreeIPA DC-01dc-0150010.10.500.11Primary domain controller
FreeIPA DC-02dc-0250010.10.500.12Replica domain controller
Caddy Reverse Proxyrp-0120010.10.200.22Public URL routing
GitLabgitlab-0120010.10.200.41Source control + CI/CD
NetBird VPNnb-0150010.10.500.25Mesh VPN gateway
QNAP NASqnap-0120010.10.200.31NFS storage (1G)
K8s MetalLB VIP61010.10.98.40All K8s ingress
OC200 Controlleroc200-0110010.10.100.10Omada SDN controller
HP#1 iLO10010.10.100.237Out-of-band management
HP#2 iLO10010.10.100.235Out-of-band management
HP#3 iLO10010.10.100.236Out-of-band management
UPS #1–4ups-01–0410010.10.100.41–44Power management
BeeLink workstationblnk-0120010.10.200.3Dev/management workstation

6. OPNsense Software Stack — Complete Feature List

Both Beelink units run OPNsense CE (Community Edition). The following features are active or planned. All are included with OPNsense CE at no cost.

6.1 Core Features (Built-In, Always Free)

FeatureConfiguration
Stateful FirewallZone-based: WAN / LAN / OOB / Homestead / VPN
CARP HAopnsense-01 primary, opnsense-02 standby, VIP preemption enabled
pfsyncState table sync on dedicated crossover link (10.10.250.0/30)
VLAN (802.1Q)VLANs 100, 200, 500, 600, 610, 700, 1000 on re1 (LAN)
Multi-WANWAN1: fiber (primary), WAN2: ready for 2nd fiber or cable (future)
WireGuard VPNStaff remote access, 4–6 users, 10.10.700.0/24 pool
OpenVPNFallback VPN for devices that cannot run WireGuard
IPsec IKEv2Mobile device VPN (iOS/Android native clients)
Unbound DNSInternal resolver with DNS-over-TLS upstream (Cloudflare 1.1.1.1, Quad9)
DHCPPer-VLAN DHCP pools with static leases for infrastructure devices
NAT/PATOutbound masquerade on WAN, port forwards for inbound HTTPS and VPN
Traffic Shaping / QoSPrioritize VPN and K8s API traffic over bulk storage replication
SyslogAll firewall events → Prometheus syslog receiver → Grafana
2FA Admin UITOTP required for OPNsense web interface (all admin accounts)
Automatic LogoffAdmin session timeout: 15 minutes idle

6.2 OPNsense Plugins (Free, Install via UI)

PluginPurpose
os-suricataIDS/IPS — Suricata with ET Open ruleset (Emerging Threats, free). Monitors WAN for known malicious traffic patterns, drops on IPS mode
os-haproxyLoad balancer — routes 12–18 public URLs to rp-01 (Caddy) with health checks; provides application-level failover to AWS or partner standby
os-acme-clientACME certificates — though Anshin uses acme.sh + Ansible for wildcard certs, this is available for OPNsense's own management certificate
os-zeekNetwork traffic analysis — logs all connection metadata, files transferred, DNS queries to a local index
os-nginxOptional: serve OPNsense admin UI on a non-standard port with mutual TLS
os-netdataHost-level metrics from OPNsense boxes → Prometheus
os-freeradiusRADIUS for future Wi-Fi 802.1X authentication

6.3 Suricata IDS/IPS Configuration

Suricata runs in IPS mode (inline, drop mode) on the WAN interface:

Rule SourceUpdate FrequencyFocus
Emerging Threats OpenDaily (free)Known malicious IPs, C2, exploit signatures
Abuse.ch / ThreatFoxDaily (free)Malware URLs, botnet C2
SANS Internet Storm CenterDaily (free)High-confidence threat intel

Suricata will alert (not drop) on the following to avoid false positives initially:

  • Port scanning originating from internet
  • SQL injection patterns against web ports
  • TLS certificate anomalies

Suricata will drop immediately:

  • Known botnet command-and-control IPs
  • Exploit kit landing page signatures
  • Log4Shell / ProxyShell exploit patterns

All Suricata alerts feed into: OPNsense → syslog → Prometheus → Grafana #alerts channel in Zoho Cliq.


7. Firewall Rules — Intent

Implementation Note

The tables below define the intent of each firewall rule. The deploying engineer will translate these into exact OPNsense rule syntax. Rule order matters — OPNsense processes rules top-to-bottom, first match wins.

7.1 WAN Interface — Inbound Rules

PriorityActionProtocolSourceDestinationPortJustification
1BLOCKAnyKnown bogon/RFC1918AnyAnyAnti-spoofing — inbound private IPs are always malicious
2PASSTCPAnyWAN VIP443HTTPS for all 12–18 public URLs (→ HAProxy → rp-01)
3PASSUDPAnyWAN VIP51820WireGuard VPN
4PASSTCP/UDPAnyWAN VIP1194OpenVPN fallback
5PASSUDPAnyWAN VIP500, 4500IPsec IKEv2
6BLOCKTCPAnyWAN VIP22SSH is VPN-only — never directly from internet
7BLOCKTCPAnyWAN VIP80HTTP redirected at DNS level — not open here
8BLOCKAnyAnyAnyAnyDefault deny — log all

7.2 LAN (Compute VLAN 200) — Inter-VLAN and Outbound Rules

PriorityActionProtocolSourceDestinationPortJustification
1PASSAnyVLAN 200FreeIPA (VLAN 500)88, 389, 636, 464Kerberos + LDAP/S + kpasswd
2PASSTCP/UDPVLAN 200VLAN 500 (dc-01/02)53Internal DNS resolution
3PASSTCPVLAN 200VLAN 610 (MetalLB)443, 80K8s ingress access
4PASSAnyVLAN 600 (K8s)VLAN 200AnyK8s pods reaching compute APIs
5PASSTCPVLAN 200Internet443, 80Software updates, Docker pulls, external APIs
6BLOCKAnyVLAN 200VLAN 100 (OOB)AnyCompute cannot reach iLO/UPS management
7PASSAnyVLAN 200AnyAnyGeneral egress (restrictable per service later)

7.3 OOB Management VLAN 100 — Strict Access Rules

PriorityActionProtocolSourceDestinationPortJustification
1PASSAnyVPN Clients (VLAN 700)VLAN 100AnyAdmin access via VPN only
2PASSTCPVLAN 100Internet443iLO firmware updates
3BLOCKAnyAnyVLAN 100AnyEverything else blocked

7.4 Storage VLANs 300/400 — Not Routed (Layer 2 Only)

OPNsense has no interface on VLANs 300 or 400. These VLANs exist only on sx3008-02 and sg3428-02, which have no router uplink. Storage traffic cannot reach the internet or any other VLAN by design.

7.5 VPN Clients VLAN 700 — Remote Access Rules

PriorityActionProtocolSourceDestinationJustification
1PASSAnyVLAN 700VLAN 100VPN users can reach OOB (iLO, UPS)
2PASSAnyVLAN 700VLAN 200VPN users can reach compute services
3PASSAnyVLAN 700VLAN 500VPN users can reach FreeIPA, monitoring
4PASSTCPVLAN 700Internet443
5BLOCKAnyVLAN 700Homestead (1000)VPN cannot reach residential devices

8. VPN Design — Remote Access for 4–6 Staff

WireGuard is the default VPN protocol. It is built into OPNsense (no plugin needed), uses the WireGuard kernel module for high performance, and the client software is free on all platforms (iOS, Android, macOS, Windows, Linux).

Architecture:

Staff Device (WireGuard client)

│ UDP/51820 — WireGuard tunnel (ChaCha20-Poly1305 encryption)


OPNsense CARP VIP (10.10.96.1 external / WAN VIP)


WireGuard interface (wg0) — VLAN 700 (10.10.700.0/24)

├── Route to VLAN 100 (OOB management)
├── Route to VLAN 200 (compute)
├── Route to VLAN 500 (services, monitoring)
└── Route to VLAN 610 (MetalLB, K8s services)

Per-user configuration:

UserWireGuard IPAccess Level
SRE Admin10.10.700.10Full — all VLANs including OOB
Engineering Director10.10.700.11Full — all VLANs including OOB
Developer 110.10.700.20VLAN 200 + 610 only
Developer 210.10.700.21VLAN 200 + 610 only
Developer 310.10.700.22VLAN 200 + 610 only
Emergency/Spare10.10.700.30Restricted — temporary access

Each user gets a WireGuard config file (or QR code for mobile) generated from OPNsense. The config file is never reused — if a device is lost or an employee leaves, their peer key is removed from OPNsense and they immediately lose access.

8.2 OpenVPN (Fallback — for Devices That Cannot Run WireGuard)

Available on TCP/1194 for corporate firewalls that block UDP, or for devices with WireGuard client issues.

8.3 IPsec IKEv2 (Mobile Devices — iOS/Android Native)

iOS and Android have IPsec IKEv2 clients built in (no app install required). For staff who need mobile access without installing a VPN app.

8.4 Two-Factor Authentication for VPN

VPN access requires certificate + 2FA TOTP (authenticator app):

  • Step 1: WireGuard keypair (cryptographic authentication)
  • Step 2: TOTP code (FreeIPA OTP integrated with OPNsense via RADIUS)

This means a stolen WireGuard config file alone is not enough to gain access.


9. Public URL Routing — 12–18 Services

9.1 Traffic Flow for All Public URLs

User Browser

│ HTTPS (port 443)

OPNsense WAN VIP (public IP: 65.182.226.114)

│ Port forward: 443 → 10.10.200.22:443 (rp-01 Caddy)
│ OR: HAProxy on OPNsense terminates TLS, routes by Host header

HAProxy on OPNsense (health-checked routing)

├── Primary backend: rp-01 (10.10.200.22) ─── Caddy ─── K8s services
└── Fallback backend: [AWS ELB or Partner Standby IP] (if rp-01 health check fails)

9.2 Known Public URLs and Their Internal Routes

The following 14 known services are currently configured in Caddy on rp-01. All use wildcard TLS certificates issued by ZeroSSL via acme.sh.

Public URLInternal UpstreamService
orchestrate.anshin.us10.10.98.40 (MetalLB, NGINX Ingress)Orchestrate main app
dev.orchestrate.anshin.us10.10.98.40Dev environment
api.orchestrate.anshin.us10.10.98.40API gateway
core-dev.orchestrate.anshin.us10.10.98.40Frappe/ERPNext core
erp.anshinhealth.net10.10.98.40ERPNext portal
knowledge.anshin.us10.10.98.40This knowledge base
gitlab.anshinhealth.net10.10.200.41 (direct)GitLab
grafana.mon.anshinhealth.net10.10.98.40 (VPN-only)Grafana monitoring
infisical.svcs.anshinhealth.net10.10.98.40Secrets management
accelerate.onnex.app10.10.98.40Onnex Accelerate
sweetpealinens.com10.10.98.40Sweet Pea Linens
(4–8 additional URLs)10.10.98.40Expansion services

9.3 HAProxy Health Check and Failover Config (Intent)

HAProxy on OPNsense monitors rp-01 with HTTP health checks every 5 seconds:

Frontend: anshin_https
Bind: 0.0.0.0:443 (WAN)
SSL: terminate with wildcard certs (loaded from Ansible-deployed K8s secrets)
Default backend: anshin_primary

Backend: anshin_primary (rp-01)
Server: rp-01 10.10.200.22:443 check inter 5s fall 2 rise 2
Health check: GET /health → expect HTTP 200

Backend: anshin_failover (AWS or Partner Standby)
Server: aws-lb [AWS ELB DNS or static IP]:443 check inter 10s
OR
Server: partner-standby [Partner external IP]:443 check inter 10s

Failover behavior: If 2 consecutive health checks to rp-01 fail (10 seconds), HAProxy routes all traffic to the failover backend automatically. When rp-01 recovers (2 consecutive successful checks), traffic shifts back. No DNS change required — failover is sub-second at the HAProxy level.


10. Application Failover Design

10.1 Failover Tiers

TierTargetTriggerRecovery
Tier 1 — PrimaryAnshin Lab (rp-01 → K8s)Always preferred
Tier 2 — AWSAWS ELB (Anshin-managed AWS deployment)rp-01 health check fails for 10sAuto by HAProxy
Tier 3 — Partner StandbyPartner-hosted infrastructureAWS also unavailableManual DNS switch or HAProxy Tier 3 backend

10.2 What Stays in Sync for Failover to Work

For a failover backend to serve users correctly, it must have:

RequirementCurrent StatusImplementation
Identical application code🔵 Needs CI/CD pipelineGitLab CI pushes to AWS + lab on every main branch deploy
Database replication🔵 Needs designMariaDB → AWS RDS read replica or periodic snapshot restore
Session state🔵 Needs designRedis/Valkey session state → AWS ElastiCache mirror
TLS certificates✅ acme.sh wildcardSame certs deployed to AWS and partner standby via Ansible
DNS TTL low🔵 Needs actionAll public URLs must be at TTL 60s to enable fast DNS failover as backup
Failover requires bilateral partner agreement

Application failover to a partner lab requires a formal agreement covering:

  • Network path (private WireGuard tunnel between Anshin Lab and partner infrastructure)
  • Data replication (no PHI until HIPAA BAA is in place with the partner)
  • Capacity (can the partner environment serve Anshin's traffic load?)
  • Monitoring (escalation path when Anshin failover triggers to the partner)

10.3 WAN Failover (Future — When Second IP Is Added)

When a second fiber or cable IP is available:

OPNsense Multi-WAN (built-in, no plugins needed):

WAN1: Fiber primary (current) ─── Gateway Group: ANSHIN_WAN, Tier 1
WAN2: Fiber-2 or Cable ────────── Gateway Group: ANSHIN_WAN, Tier 2

Policy-based routing:
- All outbound traffic → ANSHIN_WAN gateway group
- If WAN1 drops → OPNsense automatically routes via WAN2 (no user impact)
- If WAN1 recovers → traffic returns to WAN1 (preempt enabled)

Load balancing option:
- Weight WAN1:WAN2 = 2:1 (fiber 2x preference over cable)
- Both active simultaneously (bandwidth aggregation)

11. Security Controls — HIPAA Mapping

11.1 Network Security

ControlImplementationHIPAA §164.312
Perimeter firewallOPNsense CE, stateful inspection, default-deny(e)(2)(i)
IDS/IPSSuricata with ET Open + Abuse.ch rules(e)(2)(i)
Network segmentation8 VLANs, storage fully isolated, OOB isolated(e)(2)(i)
Encrypted transportTLS 1.2+ enforced, TLS 1.0/1.1 disabled at OPNsense(e)(2)(ii)
VPN for remote accessWireGuard (ChaCha20) + 2FA TOTP required(e)(1)
DNS securityUnbound with DNSSEC validation, DNS-over-TLS upstreamSupportive

11.2 Access Control

ControlImplementationHIPAA §164.312
Unique user identificationFreeIPA (LDAP/Kerberos) — every service authenticates against IPA(a)(2)(i)
Emergency accessiLO out-of-band on isolated OOB VLAN, break-glass procedure documented(a)(2)(ii)
Automatic logoffOPNsense: 15 min, Frappe: 30 min, Grafana: 60 min(a)(2)(iii)
Encryption/decryption (addressable)TLS everywhere; LUKS full-disk encryption on storage VMs when deployed(a)(2)(iv)

11.3 Audit Controls

What Is LoggedWhereRetention
All firewall allow/deny eventsOPNsense → syslog → Prometheus → Grafana90 days hot, archive
All VPN connections (who, when, from where)OPNsense WireGuard log90 days
All IDS/IPS alertsSuricata → syslog90 days
All K8s API callskube-apiserver audit log → Prometheus90 days
All Frappe/ERPNext API callsBFF Audit Log DocType90 days (purged weekly)
All admin logins to OPNsenseOPNsense auth log90 days

11.4 Incident Response — Network Events

EventDetectionResponse
Suricata critical alertZoho Cliq #alerts within 30sSRE on-call investigates → block IP at OPNsense
VPN brute force (5 failed auths)OPNsense auth log → alertAutomatic block of source IP for 24h
Unexpected port scanSuricata alertAlert + add to OPNsense blocklist
iLO/OOB access from non-VPNFirewall deny log → alertImmediate investigation
DDoS on WANSuricata + OPNsense rate limitingNull route at ISP + haproxy 429

12. Omada SDN — Switch Management

12.1 What Omada Manages

The OC200 controller manages only the TP-Link switch fabric (sx3008-01, sx3008-02, sg3428-01, sg3428-02, sg3452-01). It does not manage OPNsense — OPNsense has its own web interface and API.

12.2 OC200 Placement and Access

ItemValue
DeviceTP-Link Omada OC200 hardware controller
Power sourcePoE from ex2200p-01 port 1 (isolated OOB switch)
Management IP10.10.100.10 (OOB VLAN 100 only)
Web UIhttps://10.10.100.10:8043 (VPN required)
REST APIhttps://10.10.100.10:8043/api/v2/

12.3 Omada REST API — Claude Automation Potential

The Omada controller exposes a full REST API. A mcp-omada MCP server would allow Claude Code to:

CapabilityAPI Endpoint
List all switches and healthGET /sites/{id}/devices
Show per-port traffic statsGET /sites/{id}/stat/port
Change port VLAN assignmentPATCH /sites/{id}/ports/{portId}
Show all connected clientsGET /sites/{id}/clients
View active alertsGET /sites/{id}/alerts
Reboot a switchPOST /sites/{id}/cmd/devices

Combined with the OPNsense REST API (via mcp-opnsense), the entire network stack becomes manageable through Claude — Bryan can ask plain-English questions and get actionable answers.


13. Physical Network Topology Diagrams

13.1 Internet Edge — WAN Design

══════════════════════════ INTERNET ══════════════════════════

ISP Fiber (1× static public IP: 65.182.226.114)

┌─────────────────┐
│ ISP Modem │
│ (passthrough │
│ / bridge mode)│
└────────┬────────┘

┌──────────────┴──────────────┐
│ (splitter / modem 2nd port) │
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ opnsense-01 │ │ opnsense-02 │
│ (PRIMARY) │◄──────── │ (STANDBY) │
│ Beelink EQ12 │ pfsync │ Beelink EQ12 │
│ re0: WAN │ crossover│ re0: WAN │
│ re1: LAN ──────┤ ├── re1: LAN │
└────────┬────────┘ └────────┬────────┘
│ │
└──────────────┬─────────────┘
│ CARP VIP: 10.10.96.1
│ (both connect to sg3428-01)


══════════════════════════ INTERNAL ══════════════════════════

13.2 Internal Network Plane — 10G + 1G

sg3428-01 (1G Network Access)
┌─────────────────────────────┐
opnsense-01 ──┤ P1 SFP1────────────────────┼──── sx3008-01 Port 6 (1G uplink)
opnsense-02 ──┤ P2 │
pmx-01 eno1 ──┤ P3 │
pmx-01 eno2 ──┤ P4 │
hp4 eno1 ──┤ P7 │
rp-01 ──┤ P8 │
gitlab-01 ──┤ P9 │
dc-01 ──┤ P10 │
dc-02 ──┤ P11 │
blnk-01 ──┤ P12 │
└─────────────────────────────-┘

sx3008-01 (10G Network Spine)
┌─────────────────────────────┐
pmx-01 sfp0 ──┤ P1 │
hp2 sfp0 ──┤ P2 [future] │
hp3 sfp0 ──┤ P3 [future] │
hp4 sfp0 ──┤ P4 │
hp5 sfp0 ──┤ P5 [planned] │
(10G uplink)──┤ P6 [OPNsense 10G if added] │
sg3428-01 ──┤ P7 (1G SFP uplink trunk) │
sx3008-02 ──┤ P8 (inter-switch, mgmt only)│
└─────────────────────────────┘


│ VLANs 200, 600, 610 distributed
│ to all server 10G ports at line rate

13.3 Storage Plane — 10G ISOLATED

sx3008-02 (10G Storage Spine) ⚠️ NO WAN UPLINK
┌─────────────────────────────┐
pmx-01 sfp1 ──┤ P1 │
hp2 sfp1 ──┤ P2 [future] │
hp3 sfp1 ──┤ P3 [future] │
hp4 sfp1 ──┤ P4 │
QNAP 10G ──┤ P5 (if SFP+ NIC installed) │
hp5 sfp1 ──┤ P6 [planned] │
sg3428-02 ──┤ P7 (1G storage fallback) │
sx3008-01 ──┤ P8 (inter-switch, mgmt only)│
└─────────────────────────────┘

sg3428-02 (1G Storage Fallback) ⚠️ NO WAN UPLINK
┌─────────────────────────────┐
pmx-01 eno3 ──┤ P1 │
pmx-01 eno4 ──┤ P2 │
hp4 eno3 ──┤ P7 │
hp4 eno4 ──┤ P8 │
QNAP RJ45 ──┤ P9 │
sx3008-02 ──┤ SFP1 │
└─────────────────────────────┘

13.4 Out-of-Band Management Plane — ISOLATED

ex2200p-01 (OOB Management) ⚠️ ISOLATED
┌─────────────────────────────┐
oc200-01 ──┤ P1 (PoE powered) │
hp1 iLO ──┤ P2 │
hp2 iLO ──┤ P3 │
hp3 iLO ──┤ P4 │
hp4 iLO ──┤ P5 │
ups-01 ──┤ P6 │
ups-02 ──┤ P7 │
ups-03 ──┤ P8 │
ups-04 ──┤ P9 │
opns-01 con ──┤ P10 (USB-serial adapter) │
opns-02 con ──┤ P11 (USB-serial adapter) │
blnk-01 ──┤ P12 │
└─────────────────────────────┘

Access: VPN required (VLAN 700 → VLAN 100 allowed by firewall)

14. Open Technical Questions for Reviewer Validation

#QuestionNotes
1VLAN numbering — does this match the sre-iac IP scheme?sre-iac uses 10.20.0.0/24 (VLAN 200), 10.50.0.0/24 (VLAN 500) — see comparison doc
2pfsync: dedicated crossover cable vs. VLAN on LAN interface?Crossover is cleaner; VLAN is simpler. Recommend crossover for HIPAA isolation
3OPNsense WAN: does the ISP modem support 2-port LAN, or is a WAN splitter needed?Determines physical WAN setup for CARP standby
4QNAP storage NIC: does qnap-01 have an SFP+ port installed?If yes → sx3008-02 Port 5 at 10G. If no → sg3428-02 at 1G
5Suricata mode for first deployment: IPS (drop) or IDS (alert only)?Recommend IDS first; promote to IPS after 2-week baseline review
61G SFP modules: sx3008-01 Port 6 ↔ sg3428-01 SFP1 — are compatible 1G SFP modules available?Need 2× 1G SFP multimode or LC fiber, or 1× 0.3m DAC if same chassis
7VLAN subnets vs. existing FreeIPA DNS zones — do FreeIPA reverse zones need updating?FreeIPA zones may need PTR zone additions for 10.10.200.0/24 etc.
8Partner failover: static external IP confirmed? WireGuard peer config needed?Required before Tier 3 failover can be activated
9HIPAA BAA status with any partner hosting failover traffic?No PHI in failover path until BAA is signed and verified

15. Budget Summary (Network Hardware Only)

ItemQtyEst. UnitEst. Total
Beelink EQ12 Pro2$189$378
TL-SX3008F (10G switch)2$240$480
TL-SG3428 (1G switch)2$200$400
TL-SG3452 (48-port homestead)1$350$350
Omada OC200 controller1$80$80
10Gtek dual SFP+ NIC (servers)4$49$196
H!Fiber 1.5m DAC cables8$9.49$76
H!Fiber 0.3m DAC cables3$9.49$28
USB 3.0 to Gigabit (pfsync)2$15$30
1G SFP transceivers (inter-switch)4$8$32
Network hardware total~$2,050

Note: Juniper EX2200-24P (OOB switch), HP servers, QNAP NAS, and UPS units already on-site — not included above.


16. Document Control

RevDateAuthorChange
1.02026-03-22Anshin EngineeringInitial release — complete network and security architecture
1.12026-03-22Anshin EngineeringRemoved named reviewers; generalized reviewer questions

Deployment sequence:

  1. Deploy OPNsense CARP pair (opnsense-01, opnsense-02) — replace Cisco bridge
  2. Deploy TP-Link 10G switch fabric (sx3008-01, sx3008-02, sg3428-01, sg3428-02)
  3. Install 10Gtek NICs in all HP servers; connect DAC cables
  4. Configure VLANs — migrate from flat /20 to segmented scheme
  5. Enable WireGuard VPN — provision staff peer configs
  6. Deploy HAProxy — configure primary and failover backends
  7. Enable Suricata IDS (alert mode first, promote to IPS after baseline)
  8. Decommission Cisco bridge
  9. Validate all 14+ public URLs, VPN access, OOB management path