Network Operations Center Technician (Tier 1)
AiNet · Beltsville, MD
Apr 2025 – Present
Larger changes are reviewed and signed off by senior engineers before they run.
- Operate production infrastructure across a fleet of 13+ KVM hypervisors, 40+ Cisco/Juniper switches, and an 800-drive object-storage cluster.
- Found and scoped a critical private-key exposure in internal certificate distribution, then wrote the remediation plan: keys regenerated, not just reissued, because the renewal client reused them.
- Root-caused an object-storage cluster whose replication never settled: its ring had about 1,000 partitions spread over 800+ drives, far too coarse to balance, made worse by full and unmounted drives. Laid out the fix options for the senior team.
- Found a production VM still running on a disk image that had been deleted out from under it, and rescued it live: copied the disk from the open file handle and pivoted the guest onto the copy with no downtime.
- Rebuilding the production mail platform on fresh Ubuntu 26.04 VMs: rebuilt the mail stack, restored six databases, and moved the first 80 GB (475,000 files) in under an hour ahead of cutover.
- Restored a 10G DWDM circuit to a remote site that had been unreachable for about 10 months: moved the /30, MTU, and OSPF point-to-point config onto the newly lit channel, then cleared an asymmetric MTU mismatch that had left the adjacency stuck in EXSTART.
- Ended a recurring multi-tenant out-of-disk ticket by tracing it to monthly ZFS snapshots on a parent dataset pinning everything a 12-hour cleanup cron deleted; reclaimed 2.72 TB on a dry-run-verified destroy and fixed the recurrence at the config layer, taking the pool from 94% to about 21% used.
- Diagnosed a fleet-wide monitoring blind spot where a local disk check pasted one host's result across 84 hosts, hiding real full disks; fixed it fleet-wide, and later added 72 health checks across 18 storage nodes.
- Designed and piloted the VM fleet backup strategy: changed-block incremental capture pushed to OpenStack Swift object storage with dedup, encryption, and retention, every job monitored in Nagios.
- Built a 6-GPU inference rig from decommissioned datacenter hardware into a 48 GB VRAM pool serving a 35B-parameter model at ~33 tok/s, the compute base for an internal ops agent; handed off for agent bring-up.
- Planned and executed live RAID 0 to RAID 5 migrations with zero guest downtime, including a 3-drive rebuild on the largest host.
- Migrated a production BGP peering session from an aging 3750 to a 6500-class switch, documenting neighbor, prefix-list, and route-map config before a clean cutover.
- Deployed an OpenVAS scanner in a dedicated VM with custom profiles aligned to the asset inventory; ran the recurring monthly scan and documented findings.