WAN connectivity under investigation
Incident
Report for IT
Postmortem
During the planned WAN hardware replacement, the connection on the second fiber path began to degrade. This degradation resulted in the loss of network connectivity for the UNO campus, and also impacted the UNK authentication servers, and other various web applications.
The WAN connection started to become unstable around 10:00am, and fully degraded by 10:20am. After reviewing the options and impact, the team began rolling back to the old hardware at 10:30am. All paths and services were confirmed back online and operational by 11:00am.
ITS WAN engineering will reevaluate the fiber span and reschedule the planned work.
Resolved
All systems are operating normally. This issue is resolved.
Monitoring
All issues appear to have subsided, and all systems are performing as expected. We're continuing to monitor for any reoccurring problems.
Identified
ITS has identified the cause of the issues and is taking steps to remediate impact to affected services.
The outage is related to unplanned impacts from scheduled maintenance on the network backbone (RFC 111459 - https://status.nebraska.edu/incidents/7j5r5szl5ywh), which is being rolled back at this time.
The outage is related to unplanned impacts from scheduled maintenance on the network backbone (RFC 111459 - https://status.nebraska.edu/incidents/7j5r5szl5ywh), which is being rolled back at this time.
Investigating
A problem with the NU WAN is under investigation. Engineers are investigating the issue. This is affecting network connectivity in multiple locations including the UNO campus, VPN service, and data centers.
Questions 402-472-3970 or support@nebraska.edu
Questions 402-472-3970 or support@nebraska.edu
This incident affected: Infrastructure (Data Center Services, Network Backbone, OMAHA - Wired Network Connections, OMAHA - Wireless Network Connections) and Information Security (Virtual Private Network (VPN)).