FandO is performing inspection and remediation work on PDU3A4 feeding the Network and Tape rooms in the b725 datacenter today ~9:30am-3pm. All equipment is powered redundantly from another PDU, no outage is expected, but systems will be in a more at-risk state than normal.

SCDF Mattermost service has been updated and service has resumed, thank you for your patience. If there are any issues please report problems by opening an RT ticket in the Software queue 'RT-RACF-Software@bnl.gov'

An emergency update needs to be applied to the SCDF Mattermost service, users should not experience any interruption to usage, however if there is an issue the service will be down temporarily. This update will fix an issue where users are unable to retrieve group messages using the '+' button from the sidebar and will update the webui. Apologies for the interruption and thank you for your patience.

At this time the SCDF Mattermost service scheduled maintenance has completed and service has resumed. Users should now be able to use the service, you may be prompted to log in again if you were in the middle of a session. If there are any issues please open an RT ticket in the Software queue 'RT-RACF-Software@bnl.gov' and thank you again for your patience.

A planned maintenance of the SCDF Mattermost will be performed today, 12/01/2025, between 5-6pm EST. The service will be unavailable during this time and following the restoration of the service users may begin using it normally. Thank you for your patience during this scheduled outage.

FandO reported PDU1A3 is back up to 187F, but on the connections for all 3 phases this time.

SCDF is shedding load from row 103 by draining jobs and powering off idle nodes to get to
a stable safe state for the long holiday weekend.

This only impacts sPHENIX and shared pool condor users/experiments.

An upgrade was begun today on OpenShift. Though we were informed that it would be transparent to all, we have found that some network interruptions are occurring. The upgrade cannot be interrupted once it has started. The Main servers have completed, and we do not expect any further interruptions. However, there is a possibility of further interruption to services as the upgrade proceeds.

An unexpected incident with the electrical power system in the Scientific Computing and Data Facility (SCDF) at BNL has resulted in the temporary loss of some computing resources for sPHENIX and ATLAS Tier-1.

New user requests are not generating RT emails or tickets.

Migrating RHIC AFS servers from Red Hat Virtual environment to OpenShift. Will migrate 1 server at a time, but this will include some disruption in file access as it proceeds.