A hardware failure caused a storage controller failover on one of our server chassis early Thursday (7/15/21) morning. Servers recovered and no latency was occurring due to low load on the agreement platform. Once load increased around 8:15am, more underlying storage issues were occured. We moved production servers to another cluster to mitigate any issues while working with our hardware provider to resolve the latency. Latency was resolved with a firmware update to a certain set of hard drive model that is present in our cluster as well as the chassis firmware.