Originated: April 7, 2016
********************************************************************************

Patch-169252: Patch for OneFS 7.2.1.0 - 7.2.1.2. This patch implements a change
to address an issue where a node's baseboard management controller (BMC) stops
responding.

--------------------------------------------------------------------------------

This patch can be installed on clusters running the following OneFS versions: 
7.2.1.0 - 7.2.1.2

********************************************************************************

ENHANCEMENTS

* Bug ID 166917
  Enables OneFS to automatically reset an unresponsive BMC to restore
  communications between the BMC and the hardware sensors in a node.
 
********************************************************************************

CHANGED FILES

* /usr/local/lib/python2.6/site-packages/isi/hw/ipmi.so
  (MD5 = 36717ddb50c8d07a27b784986d0e717b)

* /usr/local/lib/python2.6/site-packages/isi/hw/hwmon/cmc.pyc
  (MD5 = f2a88bde1e9f61992805310f26e03407)

********************************************************************************

INSTALLATION/REMOVAL IMPACTS

To ensure that the changes made by this patch take effect, you must reboot 
all nodes in the cluster at the end of the patch installation process. You must 
also reboot all nodes in the cluster if you remove this patch. You can either 
reboot the nodes in succession (also known as a rolling reboot), or perform a 
simultaneous reboot of all nodes in the cluster.  
 
A simultaneous reboot can be performed from the OneFS web administration
interface or from the command-line interface. A simultaneous reboot of the 
cluster is faster than a rolling reboot; however, because all nodes are rebooted 
simultaneously, this type of reboot causes a temporary interruption of service 
while the nodes reboot.
 
A rolling reboot requires that you reboot each node in succession manually
from the command-line interface. When a node is rebooted, client connections
to that node will be disconnected and might need to be reestablished. 
 
Whether you perform a simultaneous reboot or a rolling reboot of the nodes, 
we recommend that you install the patch during an off-hours maintenance 
window to minimize the disruption of service to clients.
 
Instructions for how to perform a simultaneous reboot or a rolling reboot 
from the command-line interface appear later in this document.

********************************************************************************

IMPORTANT!  
VERIFY CLUSTER-FREE SPACE
If there is insufficient free space and you attempt to install or remove a
patch, the process might not complete successfully. An incomplete patch
installation or patch removal can leave the cluster in an inaccessible state.

Before installing or removing a patch, verify that your cluster has enough
free space to perform the procedure. For more information about maintaining
sufficient free space on an Isilon cluster, see the Best Practices Guide for
Maintaining Enough Free Space on Isilon Clusters and Pools:
https://support.emc.com/docu48119.

********************************************************************************

INSTALLING THIS PATCH

IMPORTANT!
Read INSTALLATION/REMOVAL IMPACTS before performing this procedure.
   
1. Verify that your cluster has enough free space to install this patch.

2. Open an SSH connection on any node in the cluster and log in by using the
   root account.

   NOTE: If the patch is being installed on a compliance mode cluster, log in
   using the compadmin account.

3. Copy the patch-169252.tgz file to the /ifs/data directory on the cluster.

4. Run the following command to change to the /ifs/data directory:
   
   cd /ifs/data

5. To extract the patch file, run the following command:

   tar -zxvf patch-169252.tgz

6. To install this patch, run the following command:

   isi pkg install patch-169252.tar

7. To verify that this patch is installed, run the following command:
  
   isi pkg info

8. Confirm that patch-169252 appears in the list of installed packages.

********************************************************************************

PERFORMING A SIMULTANEOUS REBOOT

After the patch is installed, manually reboot all nodes. 


1. Open an SSH connection on any node in the cluster and log on using the 
   root account.

   NOTE: If the procedure is being done on a compliance mode cluster, log in
   using the compadmin account.
 
2. Run the following cluster administration command to enter the isi config
   subsystem:
  
    isi config
 
3. To reboot all the nodes in the cluster, run the following command:
 
     reboot all
 
4. At the prompt, enter yes.

********************************************************************************

PERFORMING A ROLLING REBOOT

After the patch is installed, manually reboot each node in succession. 

1. Open an SSH connection on any node in the cluster and log on using the 
   root account.

   NOTE: If the procedure is being done on a compliance mode cluster, log in 
   using the compadmin account.
   
2. Run the following command to start the Isilon IQ configuration console: 
   
   isi config

3. Within the configuration console, reboot any node in the cluster, except
   the node that you are connected to, by running the following command,
   where <node> is the node that you want to reboot:

   reboot <node>

4. To exit the configuration console, run the following command:

   exit

5. To monitor the progress of the reboot, run the following command at any
   time from the node that you are connected to: 
   
   sysctl efs.gmp.group
   
   Output is displayed in the following format:   
   
   efs.gmp.group: <Node,Sequential number>: { <Array IDs>:<Drive IDs> }
      
   For example, 
      
   if node 6 is marked for group removal:
   efs.gmp.group: <2,15>: { 1-5:0-3, shutdown_read_only: 6 }
      
   if node 6 is rebooting:
   efs.gmp.group: <2,15>: { 1-5:0-3, down: 6 }
   
   if node 6 is back online:
   efs.gmp.group: <2,15>: { 1-6:0-3 }
   
   NOTE: This example assumes that only one node at a time is being rebooted
   and no other devices are down.  The array ID might not match the nod
   number. 
   
   For more information about the efs.gmp.group sysctl, see article 89502 on
   the EMC Online Support site.
   
6. Wait for the node to successfully reboot.
   
7. Repeat step 3 for each remaining node, except for the node that you are 
   connected to.

8. To exit the configuration console, run the following command:

   exit

9. To reboot the node that you are connected to, run the following command:
   
   shutdown -r now   

   NOTE: Running the command in step 9 will close your SSH connection to the 
   cluster.  This is true only for this step; no other step will disconnect the 
   SSH session. 

********************************************************************************

REMOVING THIS PATCH

If you need to remove this patch, complete the steps below.

IMPORTANT!
Read INSTALLATION/REMOVAL IMPACTS before performing this procedure.

1. Verify that your cluster has enough free space to remove this patch.

2. Open an SSH connection on any node in the cluster and log in using the
   root account.

   NOTE: If the patch is being removed from a compliance mode cluster, log in
   using the compadmin account.

3. To remove this patch, run the following command:

   isi pkg delete patch-169252

4. To verify that this patch was removed, run the following command:

   isi pkg info

5. Confirm that patch-169252 does not appear in the list of installed packages.

6. Reboot the cluster after removing the patch.

********************************************************************************
