Originated: April 7, 2016
********************************************************************************

Patch-169252: Patch for OneFS 7.2.1.0 - 7.2.1.2. This patch implements a change
to address an issue where a node's baseboard management controller (BMC) stops
responding.

--------------------------------------------------------------------------------

This patch can be installed on clusters running the following OneFS versions: 
7.2.1.0 - 7.2.1.2

********************************************************************************

ENHANCEMENTS

* Bug ID 166917
  Enables OneFS to automatically reset an unresponsive BMC to restore
  communications between the BMC and the hardware sensors in a node.
 
********************************************************************************

CHANGED FILES

* /usr/local/lib/python2.6/site-packages/isi/hw/ipmi.so
  (MD5 = 36717ddb50c8d07a27b784986d0e717b)

* /usr/local/lib/python2.6/site-packages/isi/hw/hwmon/cmc.pyc
  (MD5 = f2a88bde1e9f61992805310f26e03407)

********************************************************************************

INSTALLATION/REMOVAL IMPACTS

During installation or removal of this patch, clients and applications that are
connected to the cluster through BXE 10 GigE network interfaces will be 
disconnected. These connections must be reestablished after the patch
installation is complete. 

To ensure the changes made by this patch take effect, you must perform a
rolling reboot for all S210, X210, X410, NL410 and HD400 nodes in the cluster
at the end of the patch installation or removal process.
 
A rolling reboot requires that you reboot each node in succession manually
from the command-line interface. When a node is rebooted, client connections
to that node will be disconnected and might need to be reestablished. 

Alternatively, you can perform a simultaneous reboot for all nodes on the
cluster.
 
Whether you perform a simultaneous reboot or rolling reboot of the nodes, we 
recommend that you install the patch during an off-hours maintenance window
to minimize the disruption of service to clients.
 
If you cannot install the patch during a scheduled maintenance window, inform
users when the reboot will take place and how they will be affected. 
 
Instructions for how to perform a simultaneous reboot or rolling reboot from
the command-line interface appear later in this document.

********************************************************************************

IMPORTANT!  
VERIFY CLUSTER-FREE SPACE
If there is insufficient free space and you attempt to install or remove a
patch, the process might not complete successfully. An incomplete patch
installation or patch removal can leave the cluster in an inaccessible state.

Before installing or removing a patch, verify that your cluster has enough
free space to perform the procedure. For more information about maintaining
sufficient free space on an Isilon cluster, see the Best Practices Guide for
Maintaining Enough Free Space on Isilon Clusters and Pools:
https://support.emc.com/docu48119.

********************************************************************************

INSTALLING THIS PATCH

IMPORTANT!
Read INSTALLATION/REMOVAL IMPACTS before performing this procedure.
   
1. Verify that your cluster has enough free space to install this patch.

2. Open an SSH connection on any node in the cluster and log in by using the
   root account.

   NOTE: If the patch is being installed on a compliance mode cluster, log in
   using the compadmin account.

3. Copy the patch-169252.tgz file to the /ifs/data directory on the cluster.

4. Run the following command to change to the /ifs/data directory:
   
   cd /ifs/data

5. To extract the patch file, run the following command:

   tar -zxvf patch-169252.tgz

6. To install this patch, run the following command:

   isi pkg install patch-169252.tar

7. To verify that this patch is installed, run the following command:
  
   isi pkg info

8. Confirm that patch-169252 appears in the list of installed packages.

9. If the cluster contains S210, X210, X410, NL410 or HD400 nodes, perform 
   a rolling reboot for those nodes or a simultaneous reboot for all nodes.
   Instructions for how to perform a simultaneous reboot or a rolling reboot
   from the command-line interface appear later in this document.

********************************************************************************

PERFORMING A SIMULTANEOUS REBOOT

After the patch is installed, manually reboot all nodes. 


1. Open an SSH connection on any node in the cluster and log on using the 
   root account.

   NOTE: If the procedure is being done on a compliance mode cluster, log in
   using the compadmin account.
 
2. Run the following cluster administration command to enter the isi config
   subsystem:
  
    isi config
 
3. To reboot all the nodes in the cluster, run the following command:
 
     reboot all
 
4. At the prompt, enter yes.

********************************************************************************

PERFORMING A ROLLING REBOOT

After the patch is installed, manually reboot each node in succession. 

1. Open an SSH connection on any node in the cluster and log on using the 
   root account.

   NOTE: If the procedure is being done on a compliance mode cluster, log in 
   using the compadmin account.
   
2. Run the following command to start the Isilon IQ configuration console: 
   
   isi config

3. Within the configuration console, reboot any node in the cluster, except
   the node that you are connected to, by running the following command,
   where <node> is the node that you want to reboot:

   reboot <node>

4. To exit the configuration console, run the following command:

   exit

5. To monitor the progress of the reboot, run the following command at any
   time from the node that you are connected to: 
   
   sysctl efs.gmp.group
   
   Output is displayed in the following format:   
   
   efs.gmp.group: <Node,Sequential number>: { <Array IDs>:<Drive IDs> }
      
   For example, 
      
   if node 6 is marked for group removal:
   efs.gmp.group: <2,15>: { 1-5:0-3, shutdown_read_only: 6 }
      
   if node 6 is rebooting:
   efs.gmp.group: <2,15>: { 1-5:0-3, down: 6 }
   
   if node 6 is back online:
   efs.gmp.group: <2,15>: { 1-6:0-3 }
   
   NOTE: This example assumes that only one node at a time is being rebooted
   and no other devices are down.  The array ID might not match the nod
   number. 
   
   For more information about the efs.gmp.group sysctl, see article 89502 on
   the EMC Online Support site.
   
6. Wait for the node to successfully reboot.
   
7. Repeat step 3 for each remaining node, except for the node that you are 
   connected to.

8. To exit the configuration console, run the following command:

   exit

9. To reboot the node that you are connected to, run the following command:
   
   shutdown -r now   

   NOTE: Running the command in step 9 will close your SSH connection to the 
   cluster.  This is true only for this step; no other step will disconnect the 
   SSH session. 

********************************************************************************

REMOVING THIS PATCH

If you need to remove this patch, complete the steps below.

IMPORTANT!
Read INSTALLATION/REMOVAL IMPACTS before performing this procedure.

1. Verify that your cluster has enough free space to remove this patch.

2. Open an SSH connection on any node in the cluster and log in using the
   root account.

   NOTE: If the patch is being removed from a compliance mode cluster, log in
   using the compadmin account.

3. To remove this patch, run the following command:

   isi pkg delete patch-169252

4. To verify that this patch was removed, run the following command:

   isi pkg info

5. Confirm that patch-169252 does not appear in the list of installed packages.

6. If the cluster contains S210, X210, X410, NL410 or HD400 nodes, perform 
   a rolling reboot for those nodes or a simultaneous reboot for all nodes.
   Instructions for how to perform a simultaneous reboot or a rolling reboot
   from the command-line interface appear earlier in this document.

********************************************************************************
