Originated: October 27, 2015
********************************************************************************

Patch-164059: Patch for OneFS 7.2.1.1. This patch addresses multiple issues
that affect the HDFS service. 

--------------------------------------------------------------------------------

This patch can be installed on clusters running the following OneFS version: 
7.2.1.1

As of December 18, 2015, the following bugs addressed by this patch
are not yet fixed in a OneFS 7.2.1.1 maintenance release:
163360
162584
162583
161579
161650
158083

********************************************************************************
ENHANCEMENTS

* Bug ID 163360
  When verifying compatibility with a version of Apache Ambari, OneFS now
  checks only the first two digits of the Ambari version number. This enables
  OneFS to maintain compatibility when minor Ambari updates are released.  

RESOLVED ISSUES

* Bug ID 162584
  When Kerberos authentication was enabled, it was possible for a race
  condition to occur in the isi_hdfs_d process if two HDFS clients
  simultaneously attempted to establish a Simple Authentication Security
  Layer (SASL) connection to the cluster. If this condition occurred, the
  isi_hdfs_d process failed and created an hdfs.core.gz file in the
  /var/crash directory.
  
* Bug ID 162583
  Beginning in Ambari 2.0, when Ambari is upgraded, the Ambari Server sends an
  install_package request to each host's Ambari agent. Because, OneFS was not
  designed to respond to this request, during an Ambari upgrade, when the
  request was sent to the cluster, the Ambari upgrade stopped progressing
  until the Ambari database was modified to allow it to continue.

  This patch enables OneFS to correctly respond to install_package requests so
  that Ambari 2.0 and later upgrades progress as expected without manual
  intervention.  

* Bug ID 161579
  A change in the implementation of the -appendtofile parameter of the hdfs dfs
  command in Apache Hadoop 2.7 could have caused the command to fail when run
  on a OneFS cluster. If this occurred, messages similar to the following
  appeared in the OneFS command-line interface:
 
  -appendToFile: Fatal internal error
  java.lang.NullPointerException
                at org.apache.hadoop.hdfs.DFSOutputStream.isLazyPersist
				(DFSOutputStream.java:1709)
                at org.apache.hadoop.hdfs.DFSOutputStream.getChecksum4Compute
				(DFSOutputStream.java:1550)
                at org.apache.hadoop.hdfs.DFSOutputStream.<init>
				(DFSOutputStream.java:1560)
                at org.apache.hadoop.hdfs.DFSOutputStream.<init>
				(DFSOutputStream.java:1667)
                at org.apache.hadoop.hdfs.DFSOutputStream.newStreamForAppend
				(DFSOutputStream.java:1694)
                at org.apache.hadoop.hdfs.DFSClient.callAppend
				(DFSClient.java:1824)
                at org.apache.hadoop.hdfs.DFSClient.append
				(DFSClient.java:1885)
                at org.apache.hadoop.hdfs.DFSClient.append
				(DFSClient.java:1855)
                at org.apache.hadoop.hdfs.DistributedFileSystem$4.doCall
				(DistributedFileSystem.java:340)
                at org.apache.hadoop.hdfs.DistributedFileSystem$4.doCall
				(DistributedFileSystem.java:336)
                at org.apache.hadoop.fs.FileSystemLinkResolver.resolve
				(FileSystemLinkResolver.java:81)
                at org.apache.hadoop.hdfs.DistributedFileSystem.append
				(DistributedFileSystem.java:336)
                at org.apache.hadoop.hdfs.DistributedFileSystem.append
				(DistributedFileSystem.java:318)
                at org.apache.hadoop.fs.FileSystem.append
				(FileSystem.java:1164)
                at org.apache.hadoop.fs.shell.CopyCommands$AppendToFile.
				processArguments(CopyCommands.java:341)
                at org.apache.hadoop.fs.shell.Command.processRawArguments
				(Command.java:201)
                at org.apache.hadoop.fs.shell.Command.run
				(Command.java:165)
                at org.apache.hadoop.fs.FsShell.run(FsShell.java:287)
                at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
                at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
                at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
  
* Bug ID 161650
  Under some circumstances, the HDFS process on a node could have run out of 
  system memory while DataNodes were transferring data to the HDFS server. If 
  this occurred, the HDFS process on the affected node restarted, and HDFS 
  clients connected to that node were disconnected.

* Bug ID 158083
  A memory leak in the data path between DataNodes and the cluster could have
  caused the HDFS process to unexpectedly restart. If this occurred, HDFS
  clients connected to an affected node were disconnected.
 
********************************************************************************

CHANGED FILES

* /usr/bin/isi_hdfs_d
 (MD5 ab30d7465140b08f506ec64267d7c86a)

* /usr/lib/libisi_hdfs_protobuf.so.1
 (MD5 bb34fd87f39d24c85c8cf8e972f3369e)

* /usr/lib/libisi_zone.so.1
 (MD5 2ef31e7f989a9210129eda810f6e15b5)

* /usr/likewise/lib/libisilon_registry_gcfg.so.0
 (MD5 0f1189ba6692f04a348263503299da5f)

* /usr/local/share/zsh/site-functions/_isi
 (MD5 a74d93690354f66f9d640c84cf975d57)

* /usr/local/share/CLI/controllers/isi/zone/util.py
 (MD5 c9677ca041ce67e93bd84180084f535a)

* //usr/share/man/man8/isi-zone.8.gz
 (MD5 c8944ded51db03b9d8f3f0cc50a3b9de)

* /usr/likewise/share/config/lsassd.reg
 (MD5 2b59f6c4260785dc776f9499255bd930)

* /usr/share/papi/doc/1/zones/GET_output_schema.json
 (MD5 69045652fd1cd3074a450ff8d5a1d193)

* /usr/share/papi/doc/1/zones/POST_input_schema.json
 (MD5 b98ce04bd0c6d635344130876b0bd829)

* /usr/share/papi/doc/1/zones/<ZONE>/GET_output_schema.json
 (MD5 69045652fd1cd3074a450ff8d5a1d193)

********************************************************************************

INSTALLATION/REMOVAL IMPACTS

During installation or removal of this patch, the HDFS, MCP, and PAPI services 
and the OneFS web administration interface will be unavailable and HDFS clients
will be disconnected. After installation or removal of the patch is complete,
the MCP and PAPI services will be automatically restarted and HDFS clients must
reestablish a connection to the cluster.

********************************************************************************

IMPORTANT!  
VERIFY CLUSTER-FREE SPACE
If there is insufficient free space and you attempt to install or remove a
patch, the process might not complete successfully. An incomplete patch
installation or patch removal can leave the cluster in an inaccessible state.

Before installing or removing a patch, verify that your cluster has enough free
space to perform the procedure. For more information about maintaining
sufficient free space on an Isilon cluster, see the Best Practices Guide for
Maintaining Enough Free Space on Isilon Clusters and Pools:
https://support.emc.com/docu48119.

********************************************************************************

INSTALLING THIS PATCH

IMPORTANT!
Read INSTALLATION/REMOVAL IMPACTS before performing this procedure.
   
1. Verify that your cluster has enough free space to install this patch.

2. Open an SSH connection on any node in the cluster and log in by using the
   root account.

   NOTE: If the patch is being installed on a compliance mode cluster, log in
   using the compadmin account.

3. Copy the patch-164059.tgz file to the /ifs/data directory on the cluster.

4. Run the following command to change to the /ifs/data directory:
   
   cd /ifs/data

5. To extract the patch file, run the following command:

   tar -zxvf patch-164059.tgz

6. To install this patch, run the following command:

   isi pkg install patch-164059.tar

7. To verify that this patch is installed, run the following command:
  
   isi pkg info

8. Confirm that patch-164059 appears in the list of installed packages.

********************************************************************************

REMOVING THIS PATCH

If you need to remove this patch, complete the steps below.

IMPORTANT!
Read INSTALLATION/REMOVAL IMPACTS before performing this procedure.

1. Verify that your cluster has enough free space to remove this patch.

2. Open an SSH connection on any node in the cluster and log in using the
   root account.

   NOTE: If the patch is being removed from a compliance mode cluster, log in
   using the compadmin account.

3. To remove this patch, run the following command:

   isi pkg delete patch-164059

4. To verify that this patch was removed, run the following command:

   isi pkg info

5. Confirm that patch-164059 does not appear in the list of installed packages.

********************************************************************************