Many open source projects use Bugzilla, JIRA, or some other workflow where contributions are submitted via a patch file attached to a bug report or even an email. This workflow means working with a raw patch file. Working with these files can be a bit of a pain as one tries to figure out what… Read More »
"Super Heroes" (CC BY-SA 2.0) by Olaf Gradin A lot of work has been done to greatly clarify and enhance various environment variables in the Apache Hadoop shell script code. One of those places was in the usage of various _USER environment variables. Prior to 3.0.0-alpha4 In previous releases, the supported variables were: Name Description HADOOP_SECURE_DN_USER User to… Read More »
A recent Linux kernel update to workaround CVE-2017-1000366 is causing Apache Hadoop’s secure DataNode (and NFS manager) to crash on startup. (Related discussion from Red Hat and Ubuntu) If your systems are running a variant of Apache Hadoop 3.x, you can take advantage of user functions to workaround the Java Invocation API issue that causes… Read More »
One of the features in Apache Hadoop 3.x is the ability to replace how the shell scripts work without having to support a code fork. This feature makes adding support for site or OS specific features, such as resource controls around daemons, really easy. To keep things simple, let’s add Linux cgexec support for non-secure… Read More »
One of the big pain points of administrating Apache Hadoop is the ability to safely and efficiently add to the classpath. The original design in Hadoop gave users a single way to add jars: the HADOOP_CLASSPATH environment variable. This is a bit of a problem for end users, admins, and any 3rd party applications may… Read More »