"Super Heroes" (CC BY-SA 2.0) by Olaf Gradin A lot of work has been done to greatly clarify and enhance various environment variables in the Apache Hadoop shell script code. One of those places was in the usage of various _USER environment variables. Prior to 3.0.0-alpha4 In previous releases, the supported variables were: Name Description HADOOP_SECURE_DN_USER User to… Read More »
A recent Linux kernel update to workaround CVE-2017-1000366 is causing Apache Hadoop’s secure DataNode (and NFS manager) to crash on startup. (Related discussion from Red Hat and Ubuntu) If your systems are running a variant of Apache Hadoop 3.x, you can take advantage of user functions to workaround the Java Invocation API issue that causes… Read More »
A while back, Jessie Frazelle wrote and published an informative blog post on the differences between containers, zones, and jails. Since it touched on security, the blog post reminded me of a conversation that was had last year when a contributor to the Apache Yetus project asked about this blog post about one of the… Read More »
There’s been a lot of chatter in the Apache Hadoop universe lately over the role of the HDFS audit log. This includes a lot of supposition on the purpose of the original design. This is my recollection of the events that led to its creation. Unfortunately, humans do not come with ECC RAM so apologies… Read More »
One of the features in Apache Hadoop 3.x is the ability to replace how the shell scripts work without having to support a code fork. This feature makes adding support for site or OS specific features, such as resource controls around daemons, really easy. To keep things simple, let’s add Linux cgexec support for non-secure… Read More »
One of the big pain points of administrating Apache Hadoop is the ability to safely and efficiently add to the classpath. The original design in Hadoop gave users a single way to add jars: the HADOOP_CLASSPATH environment variable. This is a bit of a problem for end users, admins, and any 3rd party applications may… Read More »