Checklist for a Secure Hadoop Cluster

There are some standard best practices for securing a Hadoop cluster. When securing a cluster, we are trying to address the concerns specified in Primer on security in a Data System. Let us quickly review what are some of essential measures required in a secure Hadoop cluster. These areformed based on my years of expertise in securing Hadoop clusters  at eBay.


RequirementMore DetailsEnabled? (Y/N)
RPC Authentication Hadoop secure Mode
HTTP authentication Hadoop HTTP Authentication
WebHDFS AuthenticationWebHDFS Authentication
Block Token
Kerberos one way trust
Service Level AuthorizationService Level Authorization in Hadoop
Yarn Queue Permissions
HDFS PermissionsHDFS Permissions
Audit Logging for HDFS and Yarn
Yarn Secure ContainersYARN Secure Containers
EncryptionData Encryption
Perimeter Security




You May Also Like

About the Author: Benoy Antony

I am an Apache Hadoop Committer and has been working as an engineer/architect at companies like eBay and Paypal. Please check my LinkedIn Profile for the full profile.

Leave a Reply

Your email address will not be published. Required fields are marked *