Hadoop Eco System has many tools as you already know.Some of them are HDFS , Hive, Oozie and Falcon etc.All these tools provide REST API so that other tools can communicate with them. Every tool will have hostname and port number as part of their REST API URL.With respect to security , It is not good practice to expose internal host names and port numbers. Some body might try to attack using them.
To address this problem , we have a security tool called Apache Knox. Apache Knox is a REST API based gateway security tool that provides perimeter security for all Hadoop services.
Apache Knox hides REST API URLs of all hadoop services for external hadoop clients.They will only use REST API provided by Apache Knox . Knox will delegate external hadoop client requests to corresponding hadoop services. And before delegating hadoop client requests , Knox provides all security services configured on the cluster.
Below are some more important points of Apache Knox.
- Demo LDAP is by default available for Apache Knox.
- Kerberos is optional for Apache Knox but can easily be integrated with knox.
- External clients need not remember all REST API URLs of all hadoop services.
- Provides Audit log
- Provides authorization even including service level authorization