Integrations
Integrations
Breadcrumbs

Ranger

Integration model

Active governance

Access management requires the “Tot plugin Ldap” plugin to generate the groups that represent the DSAs and insert or remove users from those groups.

In turn, Ranger is required to synchronize those groups in order to use them in its policies.

The variables shown in the following screenshots are the properties that will need to be adapted to the LDAP and Ranger configuration in use.

att_17_for_194641994.png
att_15_for_194641994.png
att_14_for_194641994.png
att_1_for_194641994.png


This plugin will create or update HDFS and/or Hive policies to grant or revoke permissions to the groups represented by DSAs.

Required credentials

To allow the plugin to connect to Ranger, two things are required: a service user with the appropriate permissions and credentials, and certificates in the JVM to enable the SSL connection to Ranger.

Certificate generation and usage

To allow the connection between the plugin and CDP, a series of preliminary steps must be performed to enable SSL on the cluster and generate the necessary certificates that the plugin requires.

SSL/TLS activation process with CDP

Currently with Cloudera it is necessary to activate SSL/TLS to be able to use Ranger. If the wizard for this process fails, there is a high probability of leaving the cluster completely inoperable.

Preliminaries

During the execution of the wizard, Cloudera Manager must connect, in our case with the root user, to all the servers in the environment, using a password or using public/private keys.

For the public/private key pair, the root public key must be deployed to all nodes, in addition to enabling root login in the SSH server configuration.

Bash
sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/g' /etc/ssh/sshd_config
sed -i 's/PasswordAuthentication no/PasswordAuthentication yes/g' /etc/ssh/sshd_config


It is recommended to perform a manual connection test from Cloudera Manager to all servers in the Cluster beforehand.

Wizard

The wizard consists of two phases

  • Generate the CA

  • Summary screen

When the summary screen is shown to us, we must only press the finish button on the summary screen when the described operation has completed.

The wizard is launched from the Administration menu > Security > Enable TLS

att_16_for_194641994.png
Wizard::Generate the CA

This first screen of the wizard allows us to either use a CA provided to us or have Cloudera generate a self-signed CA.

In our case we select the options:

  • All existing clusters and future clusters.

  • All hosts accept the same password.

  • ssh user name and password, in our case root

att_2_for_194641994.png
Wizard::Summary

On this screen we must NOT press finish until the operation shown is complete.

The operation is:

  1. connect to the Cloudera Manager machine

  2. Restart Cloudera Manager

  3. Verify in the log when the restart has completed

Only when the Cloudera Manager service has completely restarted will we finish the wizard


att_3_for_194641994.png
Post-installation

After the wizard finishes we will need to update the obsolete configurations of the clients and applications.

TrustStores and Keystores

Cloudera stores the JKS for certificates and keys in: /var/lib/cloudera-scm-agent/agent-cert


cm-auto-in_cluster_truststore.jks

cm-auto-in_cluster_ca_cert.pem

cm-auto-global_truststore.jks

cm-auto-global_cacerts.pem

cm-auto-host_key_cert_chain.pem

cm-auto-host_key.pw

cm-auto-host_key.pem

cm-auto-host_cert_chain.pem

cm-auto-host_keystore.jks


The password can be obtained using this command:


Bash
grep -Eo "Djavax.net.ssl.trutStorePassword=[[:alnum:]]*" $(find /run/cloudera-scm-agent/process/ -name "proc.json" | grep HIVESERVER2 | sort | head -1) | sed  's/Djavax.net.ssl.trustStorePassword=//'

SSL/TLS connection with CDP

The plugin uses a Java library to connect via SSL to Ranger; it is necessary for the plugin to have access to the previously generated certificates in order to connect.

There are two ways to do this:

Include the certificate in the plugin startup command by adding the following JVM variables (required):

Bash
-Djavax.net.ssl.trustStore={ruta}\cm-auto-global_truststore.jks
-Djavax.net.ssl.trustStorePassword=****

Install the certificates in the JVM where the plugin runs


User credentials

Creation of a service user of type User is required, either created from Ranger or synchronized from LDAP.


att_4_for_194641994.png


This user needs to be present in certain HDFS and Hive policies that allow them to modify all the elements to be governed.


If the policies are to be in a particular security zone (configurable), it is necessary for the service user to be an administrator of the security zone, either nominally or by group membership.


att_5_for_194641994.png


HDFS

In the default policy “all - path” created by Ranger in HDFS, add the user that Anjana will use, granting all permissions and checking the “Delegate Admin” option.

att_8_for_194641994.png
att_9_for_194641994.png


Hive

Similarly to HDFS, in the default policy “all - database, table, column” of Hadoop SQL, add the user that Anjana will use, granting all permissions and checking the “Delegate Admin” option.

att_11_for_194641994.png
att_12_for_194641994.png


Anjana Role

For grouping the permissions governed by Anjana, it is necessary to create a role in Ranger that contains no users, groups, or roles; practically it will function as a tag on a permissions record. The role name must be Anjana Role.

att_6_for_194641994.png


Ranger Limitations

It is only possible to have one enabled policy over a particular set of objects (e.g.: it is not possible to have 2 policies acting on the same file or table at the same time, but it is possible to have one policy acting only on the file and another acting on the file and another one), for this reason the plugin will only act on policies that apply to a single resource, whether a file/folder (HDFS) or a table (Hive).

To make it easy to know which policies the plugin has created/modified, a tag will be applied to the policy (Anjana Governed). In addition, all permissions governed by Anjana will be grouped in a single record with the empty role created earlier.

att_10_for_194641994.png


With this tag, it is easy to filter which policies Anjana is intervening in and, within them, which permissions are governed by the plugin, so that if it is necessary to modify the policy manually, it can be identified as having been created by the plugin (and should not be modified).

att_7_for_194641994.png

Anjana Limitations

Due to the processing of paths at the Hive level, the path of the object in Anjana representing a Hive table must be composed of one of these two forms:

  • catalog or data source/database/table (EX: hive/db1/table1)

  • database/table (EX: db1/table1)

The path separator character “/” is configurable (see the configuration example)

In cases where Active Directory is used with sAMAaccountName, due to restrictions of that technology, the group name must not exceed 20 characters.


Plugin actions

Create/Expand policy

This action is executed when a DSA is approved in Anjana that contains a governed object with the triplet configured in the plugin. This action will be executed with a configurable delay in the plugin, because it is necessary to wait for Ranger to synchronize with the LDAP the group previously created in the Ldap plugin.

As explained in the limitations, first it will be checked whether a policy exists that acts exclusively on the resource represented by the object; if not found, one will be created. On this policy, a new access record will be created for the group that the DSA represents and will be granted the corresponding permissions (read/execute in HDFS and select in Hive), as well as including the Anjana Governed tag for visibility.

Additionally, every policy from this action will be automatically enabled. In the event that a disabled policy existed for the resource, it will be cleared of all previous permissions or exclusions before being processed.

In the case of HDFS policies, if the policy pointing to the resource is not recursive, the plugin will make it recursive, leaving it in the same state as if the plugin had created it.


Delete/Reduce policy

This action is executed when a DSA is expired in Anjana that contains a governed object with the triplet configured in the plugin, or when the governed object expires and is present in some DSA. The access records will be removed from the policy associated with the resource.

In cases where there are still accesses governed by Anjana and the policy is found to be disabled, it will be re-enabled.

In cases where, after removing Anjana's access, there are still accesses managed by the client, the Anjana Governed tag will be removed to indicate that Anjana no longer governs that resource; and in the event that those were the only existing records, the policy will also be deleted.


Configuration example

YAML
server:
 port: 15021

totplugin:
 server:
   urls:
   - http://totserver:15000/tot/
 aris:
   - ari: "anja:totplugin:im:/cloudera/hive/devQA/"
     imAri: "anja:totplugin:im:/ldap/ldap/ldap/"
   - ari: "anja:totplugin:im:/cloudera/hdfs/devQA/"
     imAri: "anja:totplugin:im:/ldap/ldap/ldap/"
   - ari: "anja:totplugin:im:/cloudera/ranger/devQA/"
     imAri: "anja:totplugin:im:/ldap/ldap/ldap/"
 connection:
   urlBaseRanger: "https://ip-10-150-100-136.eu-central-1.compute.internal:6182"
   userRanger: <user>
   pwdRanger: ******
 delayInSeconds: 0
 securityZone: "sales"
 hive:
   hiveService: cm_hive
   valueTechnologyHive: hive
   hiveAuditLogin: true
   replaceHiveClientPolicyByAnjanaPolicyName: true
   pathSplit: "/"
 hdfs:
   hdfsService: cm_hdfs
   valueTechnologyHdfs: hdfs
   hdfsAuditLogin: true
   replaceHdfsClientPolicyByAnjanaPolicyName: true

eureka:
  client:
    serviceUrl:
      defaultZone: http://totserver:15000/tot/eureka


The common configurations must be reviewed in the Anjana Data 4.4 configuration document - DS - Technical configuration of Portal and microservices


Specific configurations:

  • Connection:

    • urlBaseRanger: Ranger access URL

    • userRanger: Plugin service user

    • pwdRanger: Service user password

  • delayInSeconds: Delay in seconds, used during DSA approval; it is recommended to set it to slightly more than the LDAP synchronization time configured in Ranger. Its default value is 0.

  • securityZone: Optional field; if filled in, it indicates the security zone where the policies will be created.

  • hive:

    • hiveService: Name of the Hive service within Ranger to which the policies are to be applied.

    • valueTechnlogyHive: The value in the technology field that governed objects must have to be identified as Hive and have their policies created in its service.

    • hiveAuditLogin: Optional field, indicates whether the hive policies created or updated by the plugin should be audited. It is false by default.

    • replaceHiveClientPolicyByAnjanaPolicyName: Optional field, indicates whether the plugin is allowed to modify the name of the hive policies it updates with the name Anj_<Id of the object representing the table>. It is false by default.

    • pathSplit: Optional field, the character by which the path of a Hive object is processed to obtain its database and the table it represents. (EX: for database/table it would be the character /).

  • hdfs:

    • hdfsService: Name of the HDFS service within Ranger to which the policies are to be applied.

    • valueTechnlogyHdfs: The value in the technology field that governed objects must have to be identified as HDFS and have their policies created in its service.

    • hdfsAuditLogin: Optional field, indicates whether the HDFS policies created or updated by the plugin should be audited. It is false by default.

    • replaceHdfsClientPolicyByAnjanaPolicyName: Optional field, indicates whether the plugin is allowed to modify the name of the hdfs policies it updates with the name Anj_<Id of the object representing the file(s)>. It is false by default.