CDP Part 4: Managing Users in CDP Public Cloud with Keycloak – News Block

Leave a Comment / By /

Previous articles in the series have covered implementing a CDP public cloud environment. All the components are ready to use and it’s time to make the environment available to other users to explore all the services and run their experiments.

This article is the fourth in a series of six:

Onboarding users to work in a CDP environment and services is accomplished in two ways, by configuring an identity provider or importing users in bulk.

This article focuses on configuring Keycloak as an Identity Provider (IdP) for CDP. Keycloak allows us to configure the federated identity with CDP, since it is compatible with the Security Assertion Markup Language protocol (SAML 2.0). Identity federation allows users to log in to the CDP console via Keycloak authentication without signing up for Cloudera or creating a Cloudera account.

User onboarding and configuring Keycloak as an identity provider for CDP is done through the Keycloak web interface (management console) or using the terminal. Both approaches are covered.

From the CDP side, Account administrator either PowerUser Roles are required to complete this process.

Deploy Keycloak using an AWS EC2 instance

Clearly, we need to implement Keycloak before we start configuring the identity provider in CDP.

To deploy Keycloak with a minimal installation so that users can log in over the Internet, we deployed it with Docker on an AWS EC2 instance.

To do this, you need an EC2 instance with the following characteristics:

  • Instance type: t2.micro
  • YO SOY: ami-05e9058d6cda9ba0d
  • docker compose installed
  • public IP address
  • entry rule 8443 port

Once you have created the instance, take note of the following information:

  • IPv4 public DNS: <PUBLIC_IPV4_DNS>
  • Public IP address: <ec2_public_ip>
  • EC2 key pair: <AWS_SSH_KEY>

For a complete guide to launching an AWS EC2 instance that meets these requirements, see this article Configuring Keycloak on an EC2 Instance.

Once your EC2 instance is ready, you’re ready to begin your Keycloak deployment.

local machine:

Set required EC2 instance variables

export PUBLIC_IPV4_DNS=<PUBLIC_IPV4_DNS>
export AWS_SSH_KEY=<AWS_SSH_KEY>

export EC2_USERNAME=ec2-user

Connect to the EC2 instance:

ssh -i "~/.ssh/${AWS_SSH_KEY}.pem" ${EC2_USERNAME}@${PUBLIC_IPV4_DNS}

EC2 instance:

create the docker-compose.yml archive:

cat <<EOF > docker-compose.yml
version: '3'
services:
  keycloak:
    # The 20.0.0 keycloak image
    image: quay.io/keycloak/keycloak:20.0.0
    container_name: keycloak
    environment:
      # Keycloak admin credentials
      KEYCLOAK_ADMIN: admin
      KEYCLOAK_ADMIN_PASSWORD: Password123
    ports:
      # The HTTPS port
      - 8443:8443
      # The HTTP port
      - 8080:8080
    command: start-dev
    volumes:
      # To save the SAML metadata file
      - ./kc_config:/kc_config
volumes:
  kc_config:
    external: true
EOF

Create the outer volume folder:

Start the docker composition:

Verify that Keycloak is running using Docker:

To access Keycloak management console (Keycloak web interface) you need to disable the SSL protocol:

  • Enter the container:

    docker exec -it keycloak bash

  • Add the path required to use the kcadm.sh script:

    export PATH=$PATH:/opt/keycloak/bin

  • Provide the server and credentials for the master Kingdom:

    kcadm.sh config credentials --server http://localhost:8080/ \
--realm master \
--user $KEYCLOAK_ADMIN \
--password $KEYCLOAK_ADMIN_PASSWORD

  • Disable SSL for the master Kingdom:

    kcadm.sh update realms/master -s enabled=true -s sslRequired=none

  • The Keycloak web interface is now available at the following address: http://<ec2_public_ip>:8080.

Configuration via the Keycloak web interface

It is recommended to use the Keycloak web interface if you are new to Keycloak and want to better understand how to bind CDP with Keycloak.

If you prefer to start with an easy-to-reproduce approach, which allows you to set up and create users faster, scroll down to the Configuration Using Terminal section.

kingdom creation

Realm is one of the core concepts of Keycloak, a Realm secures and manages security metadata for a set of users.

To create a Kingdom:

  • navigate to http://<ec2_public_ip>:8080click management console:


    keycloak_admin_console01

  • Sign in with your KEYCLOAK_ADMIN and KEYCLOAK_ADMIN_PASSWORD defined in the docker-compose.yml archive:


  • Click the dropdown in the top left corner where it says teacherthen click create kingdom:


  • Fill in the name and click Create:


  • navigate to Realm Settings and click SAML 2.0 Identity Provider Metadata:


  • This opens another tab with the SAML metadata needed to register the Realm as an IdP in CDP:


  • Download this file at xml Format. The right button of the mouse > Save page as… > saml_metada.xml:


  • To disable the SSL requirement: go back to Realm Settings Select the General token, set SSL required to None and click Save:


Keycloak registration in CDP

In this step, Keycloak registers as an identity provider with CDP. This requires the previously downloaded SAML metadata file.

As a reminder, you need account manager either power user roles in your CDP account to complete this process.

To create the identity provider in CDP, the following steps are required.

  1. Access your CDP console and select management console:


  2. navigate to User Management > identity providers > Create identity provider:


  3. Create identity provider:

    • Enter the name you want to use for the CDP identity provider.
    • check the Synchronize groups at login Option: This synchronizes the user group membership in CDP with the user group membership in Keycloak.
    • In Provider metadataselect Upload file and raise the saml_provider.xml downloaded in the previous step.
    • Click Create.


  4. Copy the <ID> generated after build process


This <ID> it is added to our Keycloak configuration in the next step.

Create the CDP client in Keycloak

In this step, the CDP control plane client is created in our Keycloak implementation.

There are two ways to create a client in Keycloak:

  • Using entity descriptor: Import the client using a standard SAML entity descriptor json file.
  • Manually via the Keycloak UI using the CDP SAML service provider metadata available in the identity provider created in the previous step. (See Managing OpenID Connect and SAML clients for more details.)

In this case, we use a SAML entity descriptor file: cdp-client.

Observation: The cdp client was found in this article: How to configure single sign-on (SSO) for CDP Public Cloud in the open with Keycloak.

The author clarifies that this file is not official or provided by Cloudera, but includes the required configuration, SAML assertion, and signing certificates required by CDP.

To create a SAML 2.0 client in Keycloak by importing a configuration file, follow these steps:

  1. in the key layer management console navigate to Clientele and click Create


  2. Click Browse and upload the cdp-client. This automatically updates the Customer identification and Guy parameters Click Save


  3. To get the login page urlnavigate to the Settings tab and scroll down until IDP Initiated SSO URL Name variable.

    It could look like this: http://<ec2_public_ip>:8080/realms/adaltas-cdp-lab/protocol/saml/clients/cdp-sso.



  4. navigate to Advanced tab, replace ${CDP_IdP_ID} with the identity provider ID generated by CDP in the previous step. Scroll down and click Save


  5. try the login page url:


Manage groups and users

Now that Realm and the client are ready, we create/manage users and groups through the management console.

Create a group:

  1. navigate to groups in the menu on the left and click create a group:


  2. Fill in the name and click Save:


    Note: Be careful when choosing the GROUP_NAME since there is a list of group names that are reserved and therefore cannot be synced with CDP.

Create a user:

  1. navigate to users in the menu on the left and click create new user:


  2. Set the following parameters and click Create:

    • Username
    • Email
    • First name
    • Last name
    • Select the previously created group


  3. To set the password, go to Credentialsfill in values ​​with one click set the key:


  4. Test the user by logging in:



Configuration using the terminal

This approach is for users more familiar with Keycloak or for those who want an automated process.

As you may have already noticed, configuring Keycloak using the web interface can be tedious and a bit long. Especially the user creation task, for those dealing with large groups or even just for testing purposes, the web interface is not the best idea.

Configuration via the terminal requires the following steps:

  1. Sign in and check EC2 requirements
  2. Create credentials and log in to Realm Master
  3. Create the realm and download SAML metadata
  4. Keycloak registration in CDP
  5. Create CDP client
  6. create users
  7. Create group and add users

Sign in and check EC2 requirements

As we mentioned before, we assume that Keycloak is running using Docker Compose on an EC2 instance.

So here we are going to connect to this EC2 instance and go into the Docker container to run all the commands.

local machine:

  1. Set the required EC2 instance variables:

    export PUBLIC_IPV4_DNS=<PUBLIC_IPV4_DNS>
export AWS_SSH_KEY=<AWS_SSH_KEY>

export EC2_USERNAME=ec2-user

  2. Connect to the EC2 instance and verify that the files are there:

    ssh -i "~/.ssh/${AWS_SSH_KEY}.pem" ${EC2_USERNAME}@${PUBLIC_IPV4_DNS}

The docker container is running and should have the following layout:

./ubuntu
├── docker-compose.yml
└── kc_config
1 directories, 1 file

Now, we are ready to begin the setup process.

EC2 instance:

  1. Go inside the container:

    docker exec -it keycloak bash

  2. Navigate to the outer volume directory:

Create credentials and log in to Realm Master

This step creates a truststore credential needed to log in to the Realm master through the terminal.


export PATH=$PATH:/opt/keycloak/bin

kcadm.sh config credentials --server http://localhost:8080/ \
   --realm master \
   --user $KEYCLOAK_ADMIN \
   --password $KEYCLOAK_ADMIN_PASSWORD

Create the realm and download SAML metadata

Now it’s time to register our Realm and get the SAML metadata needed to register Keycloak as an IdP in CDP.


export REALM="adaltas-cdp-lab"

export SAML_FILE=saml_provider.xml

kcadm.sh create realms \
   --set realm=$REALM \
   --set enabled=true \
   --set sslRequired=none

curl \
  -k http://localhost:8080/realms/adaltas-cdp-lab/protocol/saml/descriptor \
  > $SAML_FILE

Since we need this file in the next step, use the following command to copy the SAML metadata on your local machine:

scp -i "~/.ssh/${AWS_SSH_KEY}.pem" ${EC2_USERNAME}@${PUBLIC_IPV4_DNS}:/home/${EC2_USERNAME}/kc_config/saml_provider.xml ./

Keycloak registration in CDP

Use the Cloudera web interface to register Keycloak as an IdP as described in the web interface approach.

NoteNote: At the time this article was written, this process could not be performed through the CDP CLI because the cdp iam create-saml-provider command did not work.

Create CDP client

In this step, we are going to create the CDP client in Keycloak using the content of the cdp_client file and the <ID> generated by CDP from the previous step.


kcadm.sh create clients --target-realm $REALM -f - << EOF
{
   "clientId": "cdp-client",
   "surrogateAuthRequired": false,
   "enabled": true,
   "clientAuthenticatorType": "client-secret",
   "redirectUris": (
       "https://consoleauth.altus.cloudera.com/saml"
   ),
   "webOrigins": (
       "https://consoleauth.altus.cloudera.com"
   ),
   "notBefore": 0,
   "bearerOnly": false,
   "consentRequired": false,
   "standardFlowEnabled": true,
   "implicitFlowEnabled": false,
   "directAccessGrantsEnabled": false,
   "serviceAccountsEnabled": false,
   "publicClient": false,
   "frontchannelLogout": true,
   "protocol": "saml",
   "attributes": {
       "saml.assertion.signature": "true",
       "saml.force.post.binding": "true",
       "saml.multivalued.roles": "false",
       "saml.encrypt": "false",
       "saml_assertion_consumer_url_post": "https://consoleauth.altus.cloudera.com/saml?samlProviderId=${CDP_IdP_ID}",
       "saml.server.signature": "true",
       "saml_idp_initiated_sso_url_name": "cdp-sso",
       "saml.server.signature.keyinfo.ext": "false",
       "exclude.session.state.from.auth.response": "false",
       "saml.signature.algorithm": "RSA_SHA256",
       "saml_force_name_id_format": "false",
       "saml.client.signature": "true",
       "tls.client.certificate.bound.access.tokens": "false",
       "saml.authnstatement": "true",
       "display.on.consent.screen": "false",
       "saml_name_id_format": "username",
       "saml.onetimeuse.condition": "false",
       "saml_signature_canonicalization_method": "http://www.w3.org/2001/10/xml-exc-c14n#"
   },
   "authenticationFlowBindingOverrides": {},
   "fullScopeAllowed": true,
   "nodeReRegistrationTimeout": -1,
   "protocolMappers": (
       {
           "name": "my-email-id-mapper",
           "protocol": "saml",
           "protocolMapper": "saml-user-property-mapper",
           "consentRequired": false,
           "config": {
               "attribute.nameformat": "URI Reference",
               "user.attribute": "email",
               "friendly.name": "my-email-friendly-name",
               "attribute.name": "urn:oid:0.9.2342.19200300.100.1.3"
           }
       },
       {
           "name": "my-groups-mapper",
           "protocol": "saml",
           "protocolMapper": "saml-group-membership-mapper",
           "consentRequired": false,
           "config": {
               "single": "true",
               "attribute.nameformat": "URI Reference",
               "full.path": "false",
               "friendly.name": "my-groups-friendly-name",
               "attribute.name": "https://altus.cloudera.com/SAML/Attributes/groups"
           }
       },
       {
           "name": "my-firstname-mapper",
           "protocol": "saml",
           "protocolMapper": "saml-user-property-mapper",
           "consentRequired": false,
           "config": {
               "attribute.nameformat": "URI Reference",
               "user.attribute": "firstName",
               "friendly.name": "my-firstname-friendly-name",
               "attribute.name": "https://altus.cloudera.com/SAML/Attributes/firstName"
           }
       },
       {
           "name": "my-lastname-mapper",
           "protocol": "saml",
           "protocolMapper": "saml-user-property-mapper",
           "consentRequired": false,
           "config": {
               "attribute.nameformat": "URI Reference",
               "user.attribute": "lastName",
               "friendly.name": "my-lastname-friendly-name",
               "attribute.name": "https://altus.cloudera.com/SAML/Attributes/lastName"
           }
       }
   ),
   "defaultClientScopes": (
       "web-origins",
       "role_list",
       "profile",
       "roles",
       "email"
   ),
   "optionalClientScopes": (
       "address",
       "phone",
       "offline_access",
       "microprofile-jwt"
   ),
   "access": {
       "view": true,
       "configure": true,
       "manage": true
   }
}
EOF

create users

We create the users by reading a csv file with the following format:

Name, LastName, username, email

Therefore, within your kc_config directory (EC2 or Docker Container) create a csv file with the information of each user.

Note: Review the user and group limits and make sure you don’t exceed these limits.

Here’s an example:

cat <<EOF > users_list.csv
Tobias, CHAVARRIA, user01, user01@example.com
Albert, KONRAD, user02, user02@example.com
Stephan, BAUM, user03, user03@example.com
EOF

Once this is done, we create the users using the following scripts:


export USERS_CSV_FILE=users_list.csv

export USERS_INITIAL_PASS="Pa55w0rd"

kc_create_user() {
  firstname=$1
  lastname=$2
  username=$3
  email=$4

  kcadm.sh create users -r $REALM \
   -s username=$username \
   -s email=$email \
   -s firstName=$firstname \
   -s lastName=$lastname \
   -s enabled=true

  kc_set_password $username $USERS_INITIAL_PASS
}

kc_set_password() {
  username="$1"
  initial_pass="$2"
  kcadm.sh set-password \
    -r $REALM \
    --username $username --new-password $initial_pass
}



import_users() {
  while read -r line; do
   IFS=',' read -ra arr <<< "$line"
   kc_create_user ${arr(0)} ${arr(1)} ${arr(2)} ${arr(3)}
  done < "$USERS_CSV_FILE"
}

import_users $USERS_CSV_FILE

Create group and add users

In this step, we are going to create a group and add ALL existing users in the given Realm in this group.


export GROUP_NAME="adaltas-students"

export USERS_ID_LIST=users_list

kcadm.sh create groups -r $REALM -s name=$GROUP_NAME

export GROUP_ID=$(kcadm.sh get groups -r $REALM | grep -o '"id" : "(^")*' | grep -o '(^")*$')

kcadm.sh get users -r $REALM | grep -o '"id" : "(^")*' | grep -o '(^")*$' >> $USERS_ID_LIST

kc_add_group() {
  userid="$1"
  groupid="$2"

  kcadm.sh update users/$userid/groups/$groupid \
    -r $REALM \
    -s realm=$REALM \
    -s userId=$userid \
    -s groupId=$groupid \
    -n
}


add_users_group() {
  n=1
  while read -r userid; do
  
  read -ra arr <<< "$userid"
  echo "Updating user: $userid"
  n=$((n+1))
  kc_add_group $userid $GROUP_ID
  done < $USERS_ID_LIST
}

add_users_group $USERS_ID_LIST

Once configured, users can use the following login page to access the Cloudera console:

  • http://<ec2_public_ip>:8080/realms/<realm>/protocol/saml/clients/cdp-sso

For additional configuration or review, you can also access the Keycloak web interface here:

  • http://<ec2_public_ip>:8080

Next Step: Configuring Ranger Policies and Permissions

At this point, Keycloak has been configured as the identity provider for CDP and we have created our user pool.

There is still one more layer of configuration needed in the user onboarding process. In the next chapter, we will show how to configure the necessary permissions and policies to allow users to exploit our end-to-end architecture.

Related Posts

Leave a Comment

Your email address will not be published. Required fields are marked *

Exit mobile version