1 Introduction

In this tutorial, we will show how to add a custom provider to the popular open source identity management solution Keycloak so that we can use it with existing and/or non-standard user storage.

2.Overview of custom providers with Keycloak

The ready-made Keycloak provides a series of standards-based integration based on protocols such as SAML, OpenID Connect and OAuth2. Although this built-in function is very powerful, sometimes it is not enough . A common requirement, especially when it comes to old systems, is to integrate users of these systems into Keycloak. In order to adapt to this and similar integration schemes, Keycloak supports the concept of custom providers.

Custom providers play a key role in Keycloak's architecture. For each main function, such as login process, identity verification, authorization, there is a corresponding service provider interface. This approach allows us to plug in a custom implementation for any of these services, and then Keycloak will use it because it is one of its own services.

2.1. Custom provider deployment and discovery

In its simplest form, a custom provider is just a standard jar file containing one or more service implementations. At startup, Keycloak will scan its classpath and use standard java.util.ServiceLoadermechanisms to select all available providers. This means that all we have to do is META-INF/servicesto create a file named after a specific service interface in the jar folder and put the fully qualified name of the implementation in it.

But what kind of services can we add to Keycloak? If we go to the server infopage on Keycloak's management console , we will see a lot of content:

In this figure, the left column corresponds to a given service provider interface (SPI for short), and the right column shows the available providers for that particular SPI.

2.2. Available SPI

Keycloak's main document lists the following SPIs:

• org.keycloak.authentication.AuthenticatorFactory : Define the operations and interaction flows required to authenticate users or client applications

• org.keycloak.authentication.actiontoken.ActionTokenHandlerFactory: Allows us to create **/auth/realms/master/login-actions/action-token**custom actions that Keycloak will perform when it reaches the endpoint. For example, this mechanism is behind the standard password reset process. The link included in the email includes such an operation token

• org.keycloak.events.EventListenerProviderFactory: Create a provider that listens to Keycloak events. EventTypeThe Javadoc page contains a list of custom available events that the provider can handle. The typical use of this SPI is to create an audit database

• org.keycloak.adapters.saml.RoleMappingsProvider: Map the SAML role received from the external identity provider to the Keycloak role. This mapping is very flexible, allowing us to rename, delete and/or add roles in the context of a given domain

• org.keycloak.storage.UserStorageProviderFactory : Allow Keycloak to access custom user storage

• org.keycloak.vault.VaultProviderFactory: Allows us to use a custom file library to store domain-specific secrets. This information can include encryption keys, database credentials and other information.

Now, this list by no means covers all the available SPIs: they are the most well-documented and, in fact, most likely to require customization.

3.Custom provider implementation

As we mentioned in the introduction to this article, our provider example will allow us to use Keycloak with a read-only custom user repository. For example, in our example, this user repository is just a regular SQL table with some attributes:

create table if not exists users(
 username varchar(64) not null primary key,
 password varchar(64) not null,
 email varchar(128),
 firstName varchar(128) not null,
 lastName varchar(128) not null,
 birthDate DATE not null
 );

In order to support this custom user store, we must implement UserStorageProviderFactorySPI and deploy it to an existing Keycloak instance.

The focus here is the read-only part. In this way, we mean that users will be able to log in to Keycloak with their credentials, but cannot change any information in the custom storage, including passwords. However, this is not a limitation of Keycloak, because it actually supports two-way updates. The built-in LDAP provider is a good example of a provider that supports this feature.

3.1. Project settings

Our custom provider project is just a regular Maven project that creates a jar file. In order to avoid our provider spending a lot of time compiling, deploying, and restarting in a regular Keycloak instance, we will use a nice trick: Embed Keycloak as a test time dependency in our project.

We have already introduced how to embed Keycloack into SpringBoot applications, so we will not go into details here . By adopting this technology, we will get faster startup time and hot reinstallation function, thus providing developers with a smoother experience. Here, we will reuse the sample SpringBoot application to run the test directly from the custom provider, so we add it as a test dependency:

<dependency>
 <groupId>org.keycloak</groupId>
 <artifactId>keycloak-core</artifactId>
 <version>12.0.2</version>
 </dependency>
 <dependency>
 <groupId>org.keycloak</groupId>
 <artifactId>keycloak-server-spi</artifactId>
 <version>12.0.2</version>
 </dependency>
 <dependency>
 <groupId>com.baeldung</groupId>
 <artifactId>oauth-authorization-server</artifactId>
 <version>0.1.0-SNAPSHOT</version>
 <scope>test</scope>
 </dependency>

We use the latest 11 series version for keycloak-coreand keycloak-server-spiKeycloak dependencies.

However, the dependencies must be built locally from Baeldung's Spring Security OAuth repositoryoauth-authorization-server .

3.2. UserStorageProviderFactoryaccomplish

Let's UserStorageProviderFactorystart our provider by creating an implementation and make it discoverable by Keycloak.

The interface contains eleven methods, but we only need to implement two of them:

• getId() : Returns the unique identifier of this provider, which Keycloak will display on its management page.

• create() : Return the actual Provider implementation.

Keycloak calls create()methods for each transaction and passes KeycloakSessionand ComponentModelas parameters . Here, a transaction refers to any operation that requires access to the user's storage. The most typical example is the login process: at some point, Keycloak will call each configured user store for a given Realm to verify credentials. Therefore, we should avoid performing any expensive initialization operations at this time, because the create()method is always called.

In other words, the implementation is very simple:

public class CustomUserStorageProviderFactory

 implements UserStorageProviderFactory<CustomUserStorageProvider> {
 @Override
 public String getId() {
 return "custom-user-provider";
 }
 @Override
 public CustomUserStorageProvider create(KeycloakSession ksession, ComponentModel model) {
 return new CustomUserStorageProvider(ksession,model);
 }
 }

We chose for the provider ID “custom-user-provider”, and our create()implementation just returned a UserStorageProvidernew instance of the implementation. Now, we must not forget to create a service definition file and add it to our project. This file should be named org.keycloak.storage.UserStorageProviderFactoryand placed in our final jar META-INF/servicesfolder.

Since we are using a standard Maven project, this means we add it to the src/main/resources/META-INF/servicesfolder:

The content of this file is only the fully qualified name of the SPI implementation:

# SPI class implementation
 com.baeldung.auth.provider.user.CustomUserStorageProviderFactory

3.3. UserStorageProvideraccomplish

At first glance, the UserStorageProviderrealization does not look like we expected. It only contains some callback methods, none of which are related to actual users. The reason is that Keycloak hopes that our providers can also implement other hybrid interfaces that support specific user management.

The complete list of available interfaces is provided in Keycloak's documentation, and they are referred Provider Capabilities.toUserLookupProvider here as for simple read-only providers, the only interface we need to implement is . It only provides a lookup function, which means that Keycloak will automatically import users into its internal database when needed. However, the original user's password will not be used for authentication. For this, we also need to achieve CredentialInputValidator.

Finally, a common requirement is to be able to display existing users in a custom store in Keycloak's management interface. This requires us to achieve another interface: UserQueryProvider. This adds some query methods and acts as the DAO of our store.

So, given these requirements, this is how our implementation looks:

public class CustomUserStorageProvider implements UserStorageProvider, UserLookupProvider, CredentialInputValidator, UserQueryProvider {
 //...private members omitted
 public CustomUserStorageProvider(KeycloakSession ksession, ComponentModel model) {
 this.ksession = ksession;
 this.model = model;
 }
 //...implementation methods for each supported capability
 }

Note that we are saving the value passed to the constructor. Later we will see how they play an important role in our implementation.

3.4. UserLookupProvideraccomplish

Keycloak uses the methods in this interface to restore the username or email of a given UserModelinstance id. In this case, id is the unique identifier of the user in the format:'f:' unique_id':'external_id

• 'f:' is just a fixed prefix, indicating that this is an affiliate user

• unique_idIs the ID of the user's Keycloak

• external_idIs the user identifier used by the given user's store. In our case, this is usernamethe value of the name column

Let's continue getUserByUsername()to implement the method of this interface from the beginning:

@Override
 public UserModel getUserByUsername(String username, RealmModel realm) {
 try ( Connection c = DbUtil.getConnection(this.model)) {
 PreparedStatement st = c.prepareStatement(
 "select " +
 " username, firstName, lastName, email, birthDate " +
 "from users " +
 "where username = ?");
 st.setString(1, username);
 st.execute();
 ResultSet rs = st.getResultSet();
 if ( rs.next()) {
 return mapUser(realm,rs);
 }
 else {
 return null;
 }
 }
 catch(SQLException ex) {
 throw new RuntimeException("Database error:" + ex.getMessage(),ex);
 }
 }

Unsurprisingly, this is a simple database query, using the provided usernameinformation to find it. There are two interesting points that need to be explained: DbUtil.getConnection()and mapUser().

DbUtilIt is a helper class that ComponentModelreturns JDBC in some way from the information contained in the constructor Connection. We will introduce its details later.

As for mapUser(), its job is to map database records containing user data to UserModelinstances. UserModelRepresents the user entity (as seen by Keycloak) and has methods to read its attributes. The implementation of this interface we provide here extends the class provided by Keycloak AbstractUserAdapter. We also added an Builderinternal class to the implementation , so mapUser()it is easy to create an UserModelinstance:

private UserModel mapUser(RealmModel realm, ResultSet rs) throws SQLException {
 CustomUser user = new CustomUser.Builder(ksession, realm, model, rs.getString("username"))
.email(rs.getString("email"))
.firstName(rs.getString("firstName"))
.lastName(rs.getString("lastName"))
.birthDate(rs.getDate("birthDate"))
.build();
 return user;
 }

Similarly, other methods basically follow the same pattern as described above, so we will not go into details about them. Please refer to the provider's code and check all getUserByXXXand searchForUsermethods.

3.5. EstablishConnection

Now, let's look at the DbUtil.getConnection()method:

public class DbUtil {
 public static Connection getConnection(ComponentModel config) throws SQLException{
 String driverClass = config.get(CONFIG_KEY_JDBC_DRIVER);
 try {
 Class.forName(driverClass);
 }
 catch(ClassNotFoundException nfe) {
 //...error handling omitted
 }
 return DriverManager.getConnection(
 config.get(CONFIG_KEY_JDBC_URL),
 config.get(CONFIG_KEY_DB_USERNAME),
 config.get(CONFIG_KEY_DB_PASSWORD));
 }
 }

We can see that ComponentModelis where all the necessary parameters are created. But how does Keycloak know which parameters our custom provider needs? To answer this question, we need to go back toCustomUserStorageProviderFactory.

3.6. Configuration metadata

For the basic contract CustomUserStorageProviderFactory, UserStorageProviderFactoryit contains metadata that allows Keycloak to query configuration properties, and also important methods to verify the assignment . In our example, we will define some configuration parameters required to establish a JDBC connection. Since this metadata is static, we will create it in the constructor and getConfigProperties()will only return it.

public class CustomUserStorageProviderFactory

 implements UserStorageProviderFactory<CustomUserStorageProvider> {
 protected final List<ProviderConfigProperty> configMetadata;
 public CustomUserStorageProviderFactory() {
 configMetadata = ProviderConfigurationBuilder.create()
.property()
.name(CONFIG_KEY_JDBC_DRIVER)
.label("JDBC Driver Class")
.type(ProviderConfigProperty.STRING_TYPE)
.defaultValue("org.h2.Driver")
.helpText("Fully qualified class name of the JDBC driver")
.add()
 //...repeat this for every property (omitted)
.build();
 }
 //...other methods omitted
 @Override
 public List<ProviderConfigProperty> getConfigProperties() {
 return configMetadata;
 }
 @Override
 public void validateConfiguration(KeycloakSession session, RealmModel realm, ComponentModel config)

 throws ComponentValidationException {
 try (Connection c = DbUtil.getConnection(config)) {
 c.createStatement().execute(config.get(CONFIG_KEY_VALIDATION_QUERY));
 }
 catch(Exception ex) {
 throw new ComponentValidationException("Unable to validate database connection",ex);
 }
 }
 }

Now validateConfiguration(), we will get everything we need to verify the passed parameters when adding the provided content to the Realm . In our example, we use this information to establish a database connection and perform verification queries. If something ComponentValidationExceptiongoes wrong, we just throw it out and signal Keycloak that the parameter is invalid.

And, although not shown here, we can also use onCreated()methods to attach logic that will be executed every time the administrator adds our provider to the Realm . This allows us to perform an initialization time logic to prepare our storage for use, which may be necessary in some cases. For example, we can use this method to modify the database and add a column to record whether a given user has used Keycloak.

3.7. CredentialInputValidatoraccomplish

This interface contains methods to verify user credentials . Since Keycloak supports different types of credentials (passwords, OTP tokens, X.509 certificates, etc.), our provider must tell it if it is supportsCredentialType()of supportsCredentialType()a given type andin isConfiguredFor()the context of a given realm and configure it. isConfiguredFor().

In our case, we only support passwords, and since no additional configuration is required, we can delegate the latter method to the former:

@Override
 public boolean supportsCredentialType(String credentialType) {
 return PasswordCredentialModel.TYPE.endsWith(credentialType);
 }
 @Override
 public boolean isConfiguredFor(RealmModel realm, UserModel user, String credentialType) {
 return supportsCredentialType(credentialType);
 }

The actual password verification happens in the isValid()method:

@Override
 public boolean isValid(RealmModel realm, UserModel user, CredentialInput credentialInput) {
 if(!this.supportsCredentialType(credentialInput.getType())) {
 return false;
 }
 StorageId sid = new StorageId(user.getId());
 String username = sid.getExternalId();
 try (Connection c = DbUtil.getConnection(this.model)) {
 PreparedStatement st = c.prepareStatement("select password from users where username = ?");
 st.setString(1, username);
 st.execute();
 ResultSet rs = st.getResultSet();
 if ( rs.next()) {
 String pwd = rs.getString(1);
 return pwd.equals(credentialInput.getChallengeResponse());
 }
 else {
 return false;
 }
 }
 catch(SQLException ex) {
 throw new RuntimeException("Database error:" + ex.getMessage(),ex);
 }
 }

Here, there are a few points to discuss. First of all, please note how we use the StorageIdimage initialized from Keycloak to UserModel,extract the external ID. We can use the fact that this id has a well-known format and extract the username from there, but it is better to use it safely here and encapsulate this knowledge in a class provided by Keycloak.

Next, is the actual password verification. For our simple and taken for granted very insecure database, the password check is trivial: just compare the database value with the value provided by the user (you can compare it by getChallengeResponse()doing it getChallengeResponse().Of course, real-world providers will require more steps, For example, generate the password and salt of the hash notification from the database, and compare the hashes.

Finally, the user store usually has some life cycles associated with the password: the longest period of use, blocked and/or inactive, etc. In any case, when implementing a provider, the isValid()method is where this logic is added.

3.8. UserQueryProvideraccomplish

UserQueryProviderThe functional interface tells Keycloak that our provider can search for users in its store. This is convenient because by supporting this feature, we will be able to view users in the management console.

The method comprises the interface getUsersCount(),for obtaining total number of users store getXXX()and several getXXX()and searchXXX()methods. The query interface not only supports searching for users, but also for searching groups, which we will not introduce this time.

Since the implementation of these methods is very similar, let's just look at one of them searchForUser():

@Override
 public List<UserModel> searchForUser(String search, RealmModel realm, int firstResult, int maxResults) {
 try (Connection c = DbUtil.getConnection(this.model)) {
 PreparedStatement st = c.prepareStatement(
 "select " +
 " username, firstName, lastName, email, birthDate " +
 "from users " +
 "where username like ? +

 "order by username limit ? offset ?");

 st.setString(1, search);

 st.setInt(2, maxResults);

 st.setInt(3, firstResult);

 st.execute();

 ResultSet rs = st.getResultSet();

 List<UserModel> users = new ArrayList<>();

 while(rs.next()) {

 users.add(mapUser(realm,rs));

 }

 return users;

 }

 catch(SQLException ex) {

 throw new RuntimeException("Database error:" + ex.getMessage(),ex);

 }

 }

As we can see, there is nothing special here: just regular JDBC code. Implementation note worth mentioning: UserQueryProviderMethods usually have paged and non-paged versions. Since the user store may have a large number of records, the non-paged version should simply be delegated to the paged version using sensible defaults. Even better, we can add a configuration parameter to define what is a "reasonable default".

4.Test

Now that we have implemented the provider, it is time to test it locally using the embedded Keycloak instance. The code of this project contains a real-time test class, we have used this class to guide Keycloak and custom user database, and then print the access URL on the console before sleeping for an hour.

Using this setting, we can verify that our custom provider is working as expected by opening the printed URL in the browser:

To access the management console, we will use administrator credentials, which can be obtained by viewing the application-test.ymlfile. After logging in, let us navigate to the "Server Information" page:

On the "Providers" tab, we can see that our custom provider is displayed along with other built-in storage providers:

We can also check whether the Baeldung realm is already using this provider. To do this, we can select it in the drop-down menu at the top left, and then navigate to the " User FederationPage:

Next, let's test the actual login to the field. We will use the account management page of the realm, where users can manage their data. Our real-time test will print this URL before going to sleep, so we just need to copy it from the console and paste it into the address bar of the browser.

The test data contains three users: user1, user2 and user3. Their passwords are the same: "changeit". After successfully logging in, we will see the "Account Management" page showing the imported user data:

However, if we try to modify any data, we will receive an error message. This is expected because our provider is read-only, so Keycloak does not allow it to be modified. Now, since supporting two-way synchronization is beyond the scope of this article, we will leave it unchanged.

5 Conclusion

In this article, we showed how to use the user storage provider as a concrete example to create a custom provider for Keycloak. The complete source code of the example can be found on GitHub .