KNOX-3330: Refactor LDAP Proxy configuration to support multiple backends

[handavid]

KNOX-3330 - Refactor Knox LDAP Proxy configuration and implementation to allow multiple backends to be simultaneously configured

What changes were proposed in this pull request?

Gateway server configurations are updated to use 'gateway.ldap.interceptor.' instead of 'gateway.ldap.backend.' to allow specifying multiple types of interceptors as well as multiple backends to the LDAP proxy.

BackendFactory has been modified to use the java ServiceLoader to load a factory for a backend class instead of a backend instance directly. This allows multiple backends of the same class to be configured. InterceptorFactory has been implemented following the same pattern.

GroupLookupInterceptor is renamed to UserSearchInterceptor to more accurately describe what it does. Multiple UserSearchInterceptors can be configured with each forwarding the search to its backend and appending the results.

A DuplicateUserFilteringInterceptor has been implemented that will filter out search Entries with the same UID that are returned from different backends.

How was this patch tested?

Unit tests were updated.

• KnoxLDAPServerManagerTest.java modified to configure interceptors instead of backends
• KnoxLDAPServerManagerTest.java modified to configure multiple backends simultaneously

Changes were manually tested against the test ldap server and an AD that I have access to.
The following configuration was added to the gateway-site.xml

    <!-- LDAP Proxy Service Configuration -->
    <property>
        <name>gateway.ldap.enabled</name>
        <value>true</value>
        <description>Enable the embedded LDAP service for user and group lookups. Set to true to enable.</description>
    </property>
    <property>
        <name>gateway.ldap.port</name>
        <value>3890</value>
        <description>Port for the LDAP service to listen on. Default is 3890.</description>
    </property>
    <property>
        <name>gateway.ldap.base.dn</name>
        <value>dc=proxy,dc=com</value>
        <description>Base DN for LDAP entries in the proxy server. Default is dc=proxy,dc=com.</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.names</name>
        <value>localldap,testad,duplicatefilter</value>
        <description>Interceptor names for LDAP service.</description>
    </property>

    <!-- Local LDAP Server -->
    <property>
        <name>gateway.ldap.interceptor.localldap.interceptorType</name>
        <value>backend</value>
        <description>Type of interceptor. Currently supported: backend, duplicateuserfilter</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.localldap.backendType</name>
        <value>ldap</value>
        <description>Type of backend. Currently supported: file, ldap. Future: jdbc, knox.</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.localldap.url</name>
        <value>ldap://localhost:33389</value>
        <description>LDAP server URL for proxy backend</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.localldap.remoteBaseDn</name>
        <value>dc=hadoop,dc=apache,dc=org</value>
        <description>Base DN of the remote LDAP server</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.localldap.systemUsername</name>
        <value>uid=guest,ou=people,dc=hadoop,dc=apache,dc=org</value>
        <description>LDAP bind DN for proxy backend authentication</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.localldap.systemPassword</name>
        <value>guest-password</value>
        <description>LDAP bind password for proxy backend authentication</description>
    </property>
    
    <!-- Test AD -->
    <property>
        <name>gateway.ldap.interceptor.testad.interceptorType</name>
        <value>backend</value>
        <description>Type of interceptor. Currently supported: backend, duplicateuserfilter</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.testad.backendType</name>
        <value>ldap</value>
        <description>Type of backend. Currently supported: file, ldap. Future: jdbc, knox.</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.testad.url</name>
        <value>ldap://test-ad.example.com:389</value>
        <description>LDAP server URL for proxy backend</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.testad.remoteBaseDn</name>
        <value>dc=test-ad,dc=example,dc=com</value>
        <description>Base DN of the remote LDAP server</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.testad.systemUsername</name>
        <value>bind user to AD</value>
        <description>LDAP bind DN for proxy backend authentication</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.testad.systemPassword</name>
        <value>password to AD</value>
        <description>LDAP bind password for proxy backend authentication</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.testad.userIdentifierAttribute</name>
        <value>sAMAccountName</value>
        <description>Attribute used for identifying users</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.testad.userSearchBase</name>
        <value>cn=users,dc=test-ad,dc=example,dc=com</value>
        <description>Search base for users</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.testad.groupSearchBase</name>
        <value>ou=groups,dc=test-ad,dc=example,dc=com</value>
        <description>Search base for groups</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.testad.useMemberOf</name>
        <value>true</value>
        <description>Whether to use the memberOf attribute for efficiency when retrieving group memberships</description>
    </property>

    <!-- Duplicate Filter Interceptor -->
    <property>
        <name>gateway.ldap.interceptor.duplicatefilter.interceptorType</name>
        <value>duplicateuserfilter</value>
        <description>Type of interceptor. Currently supported: backend, duplicateuserfilter</description>
    </property>

    <!-- END LDAP Proxy Service Configuration -->

Integration Tests

No integration test changes. PR can be updated after #1236 is merged

UI changes

no UI changes

[handavid]

@smolnar82 @lmccay

[github-actions]

Test Results

21 tests   21 ✅  1s ⏱️
 1 suites   0 💤
 1 files     0 ❌

Results for commit 33fe1fe.