NiFi: Kerberos ticket login not supported by this NiFi

I have problems with Kerberising my NiFi.

My setup is such that I have Docker and in it two containers: apache/nifi and gcavalcante8808/krb5-server. NiFi is already secured with HTTPS and Initial admin identity so I can log in with certificate to become admin without problem. So far so good.

Then if I pull up NiFi UI from browser without admin certificate in nifi-user.log appears message Kerberos ticket login not supported by this NiFi (stacktrace was shortened):

2021-02-18 10:25:39,804 INFO [main] o.a.n.a.FileUserGroupProvider Users/Groups file loaded at Thu Feb 18 10:25:39 UTC 2021
2021-02-18 10:25:39,836 INFO [main] o.a.n.a.FileAccessPolicyProvider Authorizations file loaded at Thu Feb 18 10:25:39 UTC 2021
2021-02-18 10:25:41,224 WARN [main] o.a.n.w.s.o.StandardOidcIdentityProvider The OIDC provider is not configured or enabled
2021-02-18 10:35:19,976 WARN [NiFi Web Server-30] o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: Kerberos ticket login not supported by this NiFi.. Returning Conflict response.
java.lang.IllegalStateException: Kerberos ticket login not supported by this NiFi.
    at org.apache.nifi.web.api.AccessResource.createAccessTokenFromTicket(AccessResource.java:644)
...
2021-02-18 10:35:20,041 WARN [NiFi Web Server-28] o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: OpenId Connect is not configured.. Returning Conflict response.
java.lang.IllegalStateException: OpenId Connect is not configured.
    at org.apache.nifi.web.api.AccessResource.oidcExchange(AccessResource.java:301)
...
2021-02-18 10:35:20,092 INFO [NiFi Web Server-29] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (<anonymous>) GET https://<redacted>:4321/nifi-api/flow/current-user (source ip: 172.17.0.1)
2021-02-18 10:35:20,096 WARN [NiFi Web Server-29] o.a.n.w.s.NiFiAuthenticationFilter Rejecting access to web api: Anonymous authentication has not been configured.

If I will go on and try to log in, I get Receive timed out (in nifi-bootstrap.log):

2021-02-18 10:39:00,695 INFO [NiFi logging handler] org.apache.nifi.StdOut Debug is  true storeKey true useTicketCache false useKeyTab false doNotPrompt false ticketCache is null isInitiator true KeyTab is null refreshKrb5Config is false principal is null tryFirstPass is false useFirstPass is false storePass is false clearPass is false
2021-02-18 10:39:00,697 INFO [NiFi logging handler] org.apache.nifi.StdOut      [Krb5LoginModule] user entered username: test_RW@NIFI.DEV
2021-02-18 10:39:00,697 INFO [NiFi logging handler] org.apache.nifi.StdOut 
2021-02-18 10:39:00,704 INFO [NiFi logging handler] org.apache.nifi.StdOut Java config name: /etc/krb5.conf
2021-02-18 10:39:00,755 INFO [NiFi logging handler] org.apache.nifi.StdOut Loaded from Java config
2021-02-18 10:39:00,755 INFO [NiFi logging handler] org.apache.nifi.StdOut >>> KdcAccessibility: reset
2021-02-18 10:39:00,760 INFO [NiFi logging handler] org.apache.nifi.StdOut Using builtin default etypes for default_tkt_enctypes
2021-02-18 10:39:00,760 INFO [NiFi logging handler] org.apache.nifi.StdOut default etypes for default_tkt_enctypes: 18 17 16 23.
2021-02-18 10:39:00,760 INFO [NiFi logging handler] org.apache.nifi.StdOut >>> KrbAsReq creating message
2021-02-18 10:39:00,771 INFO [NiFi logging handler] org.apache.nifi.StdOut >>> KrbKdcReq send: kdc=<redacted> UDP:8088, timeout=30000, number of retries =3, #bytes=174
2021-02-18 10:39:00,771 INFO [NiFi logging handler] org.apache.nifi.StdOut >>> KDCCommunication: kdc=<redacted> UDP:8088, timeout=30000,Attempt =1, #bytes=174
2021-02-18 10:39:30,889 INFO [NiFi logging handler] org.apache.nifi.StdOut SocketTimeOutException with attempt: 1
2021-02-18 10:39:30,890 INFO [NiFi logging handler] org.apache.nifi.StdOut >>> KDCCommunication: kdc=<redacted> UDP:8088, timeout=30000,Attempt =2, #bytes=174
2021-02-18 10:40:00,927 INFO [NiFi logging handler] org.apache.nifi.StdOut SocketTimeOutException with attempt: 2
2021-02-18 10:40:00,927 INFO [NiFi logging handler] org.apache.nifi.StdOut >>> KDCCommunication: kdc=<redacted> UDP:8088, timeout=30000,Attempt =3, #bytes=174
2021-02-18 10:40:30,966 INFO [NiFi logging handler] org.apache.nifi.StdOut SocketTimeOutException with attempt: 3
2021-02-18 10:40:30,966 INFO [NiFi logging handler] org.apache.nifi.StdOut >>> KrbKdcReq send: error trying <redacted>:8088
2021-02-18 10:40:30,967 INFO [NiFi logging handler] org.apache.nifi.StdOut java.net.SocketTimeoutException: Receive timed out
2021-02-18 10:40:30,969 INFO [NiFi logging handler] org.apache.nifi.StdOut  at java.net.PlainDatagramSocketImpl.receive0(Native Method)
...
2021-02-18 10:40:30,984 INFO [NiFi logging handler] org.apache.nifi.StdOut >>> KdcAccessibility: add <redacted>:8088
2021-02-18 10:40:30,984 INFO [NiFi logging handler] org.apache.nifi.StdOut      [Krb5LoginModule] authentication failed 
2021-02-18 10:40:30,984 INFO [NiFi logging handler] org.apache.nifi.StdOut Receive timed out

And it all ends with blank pop-up in NiFi.

I am all out of ideas, please has someone any idea what to check/set/edit/...? To me it seems like NiFi is somehow misconfigured, yet all pieces as far as I know are correct. What puzzles me even more is that despite that Kerberos ticketing is not supported, it tryies it and somehow fails.

My setup in nifi.properties:

# kerberos #
nifi.kerberos.krb5.file=/etc/krb5.conf
nifi.kerberos.service.principal=app_nifi_svc/admin@NIFI.DEV
nifi.kerberos.keytab.location=/opt/app_nifi_svc.keytab

In login-identity-provider.xml:

<provider>
    <identifier>kerberos-provider</identifier>
    <class>org.apache.nifi.kerberos.KerberosProvider</class>
    <property name="Default Realm">NIFI.DEV</property>
    <property name="Kerberos Config File">/etc/krb5.conf</property>
    <property name="Authentication Expiration">12 hours</property>
</provider>

krb5.conf:

[libdefaults]
 dns_lookup_realm = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 rdns = false
 default_realm = NIFI.DEV

 [realms]
 NIFI.DEV = {
    kdc = <redacted>:8088
    admin_server = <redacted>:8088
 }

Checked ownership and mode in NiFi container:

~> dc exec -it nifi_https ls -l /opt/app_nifi_svc.keytab
-rw-------. 1 nifi nifi 158 Feb 17 15:46 /opt/app_nifi_svc.keytab
~> dc exec -it nifi_https ls -l /etc/krb5.conf
-rwxrwxrwx. 1 nifi nifi 229 Feb 17 15:49 /etc/krb5.conf

Checked that keytab works in Kerberos container:

/ # kinit -kt /opt/app_nifi_svc.keytab app_nifi_svc/admin@NIFI.DEV
/ # klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: app_nifi_svc/admin@NIFI.DEV

Valid starting     Expires            Service principal
02/18/21 09:59:44  02/18/21 21:59:44  krbtgt/NIFI.DEV@NIFI.DEV
        renew until 02/25/21 09:59:42

Firewall is opened and telnet from NiFi container to Kerberos container connects ("hi" is my query):

root@<nifi_container>:/opt/nifi/nifi-current# telnet <redacted> 8088
Trying <ipadress of redacted>...
Connected to <redacted>.
Escape character is '^]'.
hi
X~V0T▒▒▒20210218105711Z▒        EӦ=▒                                                                      IFI.DEV▒0▒0rbtgIFI.DEVConnection closed by foreign host.

EDIT: Found a probable misconfiguration in krb5.conf: admin_server = <redacted>:8088. But change to admin_server = <redacted>:8749 didnt help either.

Solution

The true reason was somehow hidden. The problem is, that Kerberos in default configuration tries to communicate using UDP protocol. Docker on the other hand by default exposes ports for TCP protocol. Thus solution was simple - start Kerberos container with ports exposed for both TCP and UDP protocols:

docker run 
-p 8088:8088/tcp \
-p 8749:8749/tcp \
-p 8088:8088/udp \
-p 8749:8749/udp \
gcavalcante8808/krb5-server