HDP-2.4.2.0-258 installed using Ambari 2.2.2.0
I installed Zeppelin(0.6.0.2.4.2.0-258)manually and was able to execute several paragraphs in a notebook.
Now I wish to secure it step-by-step, starting with the authentication for the web UI, integrated with LDAP i.e when a user enters his credentials after hitting http://:9995/, he can proceed only if he is present in at least one of the several Unix LDAP groups as follows : devadmin datascientist develope
Before proceeding, I read several existing posts, Zeppelin doc. etc.
The conf/shiro.ini file :
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
#[users]
# List of users with their password allowed to access Zeppelin.
# To use a different strategy (LDAP / Database / ...) check the shiro doc at http://shiro.apache.org/configuration.html#Configuration-INISections
#admin = password1
#user1 = password2, role1, role2
#user2 = password3, role3
#user3 = password4, role2
# Sample LDAP configuration, for user Authentication, currently tested for single Realm
[main]
#ldapRealm = org.apache.zeppelin.server.LdapGroupRealm
ldapRealm = org.apache.shiro.realm.ldap.JndiLdapRealm
ldapRealm.contextFactory.environment[ldap.searchBase]=dc=company,dc=SE
ldapRealm.userDnTemplate = uid={0},CN=devadmin,ou=Group,dc=company,dc=SE
ldapRealm.contextFactory.url = ldap://unix-ldap.company.com:389
ldapRealm.contextFactory.authenticationMechanism = SIMPLE
shiro.loginUrl = /api/login
[urls]
# anon means the access is anonymous.
# authcBasic means Basic Auth Security
# To enfore security, comment the line below and uncomment the next one
/api/version = anon
#/** = anon
/** = authcBasic
For the ldapRealm, if I provide org.apache.zeppelin.server.LdapGroupRealm, I get the following error and Zeppelin fails to start :
ERROR [2016-09-05 14:26:14,996] ({main} ZeppelinServer.java[main]:117) - Error while running jettyServer
org.apache.shiro.config.ConfigurationException: Unable to instantiate class [org.apache.zeppelin.server.LdapGroupRealm] for object named 'ldapRealm'. Please ensure you've specified the fully qualified class name correctly.
at org.apache.shiro.config.ReflectionBuilder.createNewInstance(ReflectionBuilder.java:151)
at org.apache.shiro.config.ReflectionBuilder.buildObjects(ReflectionBuilder.java:119)
at org.apache.shiro.config.IniSecurityManagerFactory.buildInstances(IniSecurityManagerFactory.java:161)
at org.apache.shiro.config.IniSecurityManagerFactory.createSecurityManager(IniSecurityManagerFactory.java:124)
at org.apache.shiro.config.IniSecurityManagerFactory.createSecurityManager(IniSecurityManagerFactory.java:102)
at org.apache.shiro.config.IniSecurityManagerFactory.createInstance(IniSecurityManagerFactory.java:88)
at org.apache.shiro.config.IniSecurityManagerFactory.createInstance(IniSecurityManagerFactory.java:46)
at org.apache.shiro.config.IniFactorySupport.createInstance(IniFactorySupport.java:123)
at org.apache.shiro.util.AbstractFactory.getInstance(AbstractFactory.java:47)
at org.apache.shiro.web.env.IniWebEnvironment.createWebSecurityManager(IniWebEnvironment.java:203)
at org.apache.shiro.web.env.IniWebEnvironment.configure(IniWebEnvironment.java:99)
at org.apache.shiro.web.env.IniWebEnvironment.init(IniWebEnvironment.java:92)
at org.apache.shiro.util.LifecycleUtils.init(LifecycleUtils.java:45)
at org.apache.shiro.util.LifecycleUtils.init(LifecycleUtils.java:40)
at org.apache.shiro.web.env.EnvironmentLoader.createEnvironment(EnvironmentLoader.java:221)
at org.apache.shiro.web.env.EnvironmentLoader.initEnvironment(EnvironmentLoader.java:133)
at org.apache.shiro.web.env.EnvironmentLoaderListener.contextInitialized(EnvironmentLoaderListener.java:58)
at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:782)
at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:424)
at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:774)
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:249)
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:717)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
at org.eclipse.jetty.server.handler.HandlerCollection.doStart(HandlerCollection.java:229)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:172)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
at org.eclipse.jetty.server.handler.HandlerWrapper.doStart(HandlerWrapper.java:95)
at org.eclipse.jetty.server.Server.doStart(Server.java:282)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
at org.apache.zeppelin.server.ZeppelinServer.main(ZeppelinServer.java:115)
Caused by: org.apache.shiro.util.UnknownClassException: Unable to load class named [org.apache.zeppelin.server.LdapGroupRealm] from the thread context, current, or system/application ClassLoaders. All heuristics have been exhausted. Class could not be found.
at org.apache.shiro.util.ClassUtils.forName(ClassUtils.java:148)
at org.apache.shiro.util.ClassUtils.newInstance(ClassUtils.java:164)
at org.apache.shiro.config.ReflectionBuilder.createNewInstance(ReflectionBuilder.java:144)
... 29 more
If I use org.apache.shiro.realm.ldap.JndiLdapRealm,
The error :
ERROR [2016-09-05 14:29:36,153] ({qtp762227630-30} NotebookServer.java[onMessage]:207) - Can't handle message
java.lang.Exception: Invalid ticket != 16731c36-4f7e-4dd6-b567-8da934aeecd0
at org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:113)
at org.apache.zeppelin.socket.NotebookSocket.onMessage(NotebookSocket.java:56)
at org.eclipse.jetty.websocket.WebSocketConnectionRFC6455$WSFrameHandler.onFrame(WebSocketConnectionRFC6455.java:835)
at org.eclipse.jetty.websocket.WebSocketParserRFC6455.parseNext(WebSocketParserRFC6455.java:349)
at org.eclipse.jetty.websocket.WebSocketConnectionRFC6455.handle(WebSocketConnectionRFC6455.java:225)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:745)
ERROR [2016-09-05 14:29:36,159] ({qtp762227630-34} NotebookServer.java[onMessage]:207) - Can't handle message
java.lang.Exception: Invalid ticket != 16731c36-4f7e-4dd6-b567-8da934aeecd0
at org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:113)
at org.apache.zeppelin.socket.NotebookSocket.onMessage(NotebookSocket.java:56)
at org.eclipse.jetty.websocket.WebSocketConnectionRFC6455$WSFrameHandler.onFrame(WebSocketConnectionRFC6455.java:835)
at org.eclipse.jetty.websocket.WebSocketParserRFC6455.parseNext(WebSocketParserRFC6455.java:349)
at org.eclipse.jetty.websocket.WebSocketConnectionRFC6455.handle(WebSocketConnectionRFC6455.java:225)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:745)
ERROR [2016-09-05 14:29:46,150] ({qtp762227630-30} NotebookServer.java[onMessage]:207) - Can't handle message
java.lang.Exception: Invalid ticket != 16731c36-4f7e-4dd6-b567-8da934aeecd0
at org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:113)
at org.apache.zeppelin.socket.NotebookSocket.onMessage(NotebookSocket.java:56)
at org.eclipse.jetty.websocket.WebSocketConnectionRFC6455$WSFrameHandler.onFrame(WebSocketConnectionRFC6455.java:835)
at org.eclipse.jetty.websocket.WebSocketParserRFC6455.parseNext(WebSocketParserRFC6455.java:349)
at org.eclipse.jetty.websocket.WebSocketConnectionRFC6455.handle(WebSocketConnectionRFC6455.java:225)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:745)
ERROR [2016-09-05 14:29:56,150] ({qtp762227630-31} NotebookServer.java[onMessage]:207) - Can't handle message
java.lang.Exception: Invalid ticket != 16731c36-4f7e-4dd6-b567-8da934aeecd0
at org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:113)
at org.apache.zeppelin.socket.NotebookSocket.onMessage(NotebookSocket.java:56)
at org.eclipse.jetty.websocket.WebSocketConnectionRFC6455$WSFrameHandler.onFrame(WebSocketConnectionRFC6455.java:835)
at org.eclipse.jetty.websocket.WebSocketParserRFC6455.parseNext(WebSocketParserRFC6455.java:349)
at org.eclipse.jetty.websocket.WebSocketConnectionRFC6455.handle(WebSocketConnectionRFC6455.java:225)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:745)
ERROR [2016-09-05 14:30:06,151] ({qtp762227630-29} NotebookServer.java[onMessage]:207) - Can't handle message
java.lang.Exception: Invalid ticket != 16731c36-4f7e-4dd6-b567-8da934aeecd0
at org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:113)
at org.apache.zeppelin.socket.NotebookSocket.onMessage(NotebookSocket.java:56)
at org.eclipse.jetty.websocket.WebSocketConnectionRFC6455$WSFrameHandler.onFrame(WebSocketConnectionRFC6455.java:835)
at org.eclipse.jetty.websocket.WebSocketParserRFC6455.parseNext(WebSocketParserRFC6455.java:349)
at org.eclipse.jetty.websocket.WebSocketConnectionRFC6455.handle(WebSocketConnectionRFC6455.java:225)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:745)
ERROR [2016-09-05 14:30:16,151] ({qtp762227630-32} NotebookServer.java[onMessage]:207) - Can't handle message
java.lang.Exception: Invalid ticket != 16731c36-4f7e-4dd6-b567-8da934aeecd0
at org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:113)
at org.apache.zeppelin.socket.NotebookSocket.onMessage(NotebookSocket.java:56)
at org.eclipse.jetty.websocket.WebSocketConnectionRFC6455$WSFrameHandler.onFrame(WebSocketConnectionRFC6455.java:835)
at org.eclipse.jetty.websocket.WebSocketParserRFC6455.parseNext(WebSocketParserRFC6455.java:349)
at org.eclipse.jetty.websocket.WebSocketConnectionRFC6455.handle(WebSocketConnectionRFC6455.java:225)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:745)
I installed the latest stable version of Zeppelin(0.6.1) independently(NOT integrated with Ambari) and the error was gone, I am still getting LDAP authentication errors but those are because of wrong DN.
Details in this thread.