一些说明
由于项目要用到apache kylin,所以在kylin官网上下载了最新的二进制包: for HBase 0.98 (includes HDP 2.2, CDH 5.2/5.3) - apache-kylin-1.5.4.1-bin.tar.gz
之前的Hadoop集群搭建在三台物理机上,装的cdh版本号如下:
zookeeper-3.4.5-cdh5.3.1
hadoop-2.5.0-cdh5.3.1
hbase-0.98.6-cdh5.3.1
hive-0.13.1-cdh5.3.1
apache-kylin-1.5.4.1-bin

关于kylin的安装配置就免了(网上很多),只把安装和使用过程中遇到的问题贴出来,仅供参考:
1/ 启动kylin后,页面无法访问,kylin.log里报异常:
2016-11-18 16:03:29,576 INFO  [Thread-12] zookeeper.ZooKeeper:438 : Initiating client connection, connectString=10.45.6.34:2181:2181,10.45.6.31:2181:2181,10.45.40.174:2181:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@1921b556
2016-11-18 16:03:29,578 ERROR [Thread-12] imps.CuratorFrameworkImpl:543 : Background exception was not retry-able or retry gave up
java.net.UnknownHostException: 10.45.6.34:2181: Name or service not known
 at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
 at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901)
 at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293)
 at java.net.InetAddress.getAllByName0(InetAddress.java:1246)
 at java.net.InetAddress.getAllByName(InetAddress.java:1162)
 at java.net.InetAddress.getAllByName(InetAddress.java:1098)
 at org.apache.zookeeper.client.StaticHostProvider.<init>(StaticHostProvider.java:61)
 at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:445)
 at org.apache.curator.utils.DefaultZookeeperFactory.newZooKeeper(DefaultZookeeperFactory.java:29)
 at org.apache.curator.framework.imps.CuratorFrameworkImpl$2.newZooKeeper(CuratorFrameworkImpl.java:154)
 at org.apache.curator.HandleHolder$1.getZooKeeper(HandleHolder.java:94)
 at org.apache.curator.HandleHolder.getZooKeeper(HandleHolder.java:55)
 at org.apache.curator.ConnectionState.reset(ConnectionState.java:219)
 at org.apache.curator.ConnectionState.start(ConnectionState.java:103)
 at org.apache.curator.CuratorZookeeperClient.start(CuratorZookeeperClient.java:190)
 at org.apache.curator.framework.imps.CuratorFrameworkImpl.start(CuratorFrameworkImpl.java:256)
 at org.apache.kylin.storage.hbase.util.ZookeeperJobLock.lock(ZookeeperJobLock.java:67)
 at org.apache.kylin.job.impl.threadpool.DefaultScheduler.init(DefaultScheduler.java:202)
 at org.apache.kylin.rest.controller.JobController$1.run(JobController.java:89)
 at java.lang.Thread.run(Thread.java:745)
端口号2181在zk connectString里写了两遍,查看org.apache.kylin.storage.hbase.util.ZookeeperJobLock.lock的源码(ZookeeperJobLock.java:90)可知,
读取的是配置文件hbase-site.xml的hbase.zookeeper.quorum,该项只需配置Host不需要配置端口号Port,改回来之后就好了

2/ 运行官方自带的测试示例learn_kylin
hive> show tables;
OK
kylin_cal_dt
kylin_category_groupings
kylin_sales
生成三张hive表,前两张表都能在kylin页面中成功执行SQL查询,但是第三张表kylin_sales的查询报错:
2016-11-24 15:09:49,755 INFO  [http-bio-7070-exec-5] service.QueryService:261 :
==========================[QUERY]===============================
SQL: select * from kylin_sales
LIMIT 50000
User: ADMIN
Success: false
Duration: 0.0
Project: learn_kylin
Realization Names: [kylin_sales_cube]
Cuboid Ids: [99]
Total scan count: 0
Result row count: 0
Accept Partial: true
Is Partial Result: false
Hit Exception Cache: false
Storage cache used: false
Message: Error while executing SQL "select * from kylin_sales LIMIT 50000": Error in coprocessor
==========================[QUERY]===============================
2016-11-24 15:09:49,756 ERROR [http-bio-7070-exec-5] controller.BasicController:44 :
org.apache.kylin.rest.exception.InternalErrorException: Error while executing SQL "select * from kylin_sales LIMIT 50000": Error in coprocessor
 at org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:225)
 at org.apache.kylin.rest.controller.QueryController.query(QueryController.java:94)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221)
 at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136)
 at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)
 at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:743)
 at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:672)
 at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:82)
 at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:933)
 at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:867)
 at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:951)
 at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:853)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:650)
 at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:827)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
 at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
 at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
 at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
 at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
 at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
 at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
 at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
 at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
 at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:150)
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
 at org.springframework.security.web.authentication.ui.DefaultLoginPageGeneratingFilter.doFilter(DefaultLoginPageGeneratingFilter.java:91)
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
 at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:183)
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
 at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105)
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
 at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
 at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
 at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
 at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343)
 at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
 at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:209)
 at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:244)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
 at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
 at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
 at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
 at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
 at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
 at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
 at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
 at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436)
 at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078)
 at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
 at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
 at java.lang.Thread.run(Thread.java:745)
 ... ...
 Caused by: org.apache.hadoop.hbase.exceptions.UnknownProtocolException: org.apache.hadoop.hbase.exceptions.UnknownProtocolException: No registered coprocessor service found for name CubeVisitService in region KYLIN_35D0QUGYLW,,1479955424776.b053df01a747e3ace32a7fa7676edf0e.
 at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:5662)
 at org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3421)
 at org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3403)
 at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29929)
 at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
 at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
 at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:116)
 at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:96)
 at java.lang.Thread.run(Thread.java:745)
可能的原因:kylin与当前的hbase匹配有问题,在官网上找到了解决的方法:
http://kylin.apache.org/docs15/howto/howto_update_coprocessor.html
即,执行下面的shell命令:
$KYLIN_HOME/bin/kylin.sh org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI $KYLIN_HOME/lib/kylin-coprocessor-*.jar all
Logo

权威|前沿|技术|干货|国内首个API全生命周期开发者社区

更多推荐