雨翔河
首页
列表
关于
disconf 之服务器内部错误
2019-08-19 13:23
说来也奇怪，协助武汉研发部门的发版本，本来是一次很简单的版本发布，折腾到了晚上12点。 原因如下： > 发版之前进行修改 `disconf` 配置中心的配置数据，结果发现 `disconf` 的XX项目配置中心管理界面的配置文件页面为空。当时就吓坏了，以为是服务挂了，如果配置中心整个服务挂了就很严重了，马上看了下别的项目的配置中心页面是正常的， 唯独XX项目的是空白，赶紧看了下XX项目的应用日志，一切正常，由此马上F12查看 `disconf`后台的请求，发现返回了一段json错误信息，服务器内部错误。 进入生产环境查看 `disconf` 的报错日志，发现报错表现很模糊： ``` xx:xx:50.591 [http-bio-xxxxx-xxxx-exec-xx] WARN c.b.d.c.handler.MyExceptionHandler - /xxx/xx/xxx/list ExceptionHandler FOUND. com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected a string but was BEGIN_OBJECT at line 1 column 128 java.lang.IllegalStateException: Expected a string but was BEGIN_OBJECT at line 1 column 128 xx:xx:50.592 [http-bio-xxxxx-xxxx-exec-xx] WARN c.b.d.c.handler.MyExceptionHandler - details: com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected a string but was BEGIN_OBJECT at line 1 column 128 at com.google.gson.Gson.fromJson(Gson.java:815) ~[gson-2.2.4.jar:na] at com.google.gson.Gson.fromJson(Gson.java:768) ~[gson-2.2.4.jar:na] at com.google.gson.Gson.fromJson(Gson.java:717) ~[gson-2.2.4.jar:na] at com.github.knightliao.apollo.utils.data.GsonUtils.parse2Map(GsonUtils.java:44) ~[apollo-1.0.12.jar:na] at com.baidu.disconf.web.service.config.service.impl.ConfigMgrImpl.compareConfig(ConfigMgrImpl.java:306) ~[classes/:na] at com.baidu.disconf.web.service.config.service.impl.ConfigMgrImpl.getZkData(ConfigMgrImpl.java:215) ~[classes/:na] at com.baidu.disconf.web.service.config.service.impl.ConfigMgrImpl.convert(ConfigMgrImpl.java:280) ~[classes/:na] at com.baidu.disconf.web.service.config.service.impl.ConfigMgrImpl.access$000(ConfigMgrImpl.java:58) ~[classes/:na] at com.baidu.disconf.web.service.config.service.impl.ConfigMgrImpl$1.transfer(ConfigMgrImpl.java:188) ~[classes/:na] at com.baidu.disconf.web.service.config.service.impl.ConfigMgrImpl$1.transfer(ConfigMgrImpl.java:176) ~[classes/:na] at com.baidu.dsp.common.utils.ServiceUtil.getResult(ServiceUtil.java:35) ~[classes/:na] at com.baidu.disconf.web.service.config.service.impl.ConfigMgrImpl.getConfigList(ConfigMgrImpl.java:175) ~[classes/:na] at com.baidu.disconf.web.service.config.service.impl.ConfigMgrImpl$$FastClassBySpringCGLIB$$7099537e.invoke(<generated>) ~[spring-core-4.1.7.RELEASE.jar:na] at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) ~[spring-core-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:649) ~[spring-aop-4.1.7.RELEASE.jar:4.1.7.RELEASE] at com.baidu.disconf.web.service.config.service.impl.ConfigMgrImpl$$EnhancerBySpringCGLIB$$1f1102ef.getConfigList(<generated>) ~[spring-core-4.1.7.RELEASE.jar:na] at com.baidu.disconf.web.web.config.controller.ConfigReadController.getConfigList(ConfigReadController.java:90) ~[classes/:na] at com.baidu.disconf.web.web.config.controller.ConfigReadController$$FastClassBySpringCGLIB$$bed0282a.invoke(<generated>) ~[spring-core-4.1.7.RELEASE.jar:na] at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) ~[spring-core-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:717) ~[spring-aop-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85) ~[spring-aop-4.1.7.RELEASE.jar:4.1.7.RELEASE] at com.baidu.disconf.web.service.roleres.service.RoleResourceAspect.decideAccess(RoleResourceAspect.java:109) ~[classes/:na] at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_25] at java.lang.reflect.Method.invoke(Method.java:483) ~[na:1.8.0_25] at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621) ~[spring-aop-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610) ~[spring-aop-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68) ~[spring-aop-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:168) ~[spring-aop-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) ~[spring-aop-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:653) ~[spring-aop-4.1.7.RELEASE.jar:4.1.7.RELEASE] at com.baidu.disconf.web.web.config.controller.ConfigReadController$$EnhancerBySpringCGLIB$$5d4c3283.getConfigList(<generated>) ~[spring-core-4.1.7.RELEASE.jar:na] at sun.reflect.GeneratedMethodAccessor143.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_25] at java.lang.reflect.Method.invoke(Method.java:483) ~[na:1.8.0_25] at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221) ~[spring-web-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:137) ~[spring-web-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110) ~[spring-webmvc-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:776) ~[spring-webmvc-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:705) ~[spring-webmvc-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) ~[spring-webmvc-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:959) [spring-webmvc-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:893) [spring-webmvc-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:967) [spring-webmvc-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:858) [spring-webmvc-4.1.7.RELEASE.jar:4.1.7.RELEASE] at javax.servlet.http.HttpServlet.service(HttpServlet.java:624) [servlet-api.jar:na] at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:843) [spring-webmvc-4.1.7.RELEASE.jar:4.1.7.RELEASE] at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) [servlet-api.jar:na] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) [catalina.jar:7.0.68] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.68] at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) [tomcat7-websocket.jar:7.0.68] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [catalina.jar:7.0.68] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.68] at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:85) [spring-web-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [catalina.jar:7.0.68] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.68] at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:87) [spring-web-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.1.7.RELEASE.jar:4.1.7.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [catalina.jar:7.0.68] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.68] at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) [catalina.jar:7.0.68] at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) [catalina.jar:7.0.68] at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505) [catalina.jar:7.0.68] at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169) [catalina.jar:7.0.68] at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) [catalina.jar:7.0.68] at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956) [catalina.jar:7.0.68] at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) [catalina.jar:7.0.68] at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436) [catalina.jar:7.0.68] at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078) [tomcat-coyote.jar:7.0.68] at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625) [tomcat-coyote.jar:7.0.68] at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) [tomcat-coyote.jar:7.0.68] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_25] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_25] at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-coyote.jar:7.0.68] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_25] Caused by: java.lang.IllegalStateException: Expected a string but was BEGIN_OBJECT at line 1 column 128 at com.google.gson.stream.JsonReader.nextString(JsonReader.java:821) ~[gson-2.2.4.jar:na] at com.google.gson.internal.bind.TypeAdapters$13.read(TypeAdapters.java:358) ~[gson-2.2.4.jar:na] at com.google.gson.internal.bind.TypeAdapters$13.read(TypeAdapters.java:346) ~[gson-2.2.4.jar:na] at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.read(TypeAdapterRuntimeTypeWrapper.java:40) ~[gson-2.2.4.jar:na] at com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.read(MapTypeAdapterFactory.java:187) ~[gson-2.2.4.jar:na] at com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.read(MapTypeAdapterFactory.java:145) ~[gson-2.2.4.jar:na] at com.google.gson.Gson.fromJson(Gson.java:803) ~[gson-2.2.4.jar:na] ... 77 common frames omitted xx:xx:50.592 [http-bio-xxxxx-xxxx-exec-xx] INFO c.b.dsp.common.vo.JsonObjectUtils - JsonObjectBase [message={global=服务器内部错误}, sessionId=xxxxx-axx4-xx-xxx-fbxxxcx1xxa7, success=false] ``` 由此猜测是新增了配置文件，但是配置文件可能配型或者值不对或者出现了特殊字符，导致了解析失败了。 查了下应用的配置中心的代码，发现的确有新的配置文件且从生产数据库和 `zookeeper` 捞出来的数据来看配置本身含有特殊字符，猜测是中文注释,看了下我们自己的应用代码，发现开发人员使用了很奇怪的 `Map` 类型做 `key` 。 马上灵机一动在测试环境复现下这个问题，结果发现果然有办法复现，触发条件也和预想的一样。 触发条件： 使用 `Map` 类型，且首行使用中文注释，且定义的 `Map` 类型在 `disconf`不设置值。 配置好了之后，刷新 `disconf` 配置页面，发现页面报错 , 请求 `disconf` 返回json结果 `服务器内部错误`。 ### 解决方案： 换一个配置文件名称，去掉 `Map` 类型使用基本的 `String` 放置 `json` 数据, 自己通过代码转换 `json` 为对象，删除原配置文件，`disconf` 恢复正常。 根本原因没有去找，实在是没有时间去看 `disconf` 的源码。。。 #### 附上出问题的版本号： `disconf` 版本： 2.6.30
类型：工作
标签：disconf,错误
Copyright © 雨翔河
我与我周旋久
独孤影