The stack digest stability challenge

This page explains why it is not a trivial thing to compute a stable digest from a full Java stack trace, and concludes with some computation rules.

A few examples

Let's consider error stack 1

(the stack trace presented here has been cut by half from useless lines)

com.xyz.MyApp$MyClient$MyClientException: An error occurred while getting Alice's things^(msg)
  at com.xyz.MyApp$MyClient.getTheThings(MyApp.java:26)
  at com.xyz.MyApp$MyService.displayThings(MyApp.java:16)
  at com.xyz.MyApp$MyService$$FastClassByCGLIB$$e7645040.invoke()^(aop)
  at net.sf.cglib.proxy.MethodProxy.invoke()^(aop)
  at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint()^(fwk)
  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed()^(fwk)
  at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed()^(fwk)
  at sun.reflect.NativeMethodAccessorImpl.invoke0()^(aop)
  at sun.reflect.NativeMethodAccessorImpl.invoke()^(aop)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke()^(aop)
  at java.lang.reflect.Method.invoke()^(aop)
  at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs()^(fwk)
  at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod()^(fwk)
  at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke()^(fwk)
  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed()^(fwk)
  at org.springframework.aop.interceptor.AbstractTraceInterceptor.invoke()^(fwk)
  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed()^(fwk)
  at org.springframework.transaction.interceptor.TransactionInterceptor.invoke()^(fwk)
  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed()^(fwk)
  at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke()^(fwk)
  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed()^(fwk)
  at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept()^(fwk)
  at com.xyz.MyApp$MyService$$EnhancerBySpringCGLIB$$c673c675.displayThings(<generated>)^(aop)
  at sun.reflect.GeneratedMethodAccessor647.invoke(Unknown Source)^(aop)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)^(aop)
  at java.lang.reflect.Method.invoke(Method.java:498)^(aop)
  at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)^(fwk)
  at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)^(fwk)
  at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:116)^(fwk)
  at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)^(fwk)
  at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)^(fwk)
  at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)^(fwk)
  at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963)^(fwk)
  at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897)^(fwk)
  at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)^(fwk)
  at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861)^(fwk)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:624)^(jee)
  at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)^(fwk)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)^(jee)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)^(jee)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)^(jee)
  at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)^(jee)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)^(jee)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)^(jee)
  ...
  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)^(fwk)
  at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)^(fwk)
  at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)^(fwk)
  at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)^(fwk)
  at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)^(fwk)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)^(jee)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)^(jee)
  ...
  at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)^(jee)
  at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436)^(jee)
  at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078)^(jee)
  at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)^(jee)
  at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)^(jee)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)^(jee)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)^(jee)
  at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)^(jee)
  at java.lang.Thread.run(Thread.java:748)^(jee)
  ...
Caused by: com.xyz.MyApp$HttpStack$HttpError: I/O error on GET http://dummy/user/alice/things^(msg)
  at com.xyz.MyApp$HttpStack.get(MyApp.java:40)
  at com.xyz.MyApp$MyClient.getTheThings(MyApp.java:24)
  ... 23 common frames omitted
Caused by: java.net.SocketTimeoutException: Read timed out^(msg)
  at com.xyz.MyApp$HttpStack.get(MyApp.java:38)
  ... 24 common frames omitted

~~Strike out elements~~ may vary from one occurrence to the other:

~~error messages~~^(msg) often contain stuff related to the very error occurrence context,
~~AOP generated classes~~^(aop) may vary from one execution to another.

Italic elements are somewhat not stable, or at least useless (purely technical). Ex:

JEE container stuff^(jee): may change when you upgrade your JEE container version or add/remove/reorganize your servlet filters chain for instance,
Spring Framework^(fwk) underlying stacks (MVC, security) for pretty much the same reason,
AOP and dynamic invocation^(aop): purely technical, and quite implementation-dependent.

Only bolded elements are supposed to be stable.

Now let's consider error stack 2

(shortened)

com.xyz.MyApp$MyClient$MyClientException: An error occurred while getting Bob's things^(msg)
  at com.xyz.MyApp$MyClient.getTheThings(MyApp.java:26)
  at com.xyz.MyApp$MyService.displayThings(MyApp.java:16)
  at com.xyz.MyApp$MyService$$FastClassByCGLIB$$07e70d1e.invoke()^(aop)
  at net.sf.cglib.proxy.MethodProxy.invoke()^(aop)
  at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint()^(fwk)
  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed()^(fwk)
  at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed()^(fwk)
  at sun.reflect.NativeMethodAccessorImpl.invoke0()^(aop)
  at sun.reflect.NativeMethodAccessorImpl.invoke()^(aop)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke()^(aop)
  at java.lang.reflect.Method.invoke()^(aop)
  at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs()^(fwk)
  at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod()^(fwk)
  at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke()^(fwk)
  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed()^(fwk)
  at org.springframework.aop.interceptor.AbstractTraceInterceptor.invoke()^(fwk)
  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed()^(fwk)
  at org.springframework.transaction.interceptor.TransactionInterceptor.invoke()^(fwk)
  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed()^(fwk)
  at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke()^(fwk)
  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed()^(fwk)
  at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept()^(fwk)
  at com.xyz.MyApp$MyService$$EnhancerBySpringCGLIB$$e3f570b1.displayThings(<generated>)^(aop)
  at sun.reflect.GeneratedMethodAccessor737.invoke(Unknown Source)^(aop)
  ...
Caused by: com.xyz.MyApp$HttpStack$HttpError: I/O error on GET http://dummy/user/bob/things^(msg)
  at com.xyz.MyApp$HttpStack.get(MyApp.java:40)
  at com.xyz.MyApp$MyClient.getTheThings(MyApp.java:24)
  ... 23 common frames omitted
Caused by: java.net.SocketTimeoutException: Read timed out^(msg)
  at com.xyz.MyApp$HttpStack.get(MyApp.java:38)
  ... 24 common frames omitted

You may see in this example that most of the strike elements have slight differences from error stack 1 (messages and generated classes names).

Nevertheless it is the same exact error (despite the context is different as it applies to another user), and the goal here is to be able to count them as two occurrences of the same error.

Now let's consider error stack 3

(shortened)

com.xyz.MyApp$MyClient$MyClientException: An error occurred while getting Alice's things^(msg)
  at com.xyz.MyApp$MyClient.getTheThings(MyApp.java:26)
  at com.xyz.MyApp$MyService.displayThings(MyApp.java:16)
  at com.xyz.MyApp$MyService$$FastClassByCGLIB$$e7645040.invoke()^(aop)
  at net.sf.cglib.proxy.MethodProxy.invoke()^(aop)
  at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint()^(fwk)
  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed()^(fwk)
  at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed()^(fwk)
  at sun.reflect.NativeMethodAccessorImpl.invoke0()^(aop)
  at sun.reflect.NativeMethodAccessorImpl.invoke()^(aop)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke()^(aop)
  at java.lang.reflect.Method.invoke()^(aop)
  ...
Caused by: com.xyz.MyApp$HttpStack$HttpError: I/O error on GET http://dummy/user/alice/things^(msg)
  at com.xyz.MyApp$HttpStack.get(MyApp.java:40)
  at com.xyz.MyApp$MyClient.getTheThings(MyApp.java:24)
  ... 23 common frames omitted
Caused by: javax.net.ssl.SSLException: Connection has been shutdown: javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown^(msg)
  at com.sun.net.ssl.internal.ssl.SSLSocketImpl.checkEOF(SSLSocketImpl.java:1172)
  ... 24 common frames omitted

Here, you can see that the first and second errors are the same as in error stack 1, but the root cause is different (SSLException instead of SocketTimeoutException).

So in that case we don't want the top error digest computed for error stack 3 to be the same as for error stack 1.

Stack digest computation rules

As a conclusion, stack digest computation applies the following rules:

a stack digest shall not compute with the error message
a stack digest shall compute with it's parent cause (recurses)
in order to stabilize the stack digest (over time and space), it's recommended to exclude non-stable elements