Comments on: RE: Generics Microbenchmark http://blog.anthonychaves.net/2007/03/13/re-generics-microbenchmark/ Life is software and jujitsu Mon, 01 Feb 2010 17:22:12 +0000 http://wordpress.org/?v=2.9.1 hourly 1 By: Kirk http://blog.anthonychaves.net/2007/03/13/re-generics-microbenchmark/comment-page-1/#comment-629 Kirk Thu, 15 Mar 2007 15:54:03 +0000 http://blog.anthonychaves.net/personal/2007/03/13/re-generics-microbenchmark#comment-629 ooops, should have said, glad to see that you finally came to a good conclusion. -- Kirk ooops, should have said, glad to see that you finally came to a good conclusion.

– Kirk

]]> By: Kirk http://blog.anthonychaves.net/2007/03/13/re-generics-microbenchmark/comment-page-1/#comment-628 Kirk Thu, 15 Mar 2007 14:35:27 +0000 http://blog.anthonychaves.net/personal/2007/03/13/re-generics-microbenchmark#comment-628 The problem with Microbenchmarks is that it is very difficult to understand what you are really benchmarking. One of the tools that is absolutely necessary in helping you to understand if you're benchmark has any validity to it is statistics. Sorry if this sounds harsh but one has to automatically reject outright the results of any benchmark that only includes an average. You need to include variance at a minimum. But I would suggest that max, min and medium are also very useful when evaluating the results of an MBM. IMHO (I've not run this benchmark myself yet), it looks biased against generics (GC issues). Also there is no consideration of what hotspot is doing. I would suggest that these issues need to be taken care of as well as the two test be separated into different tests that are run separately. Once these changes are in place I would be surprised if there is a difference between the two test cases. Kind regards, Kirk Pepperdine The problem with Microbenchmarks is that it is very difficult to understand what you are really benchmarking. One of the tools that is absolutely necessary in helping you to understand if you’re benchmark has any validity to it is statistics. Sorry if this sounds harsh but one has to automatically reject outright the results of any benchmark that only includes an average.

You need to include variance at a minimum. But I would suggest that max, min and medium are also very useful when evaluating the results of an MBM.

IMHO (I’ve not run this benchmark myself yet), it looks biased against generics (GC issues). Also there is no consideration of what hotspot is doing. I would suggest that these issues need to be taken care of as well as the two test be separated into different tests that are run separately.

Once these changes are in place I would be surprised if there is a difference between the two test cases.

Kind regards,
Kirk Pepperdine

]]> By: Geoffrey Wiseman http://blog.anthonychaves.net/2007/03/13/re-generics-microbenchmark/comment-page-1/#comment-624 Geoffrey Wiseman Wed, 14 Mar 2007 10:01:12 +0000 http://blog.anthonychaves.net/personal/2007/03/13/re-generics-microbenchmark#comment-624 Just to clarify: in my case, I was comparing these two: List = new ArrayList(); List = Collections.checkedList( new ArrayList(), String.class ); The latter performs more slowly for me. Just to clarify: in my case, I was comparing these two:
List = new ArrayList();
List = Collections.checkedList( new ArrayList(), String.class );

The latter performs more slowly for me.

]]>