JBrainy: Micro-benchmarking Java Collections with Interference (Work in Progress Paper)

Software developers use collection data structures extensively and
are often faced with the task of picking which collection to use.
Choosing an inappropriate collection can have major negative
impact on runtime performance. However, choosing the right collection
can be difficult since developers are faced with many possibilities,
which often appear functionally equivalent. One approach
to assist developers in this decision-making process is to microbenchmark
datastructures in order to provide performance insights.
In this paper, we present results from experiments on Java collections
(maps, lists, and sets) using our tool JBrainy, which synthesises
micro-benchmarks with sequences of random method calls.
We compare our results to the results of a previous experiment on
Java collections that uses a micro-benchmarking approach focused
on single methods. Our results support previous results for lists, in
that we found ArrayList to yield the best running time in 90% of
our benchmarks. For sets, we found LinkedHashSet to yield the
best performance in 78% of the benchmarks. In contrast to
previous results, we found TreeMap and LinkedHashMap to yield better
runtime performance than HashMap in 84% of cases.