Last month I attended a superb presentation on JBoss Cache by the project lead, Manik Surtani. Luckily only six people showed up, so were were able to chat to Manik, and really pick his brains on this stuff. Talking to Manik gave me a huge amount of confidence in the JBoss Cache product.
Some background to all of this: we've all probably kept something that's frequently used, and infrequently changing, in memory to save a trip to a database. That's one of the things you can use a cache like JBoss Cache for, but that are other benefits too: the preference for code that's been written elsewhere (many eyes etc); a smart way to do invalidation; a way to distribute a cache between servers; reporting tools... and so on.
JBoss Cache isn't the only solution out there. There are a bunch of APIs you can use to add caching into your application, and maybe one day the Java Temporary Caching API (JSR 107) standard will mean you can use one API across all the implementations. I was wondering... how do they compare. I decided to have quick look at performance for three of the popular ones.
I'm looking at:
I have a scenario in mind, where there are a lot of cache hits and only a few cache stores. In particular I wanted to know how the implementations perform under concurrent access, such as a heavily loaded web site: what happens to the cache performance as load increases.
To set the stage here's a graph showing the performance of a work load when using the ye olde synchronized
Here's what you're looking at. Along the x-axis is the number of threads concurrently accessing the cache. The y-axis is showing normalized throughput (so bigger is better on the y-axis), averaged over 5 runs. What I've done is create a worker that picks a random number, looks in the cache to see if it's there, and if it's not it associates a list against the number. There are only 400 numbers to pick from, and the worker tries this 40,000 times (weird numbers, but it's relevant to the application I have in mind). Each thread equates to one worker, and all the workers share the same cache instance. You can download the source code for this, if you want the details.
So focus on the orange curve in the middle first, which is the graph for my "noop" cache. It's doing pretty much what you'd expect. As the number of threads increases, the machine is able to perform more operations in the same amount of time, up to around 32 threads. After that, performs starts to fall off.
The purple line tells you that you don't want to use a
The green line shows you that the concurrent APIs in Java have come a long long way since
So much for the baseline measurements. Now let's see what happens with the Java caching implementations:
The first thing I need to say is that these tests are for local, non-replicated, non-transactional caching. I.e., it's the simplest case. I've also only done the bare minimum configuration on the caches to make them work. I'm sure they are tweekable. Also note that the y-axis values are an order of magnitude lower than the baseline graph.
It looks like JCS have really tuned the single thread case, but after that there's not a lot to choose between that and EHcache in terms of performance. Neither seem to suffer under increasing thread load. On the whole, I prefer the JCS API to the EHcahce one.
Don't be overly put off by the JBoss performance curve. Sure it looks a lot lower, but we're talking a total of 300ms difference at 4 threads for a total of 160,000 cache requests: there are bigger considerations that those performance measures.
So the conclusion from all of this is... they're all pretty good. I was hoping one would run so badly I could rule it out, but that's not the case, and I'm going to have to look at features.
For the simple case, I'm going to opt for JCS based on these numbers; but beyond that the numbers tell you nothing of use. For example, if you need transactional support, or want an object cache, then it has to be JBoss as that's the only one with those features. Or you may prefer one API other the others in terms of style.
As a final comment, do not trust these curves for replicated or distributed caches: the issues there in measuring performance add a lot of complexity.
Some background to all of this: we've all probably kept something that's frequently used, and infrequently changing, in memory to save a trip to a database. That's one of the things you can use a cache like JBoss Cache for, but that are other benefits too: the preference for code that's been written elsewhere (many eyes etc); a smart way to do invalidation; a way to distribute a cache between servers; reporting tools... and so on.
JBoss Cache isn't the only solution out there. There are a bunch of APIs you can use to add caching into your application, and maybe one day the Java Temporary Caching API (JSR 107) standard will mean you can use one API across all the implementations. I was wondering... how do they compare. I decided to have quick look at performance for three of the popular ones.
I'm looking at:
- JBoss Cache, version 2.0.0 RC3. LGPL license.
- JCS: Java Caching System, version 1.3. Apache license.
- Ehcache, version 1.3. Apache license.
I have a scenario in mind, where there are a lot of cache hits and only a few cache stores. In particular I wanted to know how the implementations perform under concurrent access, such as a heavily loaded web site: what happens to the cache performance as load increases.
To set the stage here's a graph showing the performance of a work load when using the ye olde synchronized
Hashtable, the spanky new ConcurrentHashMap and a "noop" implementation that neither stores nor returns anything. The idea here is just to provide a sanity check of how my laptop performs under increasing load.Here's what you're looking at. Along the x-axis is the number of threads concurrently accessing the cache. The y-axis is showing normalized throughput (so bigger is better on the y-axis), averaged over 5 runs. What I've done is create a worker that picks a random number, looks in the cache to see if it's there, and if it's not it associates a list against the number. There are only 400 numbers to pick from, and the worker tries this 40,000 times (weird numbers, but it's relevant to the application I have in mind). Each thread equates to one worker, and all the workers share the same cache instance. You can download the source code for this, if you want the details.
So focus on the orange curve in the middle first, which is the graph for my "noop" cache. It's doing pretty much what you'd expect. As the number of threads increases, the machine is able to perform more operations in the same amount of time, up to around 32 threads. After that, performs starts to fall off.
The purple line tells you that you don't want to use a
Hashtable for concurrent access. There's a lot of synchronization and waiting happening (probably) as threads compete to access the shared object.The green line shows you that the concurrent APIs in Java have come a long long way since
Hashtable. The ConcurrentHashMap implementation is blazing fast and doesn't seem to suffer too badly under increasing load. In fact, it has a better throughput than my noop code (possibly because noop always causes both a get and a put on the cache).So much for the baseline measurements. Now let's see what happens with the Java caching implementations:
The first thing I need to say is that these tests are for local, non-replicated, non-transactional caching. I.e., it's the simplest case. I've also only done the bare minimum configuration on the caches to make them work. I'm sure they are tweekable. Also note that the y-axis values are an order of magnitude lower than the baseline graph.
It looks like JCS have really tuned the single thread case, but after that there's not a lot to choose between that and EHcache in terms of performance. Neither seem to suffer under increasing thread load. On the whole, I prefer the JCS API to the EHcahce one.
Don't be overly put off by the JBoss performance curve. Sure it looks a lot lower, but we're talking a total of 300ms difference at 4 threads for a total of 160,000 cache requests: there are bigger considerations that those performance measures.
So the conclusion from all of this is... they're all pretty good. I was hoping one would run so badly I could rule it out, but that's not the case, and I'm going to have to look at features.
For the simple case, I'm going to opt for JCS based on these numbers; but beyond that the numbers tell you nothing of use. For example, if you need transactional support, or want an object cache, then it has to be JBoss as that's the only one with those features. Or you may prefer one API other the others in terms of style.
As a final comment, do not trust these curves for replicated or distributed caches: the issues there in measuring performance add a lot of complexity.
12 Comments:
Is there an interface to memcached?
Memcached rules.
Richard --
For a more robust distributed cache-implementation (aka, In-Memory Data Grid), you should look at GigaSpaces.
The throughput numbers you are getting here are far inferior to what you'll see from GigaSpaces. Not to mention what you will get in more complex scenarios such as transactions, multi-tiered cache, etc.
Any reason you did not evaluate whirlycache (https://whirlycache.dev.java.net/)? I have had great results with it on several projects.
Thanks for the suggestions for other caching systems. Proper across-the-network distributed caching is not something I absolutely need today, so I don't have a solid scenario to test out.
Regarding Whirlycache: I didn't test it, because I didn't know it existed. So... thanks for the URL.
I have implemented public website that uses ehcache to cache xml snippets as data elements - its been running in production from 6months - looks pretty good in terms of performance
Memcached is usually a very bad idea for Java. Java caching providers are much faster, mostly due to serialization overhead for memcached.
Yes there is an interface to memcached. Don't use it if you're not already running memcached.
e.g. see http://gregluck.com/blog/archives/2007/05/comparing_memca.html
for an ehcache and memcached comparison. Ehcache is up to several hundred times faster for local operations and up to 10 times faster for disk stored caches.
Peace
-stephan
Reposita Project Lead
Richard,
If you don't need the features of these other caches, like distributed access etc, why would you pick anything but ConcurrentHashMap? It is the fastest afterall...
There's also OSCache in the OSS Caching arena.
Check out http://www.terracotta.org/ It lets you start w/ EHCache today and cluster EHCache when you are ready. It will also mimick the JBossCache interface completely so if you want to start there, you can add clustering via Terracotta later just like with EHCache. Also working on OSCache and we are thinking about WhirlyCache.
Incidentally, I work for Terracotta and happen to know of 3 use cases where the customer confirmed it was 100X faster than JBossCache: http://www.terracotta.org/confluence/download/attachments/12545/TerracottaTreeCache.pdf?version=1
And, Terracotta is OSS like _most_ of the things already mentioned above.
Taylor: thanks for the comment. I do need the features because (for a starter) I don't want the cache to grow without limit.
Cheers - Richard
Richard,
I was just browsing the source code for JCS, turns out it uses a Hashtable under the covers. Since you found ConcurrentHashMap vs. Hashtable to be faster in your particular test, it might be worth your while to change the instantiation in JCS of Hashtable to ConcurentHashMap which should have similar syncrhonization properties so might "just work".
The Hashtable is instantiated in org.apache.jcs.memory.AbstractMemoryCache around line 101.
Cheers.
Do not underestimate the single thread case. Even in heavily threaded applications, the in memory caches are so fast (basically a table lookup) that threads do not spend time there. In that case there is rarely a wait or concurrency scenario. If you put this cache in front of a database call (to cache database results) the time spent in the lookup is so marginal comparing to the database call that you will rarely get more than one thread dealing with the cache.
This really put JCache in different light.
If you are really into squeezing numbers with minimal work, apache LRU map will give you performance similar to synchronized hashmap. It is limited in size, and adding time expiry is a few lines of code.
But having said all that, those are still marginal numbers comparing to a single database access which requires network.
Regarding performance under load with JBoss Cache, take a look at this:
http://jbosscache.blogspot.com/2008/07/mvcc-has-landed.html
I expect to be causing quite a lot of disruption among open src distributed caches with this release. ;)
Post a Comment
<< Home