When creating software, two people will never write the same implementation of a method or system of non-trivial design. Creating software is a problem solving process and there are usually many ways to solve one problem. The solutions may differ in elegance and efficiency while giving the same output for a given set of inputs. A correct solutions is a correct solution regardless of implementation. (more…)
Today I let out a dejected sigh after reading a response to my latest post on the Joel on Software message board. My post was just a reminder about the BostonScalability User Group meeting that takes place next Wednesday the 28th. I post the meeting announcements on JoS and a good number of people hear about the meetings from the JoS board. Every month a few new people tell me they found the group through JoS. So why did I sigh after reading the response to the meeting reminder?
I’m developing a Java/Spring web application deployed on JBoss 4.2.2 that makes heavy use of memcached in order to reduce the latency experienced by the user. The user interacts with data that is (unbeknownst to them) in the cache. In order to make changes permanent a message is placed on a queue that is picked up by an MDB. The message is just the key used to access the data in the cache. The MDB takes the data out of the cache and then saves it to the database. This greatly improves the user experience because they do not have to wait for the next page to load while costly database writes occur. So far, so good.
In this case I am unfortunately duplicating data by separating the reads from the writes. The data lives in the Hibernate session used by the read-only app server in order to load the data in the first place. It also has to be loaded in the write-only app server in order for that Hibernate session to work with it, which brings me to the point of this post: Is there any way that I can use Terracotta to pool my Hibernate-managed objects into one pool used by both the read-only and write-only app servers? I think I would like to pool my Hibernate second-level cache. It seems like Terracotta can get the job done.
That said, I’m anxious to explore this topic more when Orion Letizi from Terracotta Tech comes to speak at the Boston Scalability User Group next Wednesday on May 28th. The meeting starts at 6 p.m. at the IBM Innovation Center in Waltham, MA. Of course there will be pizza and soda (sponsored by Terracotta – thank you!) so come to the meeting to have some food and explore what Terracotta can do for your project. All the details, including directions, are on the Boston Scalability User Group web site. Remember, there will be pizza!
Here is the use case: I want to search all of my bookmarks by multiple tags with an “and” operation. That means I want to see bookmarks that are tagged with Java AND JavaEE, not just Java and not just JavaEE. I want to see bookmarks tagged with C# AND 3.5 not just C# and not just 3.5. For most of the project I’m using Hibernate and the Criteria API with a lot of success. In this case I was not able to massage Hibernate into creating the correct query and that’s fine. Hibernate has its uses and it is acceptable to write custom SQL when the need arises. I wrote a method to build up the query and execute it. Here is part of the method that builds up the query:
Right away it looks like this query will get very expensive for large tag sets. For smaller sets the query performs in a reasonable amount of time but more than 12 tags makes the query take quite a while. This is exacerbated by the fact that this query will be executed multiple times if there is a sufficiently large result set (default is over 25) to give the user a paginated view. This totally flies in the face of being nice to the database. Hammering it with a complex query over and over won’t do the users any favors.
What are my options here? I can work on making this query less complex (something that I will certainly do), but there is still the problem that, no matter how nice this query becomes, the query will be executed over and over as a user pages through the results. The complexity of the query and its repeated execution makes the result set a good candidate for caching. Caching the result set in memory with a key tied to the user or their session will allow the expensive query to be executed once with the bookmarks stored in a cache. When the user pages through the results the cache will be consulted first where the result set will be stored. I’m in the process of selecting a caching package for this project so I will post my selection and integration process soon.
Until then, I’d be happy to hear any suggestions on making this query a little nicer while preserving its “andness”.
Lately I’ve been working on a project that keeps my web bookmarks sync’d across multiple computers and different web browsers. The project is implemented in Java using JBoss 4.2.2, MySQL 5.0 and Spring 2.0. It’s not a very big project right now, only about 40 class files and some small number of configuration files, but the amount of data the application deals with can get fairly large.