Building software modules within a team requires communicating, at a minimum,  method or class signatures and a general idea of the functionality.  This may include a detailed design document clearly specifying method signature and a pseudo-code implementation or method stubs and comments about what the method should do right in the source file.  One thing that should never be used for communicating a signature and functionality is dummy code.

Dummy code looks like real code.  It is real code but it rarely works.  Even if it compiles it doesn’t produce the correct result when run.  Nor are there any tests that prove it (in)correct.  Dummy code exists only for the sake of its author’s gratification.

Rather than talk about the desired functionality through some more productive means the dummy code author thinks he is helpfully providing a guesstimate of correct implementation.  What he has actually done is provide a jumbled mess of non-working code cleverly disguised as a working part of the project when he checks it in to the SCM.  The dummy code author thinks someone else will come along and provide the correct implementation “later”.

What he doesn’t realize is that his dummy code is almost indistinguishable from code that should be part of the project.  The next person to try to reuse this code is in for hours of WTF moments trying to figure out WTF the code is supposed to do and why it’s got code branching five levels deep.

At my first post-college job I had a wonderful mentor.  Thys (pronounced Tays, like “taste” without the second ‘t’) and I were talking one day when he asked me a question.  “Anthony, if you were given a task of writing software that would only be run once would you do it quick-and-dirty or with a well-measured, engineered approach?”

How bad could “quick-and-diry” be if the software would run only once?  I thought about it for a moment before answering.  Quick and dirty.

Thys said he would never write quick and dirty software.  Doing so figuratively creates a  monkey on the developerment team’s back.  The software that only needs to run once inevitably needs to run a second time and a third time and eventually run regularly.  Thys helped me realize that software evolves and escapes.  Useful software will run indefinitely.  Any time and effort saved on the initial write will be lost 10 times (I made that number up.  It is actually higher.  Any want to provide real data or examples?) over on the maintenance, bug fixes and rewrites.

Writing dummy code is writing software quick and dirty without any idea of the implementation details.  Bad code and bad algorithms.  The worst of both worlds.  It’s been a long time since Thys and I had that conversation but the years have proven him right again and again.  Any code worth checking in to SCM is worth writing well.  That includes unit tests.

Dummy code has no place in SCM.  If you’re not going to correctly implement a method you stub out then don’t provide an implementation at all.  Leave some comments around the method stub if you have to.  Open a ticket in the issue tracking system to let someone know the method needs an implementation and describe the input and output.  Let the implementer determine which algorithms and data structures to use.

Dummy code is a hazard to software projects.  It violates the principle of least astonishment.  Next time you think about writing some remember that the next person that uses the code will waste time figuring out why it doesn’t work as expected.

If you check code in to your SCM where other people can pull it, take a look at what you’re checking in.  Does it work?  Does it work well?  Are there tests that prove it?  Helping someone by coding less is sometimes the best help of all.

I don’t make any money running the group.  In fact it costs me money.  I do a good amount of planning, coordinating, buying, booking and raffle prize finding for the group.  I love seeing 30+ people show up for our after work meetings.   It’s a lot of work but I do it because I am passionate about emerging trends in application scalability.  It’s a labor of love that I find professionally and personally rewarding.  Like all good geeks I maintain a healthy hobby writing software outside business hours.

I’m interested in building bigger, better software.  Batch applications, database interactions, web applications, CPU intensive algorithims should be as effective as possible.  Effective does not necessarily mean efficient.  Effectiveness is based on price, time-to-market, performance, maintenance cost, etc.  Analyzing these facets is important. The tools used to arrive at the solution are important, too.

BostonSUG has wonderful speakers who generously donate their time to speak at our meetings.  These people come in from all over the country.  Just a few examples, we’ve hosted Orion Letizi and Nikita Ivanov from San Francisco, Billy Newport from Minnesota and Mike Culver from Washington.  We’ve had numerous local speakers who we appreciate just as much as those that fly to see us.  Patrick Peralta, Tom O’Hare and Rakesh Chaudhary had large audiences.  Each of these speakers is an expert in a different facet of building scalable applications.  They are industry leaders in compute clouds, data grids, object caching, streaming media and more.

One thing we haven’t touched yet is building a web site that performs well.  The person that replied to the meeting reminder said:

save your time:
only put strings in session; and only put one item per-user in the session; unless they are doing some heavy form processing.
use native clustering.
put professional load balancer in from.
max pipe.

Thanks.  That is a helpful reply.  It immediately invalidates the hours of insightful discussion we’ve had at past BostonSUG meetings that have spent less than five minutes total dedicated to the problem to which he so generously offers his solution</sarcasm>.

This is a picture of what is described in the reply post.  As an industry we have known this works for a long time.  That’s why we don’t talk about the architecture depicted by this diagram at BostonSUG.  Sure, we’ll talk about HOW an app server achieve scalability.  We’ll talk about the implementation, not just about using JBoss or Mongrel.  We got a good look inside GridGain.  We had a guided tour around Terracotta.  BostonSUG evokes deeper discussion than “small sessions, load balancer, database cluster, done!”.  You should do that anyway.

Past meetings aside we have a lot to talk about this year.  Rails 3, Java 7, CloudFront, Hibernate Shards, Google AppEngine, tons of topics that push the industry foward.  If you’re interested in evolving and not building the same old software then come to BostonSUG.  If you’re interested in building what we know works (and have known since pre-2000) then it’s not the meeting for you.

I didn’t intend for this to turn into a meeting plug but I might as well include the link to the meeting announcement.  Wednesday January 28, 2009 @ 6 p.m.  IBM Innovation Center Waltham, MA 02451

In this case I am unfortunately duplicating data by separating the reads from the writes. The data lives in the Hibernate session used by the read-only app server in order to load the data in the first place. It also has to be loaded in the write-only app server in order for that Hibernate session to work with it, which brings me to the point of this post: Is there any way that I can use Terracotta to pool my Hibernate-managed objects into one pool used by both the read-only and write-only app servers? I think I would like to pool my Hibernate second-level cache. It seems like Terracotta can get the job done.

That said, I’m anxious to explore this topic more when Orion Letizi from Terracotta Tech comes to speak at the Boston Scalability User Group next Wednesday on May 28th. The meeting starts at 6 p.m. at the IBM Innovation Center in Waltham, MA. Of course there will be pizza and soda (sponsored by Terracotta – thank you!) so come to the meeting to have some food and explore what Terracotta can do for your project. All the details, including directions, are on the Boston Scalability User Group web site. Remember, there will be pizza!

Right away it looks like this query will get very expensive for large tag sets. For smaller sets the query performs in a reasonable amount of time but more than 12 tags makes the query take quite a while. This is exacerbated by the fact that this query will be executed multiple times if there is a sufficiently large result set (default is over 25) to give the user a paginated view. This totally flies in the face of being nice to the database. Hammering it with a complex query over and over won’t do the users any favors.

What are my options here? I can work on making this query less complex (something that I will certainly do), but there is still the problem that, no matter how nice this query becomes, the query will be executed over and over as a user pages through the results. The complexity of the query and its repeated execution makes the result set a good candidate for caching. Caching the result set in memory with a key tied to the user or their session will allow the expensive query to be executed once with the bookmarks stored in a cache. When the user pages through the results the cache will be consulted first where the result set will be stored. I’m in the process of selecting a caching package for this project so I will post my selection and integration process soon.

Until then, I’d be happy to hear any suggestions on making this query a little nicer while preserving its “andness”.

I used jhat to find out that each bookmark object uses 1.5 kilobytes of memory. It doesn’t take much to see that a LOT of bookmarks will fit into 8 gigs of memory on a medium sized server. It would take hundreds of thousands of users to generate enough bookmark data to fill that memory. At this point it’s clear the application server box can handle the data load (that isn’t to say that it can handle the network load, however).

I spent some time coming up with a nice data model and ORM design so accessing the data once it’s in memory is lightning fast, maybe even faster. But what about when it’s on a hard disk, deep in the database? Dragging my 700 bookmarks out of the database when I only want to view the 25 most frequently visited is a bit overkill, especially when you consider dragging everyone else’s bookmarks out at the same time. Even though this is still a small-time application I decided to be nice to the database anyway and implement some kind of paging solution. Spring Web MVC doesn’t offer much that I’m aware of in regard to pagination help so it looks like I’m on my own.

Pagination is something that must take place in the model with the help of the view. Right away we’ve got two things to deal with, though it’s best to deal with them in isolation. The first thing I did was get the model right. I’m using Hibernate for ORM and the Criteria API makes it easy to set offsets and limits. That is, I can ask for the 50 next items (limit) starting at row 20 (offset).

The Criteria API is great at getting your data into a workable format without HQL or SQL (not that either of them is a bad thing. I enjoy working with them). Let’s take a look at the method that gets some new bookmarks.

public List<Bookmark> getNewestBookmarks(int offset, int numBookmarks) { DetachedCriteria c = DetachedCriteria.forClass(Bookmark.class); c.addOrder(Order.desc("createdDate")); return getHibernateTemplate().findByCriteria(c, offset, numBookmarks); }

So this method gets the model for us, but where do the offset and limit come from? From the controller, of course. This method is called from the controller like so:

public ModelAndView newest(HttpServletRequest request, HttpServletResponse response) { int numberOfBookmarks = requestUtils.getSafeValueFromRequest(request, "numberOfBookmarks", 250, defaultNumberOfNewest); int offset = requestUtils.getSafeValueFromRequest(request, "offset", 300, 0); List bookmarks = bookmarkService.getNewestBookmarks(offset, numberOfBookmarks); Map model = requestUtils.generateBookmarkModel(offset, numberOfBookmarks, 300, bookmarks); model.put("bookmarkListName", "Newest Bookmarks"); return new ModelAndView("index", model); }

After sanity checking the offset and limit received from the request they are passed on to the getNewestBookmarks method to create and execute a somewhat unsightly query. At this point, rather than gathering up all 700 of my bookmarks only 25 or so are returned. What if I one day want to view the top 50 most visited bookmarks though? Or what if I want to scroll through them one page at a time? Passing in 0, 25 for the offset and limit of every method call won’t cut it. The view needs to tell us how many bookmarks to retrieve and where to start the search.

Let’s say I’m viewing the top 25 bookmarks in my collection and I want to view 26-50. Somehow the view has to tell the controller to tell the model about the desired offset and limit. This can be done by passing them in as request parameters. In the code sample above, the HttpServletRequest object passed in to the controller contains values for offset and limit (numberOfBookmark) variables. The JSP code below shows how to pass these parameters when a user clicks a link. (Actually I’m having a hard time getting the JSP to show up right when the code is rendered as HTML. Just click the link and view the entire file instead.)

JSP Pagination Operations

This page uses the JSTL core tag library to build the links that display the “previous/next/more/less” options. Right after the body tag there is code to create four URLs, one for each action. Each one starts off with a name and a value. c:url var=”viewMoreUrl” value=”" creates a page-scoped variable called viewMoreUrl. This variable will later be used to display the link that allows me to display five more bookmarks when I view the list. The value is left blank which tells the tag to use the current base URL in creating the new one. If I’m on a page that is located at /bookmarks/pages/viewUserBookmarks.html the new URL will start off as /bookmarks/pages/viewUserBookmarks.html.

Adding parameters to a URL is easy. Using the c:param tag we get c:param name=”numberOfBookmarks” value=”${numberOfBookmarks + 5}”. This tag appends a variable called numberOfBookmarks to the base URL described in the previous paragraph. The value is a JSP expression that uses the numberOfBookmarks value passed in the model and increases it by five to pass back in the request. Because we want the link only to display five more bookmarks and not change the offset that value will remain the value passed out in the model. That gives us a URL that looks like /bookmarks/pages/viewUserBookmarks.html?numberOfBookmarks=30&offset=0.

When this URL is accessed the request values for offset and numberOfBookmarks are exposed in the HttpServletRequest object for use by the controller. When the getNewestBookmarks method is called the offset and limit are 0 and 30 instead of 0 and 25.

Obtaining a new offset value is done like getting a new limit value. We start off with the same base URL and leave the numberOfBookmarks value unchanged. We need to change the offset value though. c:param name=”offset” value=”${offset + numberOfBookmarks}” creates the offset value of the URL. If we are viewing the newest bookmarks (offset 0) and we are viewing 30 bookmarks when we click the link we should see bookmarks 31-60.

Here is an interesting thing to consider: what if a user goes to the second page using the default values. The user will be viewing bookmarks 26-50. The user then clicks the “view more” link three times and is now viewing 40 bookmarks. The “previous” will now request that the offset be -15. Unfortunately we can’t have negative bookmarks. In this case the controller has some sanity checking built in to it. The input parameter values must always be validated before they are used. The controller uses a utility method to get a safe value for the offset and limit and ensures that neither can be a negative value. This safe value is then passed back to the view once the model has been built. Even though the offset in the request says -15 the controller will only go as low as 0. More interesting work takes place in the generateBookmarksModel method, which I will talk about more next week.

What did I learn from making this effort? Well, giving 8GB of RAM to an application server is a little overkill for a domain model like this. The pagination that I came up with is still rudimentary, but a little more work could add more features, namely displaying page number links and the ability to change the number of bookmarks added at a time.

What is gained from this effort? Being nice to the database opens up a few doors. It dramatically improves scalability because it allows these requests to be stateless. If an app server queried the database for all of my bookmarks then my session would have to be directed to that app server every time I made a request in order to not incur the cost of hitting the database again by another app server. If round-robin load balancing is used the Hibernate session on each app server would contain a copy of my data and that’s a waste of resources, most notably it’s a hit to the database that didn’t have to happen.

Reducing the load on the database by getting only what we need and stateless sessions gives me the option of scaling the database with less difficulty. A mostly-read application can get by with fewer hits to the write master and do a majority of their work from read-only slaves. Spreading complex queries over multiple read slaves is much preferable to every app server hitting one database for every query. By having stateless requests I can have more app servers (using less memory) hitting more database servers without maxing out the CPU utilization due to the cost of complex queries.

What should I do with the unused memory in the app server boxes? Rather than give it to one app server instance for use as a cache it should be given to the pool of application servers for use as a cache. One way to do that is with memcached. I’ll talk about that more in the future too.

As always please leave comments. I enjoy getting feedback on what I’ve written.