When creating software, two people will never write the same implementation of a method or system of non-trivial design. Creating software is a problem solving process and there are usually many ways to solve one problem. The solutions may differ in elegance and efficiency while giving the same output for a given set of inputs. A correct solutions is a correct solution regardless of implementation. (more…)
A few of my most recent jobs have been with start-ups. Working with a start-up can be stressful due to the urgency of getting to a product release. Some of the employees of one particular start-up I worked with put in over 70 hours per week. This wasn’t because they were under-staffed. There were plenty of people to do the work but the company leadership was unable to come up with any sort of plan or priorities for the work. What they had was people furiously working on projects that were irrelevant to the company. Some people were working on projects that wouldn’t be needed for years, if ever. One thing I recall about this company is that a lot of things were broken.
Over the last few years I’ve worked with a few different styles of CEO. Some were great. They cared about employee morale and worked to solve problems created by problematic career middle-managers. They inspired employees and worked to help everyone realize their contribution to the company’s success. Other CEOs I’ve worked with didn’t have a handle on issues in the company. They almost gleefully stifled creativity and smothered morale. I’d like to share a tragic story about one of these CEOs.
I’ve been digging around lately for a Boston area user group dedicated to architectural scalability and I haven’t been able to find one. Other user groups that I regularly attend have meetings centered around scalability once in a while, but I’m looking for something with a schedule dedicated to the topic. It’s a hot topic in the industry right now with a lot happening on different fronts and there should be some ongoing professional discussion dedicated what goes into growing an application.
Here are some of the topics I want to talk about:
- Data growth and access – What kind of DBMS topologies allow maximum scalability without breaking the budget? What if the budget wasn’t a problem? How do you migrate your data model from hundreds of users to millions of users? Should you partition user data across shards or keep it in a central database? How do you profile data access paterns? Is your application mostly-read or mostly-write? When should you use an LDAP directory for data storage? When should you use MySQL and when should you use Oracle?
- Application server scaling – app server clustering, web server integration, load balancing, application session management, data caching
- Web Tier – load balancing reverse proxies, data caching, working with HTTP, considerations for exposing functionality via REST
- Language conisderations and platform choices – Dynamic languages vs. Static languages, Linux vs. Windows, Solaris vs. Linux, RedHat vs. Oracle, IBM vs. Oracle – How do you make these choices? What evaluation critera are most important? Which ones are misleading?
- Framework scalability – How can you scale if your framework can’t? Does Rails really scale better than Spring Web MVC? Where do PHP frameworks fit in? What are the alternatives? This is where we will investigate what you gain and lose by binding yourself to a particular framework, how to keep the coupling to a minimum and how to use your framework as a solid foundation instead viewing it as a cage.
- Emerging technology – Should you become familiar with Map Reduce and Hadoop? What kind of impact do object databases have on your application? Should you buy your own Sun or Dell boxes or use Amazon’s EC2 and S3?
- Application architecture – How do you write an application that scales? What does your application look like as it grows from servicing hundreds to millions of users? How does it handle session management? How does it access datastores? Are there any design patterns that are helpful? What are the anti-patterns to be aware of?
Like I said, I haven’t found a group dedicated to discussing these topics. If you know of one in the Boston area please let me know. Assuming there isn’t one I am willing to start one. I’d like to start off small and meet at coffee shops around Burlington or Lowell. I have no delusions that this is going to start off or even become as big as NEJUG is now. If it starts off as a few people getting together to talk about scalability trends, cool caching solutions and specific products then I’d call it a good start.
The meeting location is still TBD and will be based on how many people are interested in attending. It will probably be somewhere in Burlington, MA. There is no planned presentation at this time. Instead we will have a meet and greet and then discuss scalability trends and news, what approaches to scalability are commonly used now and what are the plans for the future.
If you are interested in coming or in finding out more information please email me at <my first name>@<my domain name>.<my tld> or leave a comment below. Please make sure to include your email address when you fill out the form so that I can get in touch with you – your email address will not be displayed on my site.
This is the final part (for now) of the thoughts I’ve collected on improving the relationship between software developer and QE to find high-risk bugs sooner. You might want to check out parts one, two, three and four before continuing here.
To some people it might seem that a lot of what I’ve written about here is common sense. The condensed version of this text might be “go talk to QE”. But surprisingly there are exceedingly few people that actually do it. (more…)