Agile Software Testing With Spring Framework

Spring Framework has a great support for typical integration testing requirements of enterprise java applications.

We used this testing framework in a JEE project last year when I worked as the architect in the project. Using Spring Testing helped us a lot in not having to code the "mundane" tasks when using Spring, such as loading Spring Application Context, DI, Transaction Management in the test methods, JDBC helper methods to mention some.

I wrote an article (Software Testing With Spring Framework) on InfoQ about this support with a sample loan processing application.

With Spring getting more and more popular every day, are there any others who have used the test classes provided by Spring? If so, what has your experience been?

SpringSource's Adrian Colyer Details Spring in Production

Adrian Colyer from SpringSource (formerly called Interface21) hosted a webinar on "Spring In Production" topic three weeks ago. The presentation covered the topics on Spring Runtime Kernel architecture, how Spring supports enterprise services like transactions, data access, security, and messaging, and how to tune a Spring-powered application. A white paper on this presentation is now available for download from their website.

Read more on this post on InfoQ.

Spring is like BASF

I am a big fan on Spring Framework and I have been using it in various projects I have been the architect on for past few years. Its IOC container is just the beginning. AOP support and Template framework for JDBC, LDAP, JMS, JCA, JNDI, Security, OSGI, and even Integration Testing support makes Spring a powerfuly integration middleware framework no matter what other open source or commercial frameworks you are using in the technology stack of your application.

I always joke with my colleagues and friends that Spring is like BASF whose motto is "We don't make a lot of products you buy; we make a lot of products you buy better".

Same is the case with Spring Framework, it doesn't try to be everything to everybody. But the seamless integration of the framework into any other framework whether it's Hibernate, JPA, JMX , or dynamic scripting languages makes Spring's unofficial motto as "We don't make lot of the frameworks you use in your enterprise applications; we make lot of the frameworks you use in your enterprise applications way better!".

And the vision and roadmap SpringSource team is moving forward with in regards to Spring Framework is a proof that Spring is here to stay and it will dictate how Java EE middleware architecture will evolve in next few years.

Service Oriented Objects

One of my colleagues at work pointed me to the article by Bill Venners (Artima) on "Service oriented Objects". It's an interesting read.

Bill takes the statement Booch made on the objects that the "objects have state, behavior, and identity" and goes on to contrast different types of objects based on their state, behavior, and identity characteristics. I am copying a snippet from Bill's post below.

"Although in theory every object has state, behavior, and identity, in practice different object designs use state, behavior, and identity differently. Granted, most object designs I have encountered have had both interesting state and interesting behavior, as predicted by Booch's statement. I call this most common kind of object Service-Oriented. I have often encountered objects, however, that have little or no interesting behavior. These objects, which I call Messengers, are composed primarily of state. On the other hand, I have on occasion encountered objects that have little or no interesting state. These objects, dubbed Flyweights by the Design Patterns book, are composed primarily of behavior. Lastly, some objects are immutable, which means that once their state is established at the beginning of their lifetimes, the state never changes. Although every object does indeed have a unique identity, immutable objects are differentiated more often by value than by identity."

This article changed the way I have been thinking about Objects (mainly from the state and behavior aspects; not from the identity viewpoint).

I look forward to more articles from Bill on this topic.

Architecting For Performance And Scalability - Panel Discussion @ QCon

At QCon two week ago, I attended a panel discussion on topic "Architecting For Performance And Scalability" moderated by Floyd Marinescu. The panelists, in the order that they made the starting comments, were Cameron Purdy (Oracle), Ari Zilka (Terracotta), Randy Shoup (eBay), Peter Nickolov (3Tera), Brian Zimmer (Orbitz), and Nati Shalom (GigaSpaces).

Here are some notes I took during panel discussion:

Starting Comments:

Cameron:
We have to architect the systems to run on many servers right from the design phase.

Ari:
There are two types of applications:

• load balanced
• partitioned

These are contrasted based on scalability v. availability comparison.

Load Balanced: These apps have the high availability but they are hard to scale.

Partitioned: These applications are based on "divide and conquer" philosophy (examples: Google's MapReduce and Lucene's Hadoop). They are scalable but it's hard to make them highly available.

Randy:

• We have to make tradeoffs with manageability and cost.
• No system is 100% in all the above mentioned areas.
• At eBay, the resource tier is partitioned.
• They use 100 different logical databases.
• Application data is split among these partitioned databases.
• Application server knows which split or slice to go to get the data.
• And all this is abstracted from the client.

Brian:

• They have 4 tiers in the Orbitz application.
• Architecture should be based on the fact that the application will evolve in an "organic growth" fashion.

Peter:

• We should first get a system productized only then scalability makes sense.
• Enterprise Applications should be scalable in the following 3 areas:
• Application - code side (examples: MapReduce, AMP apps)
• Infrastructure (grid/utility computing)
• Architecture - backups, failover
  

Nati:

• Stateful applications are scalable linearly.
• He talked about 2 types of scalability:
• Data
• Operational
• One option for the scalability is to move database off to a separate layer
• Nati talked about end-to-end scalability v. performance. He said the scalability shouldn't be an after-thought in the software development process. We should include it right from the design and development stages.
• We can't solve the scalability at a silo level.
• Items to consider to communicate and collaborate with the development teams in the company are:
• Framework
• Guidelines
• Make scalability a common knowledge
  

The panelists then talked about Green Computing where data center space and data center power could become the problems. And Virtualization also helps in this area.

Questions from audience:
Q. RIA and AJAX based applications are more chatty - how do they affect the scalability?
If you can't split it, you can't scale it. There are two partition types at eBay: functional and horizontal. All the partitions are seen as a single logical transaction to the client. The architecture transition should be from one largest database to several service level databases. It is often much easier way to design this way than to use transaction in everything.

Other challenge is "State" (metadata around the conversation).

Ari talked about Open source Flash technology with the example of RedFive where they are pushing the streams thru Terracotta. He talked about static stream ala YouTube and real-time stream ala Webex. Change the way you think and look at the end-goal.

Q. How should the scalability be managed in non-AJAX applications which are more chatty?
Non-AJAX applications are very different than what it is in AJAX applications when it comes to the interaction with the server in terms of number of requests. We also have to deal with keep-alive issues in this scenario.

The panelists recommended that we make conscious decisions when it comes to scalability. One example is to measure the response time for a really simple server (aka null & void servlet) and see how close to zero seconds the response time for this servlet is. Also, good separation of concerns is very important.

Nati talked about SLA based computing and SPOF in a partitioned environment. He mentioned
master worker pattern (using spaces) and scatter gather pattern which decompose the task at hand and recompose after the task has been completely executed.

Ari explained the difference between a Hub & Spoke v. peer to peer architecture. He said we should analyze and understand the consumption patterns of the data and ask questions like "must you share everything everywhere?"

He said in partitioned applications: p-2-p and hub-spoke architectures are the same.

QCon - Trends in Agile Development by Kent Beck

I attended the keynote presentation by Kent Beck (Agitar Fellow) on "Trends in Agile Development" topic. This was the first session in QCon 2007 to officially kick-off the conference (there were tutorials on Monday and Tuesday but the actual conference started on Wednesday).

Kent started his presentation talking about the business trends in the current agile software development environment.

He listed the following 4 factors that are influencing these business trends in agile development.

• Accountability
• Responsibility
• Transparency
• Relationship

He said "Transparency is power" and keeping secrets in a software development project is a position of weakness. He also said the trend towards transparency continues.

Kent said that "off-shoring" is not a quest for lowering costs, but to find reliable off-shore business and technical partners and mentioned about the quality of projects implemented abroad.

Talking about software defects, he jokingly said that the bugs are "little things that crawl into code while we are asleep".

He also said that social evolution and new generation of business professionals are impacting the way we develop and manage the agile projects.

Kent compared how the sales people work, based on quotas v. sales and he said it should be the same for software development (IT) professionals as well where our performance is dependent on what we deliver with quality (sales) than what we commit to deliver (quotas).

The reliability of software is as important as the reliability of the software development process followed to develoop that software.

Speaking of the the users of agile methodologies, Kent said Agile methodologies are being used in large and small companies as well as large and small project teams.

He talked about two types of skills that are essential to excel in an agile development environment: Social and Technical Skills. He said social skills are more important than technical skills and suggested that every one should learn and improve the listening skills.

He also talked about "Appreciative attitude" and "Emotional Intelligence" as the important ingredients in a successful agaile development team.

Talking about Integrity, Kent gave an example of committing to new features in a project when the features already committed to deliver aren't even completed yet. For example, we shouldn't do things like promising additional features when the current commitments are not done, saying "I am not done but I will give you more later".

In Technical Skills category, he listed the following three items:

• Incremental Design
• Automated Testing
• Self Education

Kent mentioned that his new book "Implementation Patterns" is out (Read this book's review on InfoQ).

He suggested that we take a minute at the end of the daily stand-up meetings and appreciate on one good thing we did as a team. We should ask the following questions to stay successful in agile projects:

• What does your team do well and what you can do to do more of it?
• How could you use agile techniques to enhance your strengths?

Responding to a question from the audience, he said the geographic distribution has nothing or little to do with agility. Agility is an attitude and it can be learned and adapted in both co-located and geographically distributed project teams.

Kent also said we need to augment XP with Domain Driven Design (DDD) techniques and follow the usability practices at the same time. For small scale designs, we should use Domain Specific Language (DSL) based architectures.

He concluded the presentation by saying that designing or coding directly shouldn't be the question but what's more important is to ask questions like "what are our principles?" and "what business problems are we trying to solve in the project".

Kent listed some good resources on Agile methodologies and XP based software development:

• Article on Getting Started with XP (www.threeriversinstitute.org)
• Dale Emery’s web site
• XP Mailing lists
• Agitar dashboards
• Test-Driven Development: By Example
• Extreme Programming Explained, 2nd Edition
• Implementation Patterns

What I Liked About QCon 2007

Like I mentioned in the previous blog entry, I attended QCon 2007 last week. There are several things about this conference that I liked more than the other conferences I attended in the recent past including JavaOne. I thought I would list what I liked about QCon conference here.

• All the tracks were excellent covering the latest and real world practical business and technology areas such as Architecture Quality (Part 1 and Part 2), Architecting for Performance and Scalability, SOA (REST to be specific), Agile Software Development, Client Technologies, Emerging Technologies, Application Security, and .NET. They were as diverse as they were of quality. The other tracks on Ruby and "Architectures You Always wondered about" gave me a good introduction to the new and popular scripting language and learn the real-world experiences of the architects who have designed and have been managing the large scale websites respectively.

• Having the presenters from end-user companies like eBay, Yahoo, Orbitz, and LinkedIn gave the attendees a great perspective on the new trends in technology/design solutions these architects have come up with to tackle the scalability and latency architectural challenges in their projects. It was refreshing to hear they emphasize the "simplicity" in the design and architecture of the web applications instead of using heavy-weight technologies such as EJB, XA transactions etc.

• Speaker-To-Attendee ratio was higher compared to other major conferences, which really helped the attendees to interact with the speakers and panel members more closely than at the other conferences. I think this is what makes it worth to pay for and attend the IT/Java conferences rather than just sit there and stare at the slides for an hour, then go to the next presentation and do the same thing all over again.
  

• The 30-minute break between each session (instead of 15 minute break that other conferences have between sessions) made a big difference too. This gave the attendees a chance to talk to the presenter(s) of the session they just attended with any follow-up questions, catch up with other attendees on if and how they are using the technologies or frameworks that were just presented, get a cup of coffee, and still have time to decide which session to attend next.

• Technical Panels were very thought provoking. The first panel discussion I attended was on Architecting for Scalability and Performance on Wednesday morning. The panel was well represented by architects from both the vendor and the end-user companies. This made the discussions unbiased, no self-promotion, and more interesting and informative to the attendees.
  

• BOF on GigaSpaces on Thursday evening was great. I attended this BOF one hosted by Owen Taylor from GigaSpaces. Owen is a great teacher and he spent extra time helping us get the demo application running using GigaSpaces API. I am really thrilled to learn more about this technology. I read some articles about Space Based Architecture (SBA) before the conference, but this was my first time listening and learning more about it from the architects and developers who have founded this technology.

• The Keynote Panel (and the last presentation of the conference) on Friday evening was innovative and interesting. This was hosted by none other than Martin Fowler. The panelists were the hosts of the technical tracks in the conference. It was interesting because this panel discussion was kind of to summarize the whole conference in about an hour and at the same time talk about what's coming in the Java/.NET/Agile/SOA areas.

Floyd and InfoQ team (along with JAOO group) did a great job in coordinating the conference which was a great experience to be part of. I look forward to attending the next QCon conference.

QCon 2007 in San Francisco

I was at the QCon conference in San Francisco last week. It was a great experience to be there. I learned a lot not only from the presentation speakers and panelists but also from the attendees who came from different countries (England, Syria, Australia to name a few) and companies. Thanks to Floyd and the InfoQ team, the conference was a great success. I met Eric Evans (Domain Driven Design), Nati Shalom and Owen Taylor (from GigaSpaces), Ari Zilka and Jonas Boner (Terracotta), Dan Pritchett and Randy Shoup (eBay), Brian Zimmer (Orbitz) to name a few distinguished presenters at the conference. I will be posting more about the sessions I attended during the conference in next few days.

First message on the blog

I just joined this blog site.