Architecting For Performance And Scalability - Panel Discussion @ QCon

At QCon two week ago, I attended a panel discussion on topic "Architecting For Performance And Scalability" moderated by Floyd Marinescu. The panelists, in the order that they made the starting comments, were Cameron Purdy (Oracle), Ari Zilka (Terracotta), Randy Shoup (eBay), Peter Nickolov (3Tera), Brian Zimmer (Orbitz), and Nati Shalom (GigaSpaces).

Here are some notes I took during panel discussion:

Starting Comments:

Cameron:
We have to architect the systems to run on many servers right from the design phase.

Ari:
There are two types of applications:

• load balanced
• partitioned

These are contrasted based on scalability v. availability comparison.

Load Balanced: These apps have the high availability but they are hard to scale.

Partitioned: These applications are based on "divide and conquer" philosophy (examples: Google's MapReduce and Lucene's Hadoop). They are scalable but it's hard to make them highly available.

Randy:

• We have to make tradeoffs with manageability and cost.
• No system is 100% in all the above mentioned areas.
• At eBay, the resource tier is partitioned.
• They use 100 different logical databases.
• Application data is split among these partitioned databases.
• Application server knows which split or slice to go to get the data.
• And all this is abstracted from the client.

Brian:

• They have 4 tiers in the Orbitz application.
• Architecture should be based on the fact that the application will evolve in an "organic growth" fashion.

Peter:

• We should first get a system productized only then scalability makes sense.
• Enterprise Applications should be scalable in the following 3 areas:
• Application - code side (examples: MapReduce, AMP apps)
• Infrastructure (grid/utility computing)
• Architecture - backups, failover
  

Nati:

• Stateful applications are scalable linearly.
• He talked about 2 types of scalability:
• Data
• Operational
• One option for the scalability is to move database off to a separate layer
• Nati talked about end-to-end scalability v. performance. He said the scalability shouldn't be an after-thought in the software development process. We should include it right from the design and development stages.
• We can't solve the scalability at a silo level.
• Items to consider to communicate and collaborate with the development teams in the company are:
• Framework
• Guidelines
• Make scalability a common knowledge
  

The panelists then talked about Green Computing where data center space and data center power could become the problems. And Virtualization also helps in this area.

Questions from audience:
Q. RIA and AJAX based applications are more chatty - how do they affect the scalability?
If you can't split it, you can't scale it. There are two partition types at eBay: functional and horizontal. All the partitions are seen as a single logical transaction to the client. The architecture transition should be from one largest database to several service level databases. It is often much easier way to design this way than to use transaction in everything.

Other challenge is "State" (metadata around the conversation).

Ari talked about Open source Flash technology with the example of RedFive where they are pushing the streams thru Terracotta. He talked about static stream ala YouTube and real-time stream ala Webex. Change the way you think and look at the end-goal.

Q. How should the scalability be managed in non-AJAX applications which are more chatty?
Non-AJAX applications are very different than what it is in AJAX applications when it comes to the interaction with the server in terms of number of requests. We also have to deal with keep-alive issues in this scenario.

The panelists recommended that we make conscious decisions when it comes to scalability. One example is to measure the response time for a really simple server (aka null & void servlet) and see how close to zero seconds the response time for this servlet is. Also, good separation of concerns is very important.

Nati talked about SLA based computing and SPOF in a partitioned environment. He mentioned
master worker pattern (using spaces) and scatter gather pattern which decompose the task at hand and recompose after the task has been completely executed.

Ari explained the difference between a Hub & Spoke v. peer to peer architecture. He said we should analyze and understand the consumption patterns of the data and ask questions like "must you share everything everywhere?"

He said in partitioned applications: p-2-p and hub-spoke architectures are the same.