Vertical Scaling vs Horizontal Scaling

Vertical: (otherwise known as scaling up) means to add more hardware resources to the same machine, generally by adding more processors and memory.

• Expensive
• Easy to implement (generally, no change required in your application)
  
• Single point of failure (if main server crashes, what do you do?)
  

Horizontal: (otherwise known as scaling out, hence the name of this blog) means to add more machines into the mix, generally cheap commodity hardware (like that cheap computer sitting under your desk).

• Cheap(er) - at least more linear expenditures
  
• Hard to implement (much harder than vertical)
  
• Many points of failure and therefore can usually handle failures elegantly
  

Which begs me to ask the question, which one is actually more cost effective considering the engineering time required to make Horizontal Scaling work seamlessly?

And here is a wonderful video by Ted Cahall, CIO, CNET where he explains scaling out (horizontally) on a whiteboard with nice colorful markers.