The Art of Capacity Planning: Scaling Web Resources in the Cloud

In their early days, Twitter, Flickr, Etsy, and many other companies experienced sudden spikes in activity that took their web services down in minutes. Today, determining how much capacity you need for handling traffic surges is still a common frustration of operations engineers and software developers. This hands-on guide provides the knowledge and tools you need to measure, deploy, and manage your web application infrastructure before you experience explosive growth.

In this thoroughly updated edition, authors Arun Kejariwal (MZ) and John Allspaw provide a systematic, robust, and practical approach to capacity planning--rather than theoretical models--based on their own experiences and those of many colleagues in the industry. They address the vast sea change in web operations, especially cloud computing.

• Understand issues that arise on heavily trafficked websites or mobile apps
• Explore how capacity fits into web/mobile app availability and performance
• Use tools for measuring and monitoring computer performance and usage
• Turn measurement data into robust forecasts and learn how trending fits into the planning process
• Examine related deployment concepts: installation, configuration, and management automation
• Learn how cloud autoscaling enables you to scale your app's capacity up or down

Arun Kejariwal (Twitter) is a software engineer at Twitter, where he works on research and development of novel techniques for time series analysis. He open sourced R packages - AnomalyDetection and BreakoutDetection - which have been widely used in a variety of domains such as, but not limited to, medicine, social ecology, sports. Prior to joining Twitter, Arun worked on research and development of practical and statistically rigorous methodologies to deliver high performance, availability, and scalability in large-scale distributed clusters. Some of the techniques he helped develop have been published in peer-reviewed international conferences and journals.

John Allspaw is currently Operations Engineering Manager at Flickr, the popular photo site. He has had extensive experience working with growing web sites since 1999. These include online news magazines Salon.com, InfoWorld.com, Macworld.com and social networking sites that experienced extreme growth (Friendster and Flickr). During his time at Friendster, traffic increased 5X. He was responsible for their transition from a couple dozen servers in a failing data center to over 400 machines across two data centers, and the complete redesign of the backing infrastructure. When he joined Flickr, they had 10 servers in a tiny data center in Vancouver; they are now located in multiple data centers across the US. Prior to his web experience, Allspaw worked in modeling and simulation as a mechanical engineer doing car crash simulations for the NHTSA.