Ginneblog

Christa Stelzmuller, Chief Data Architect at MySpace.com, spoke Wednesday night to the San Francisco SQL Server User Group about the MySpace Service Broker.  Last summer, Christa spoke to the Silicon Valley SQL Server User Group about the MySpace Data Architecture.  MySpace is an amazing example of what can be done with SQL Server.

Christa started her presentation with a description of Service Broker, and the challenges they faced creating it.  She then covered basic features, advanced features, and the major use cases.  She concluded with a roadmap of their continuing development plans, and some fun examples of how their developers have sometimes used Service Broker to solve their problems in somewhat misguided ways.

Keep an eye out on CodePlex, where her team will be posting their work.  We’ll get a chance to speak more with Christa in early November at the PASS Community Summit in Seattle.

 Send to Twitter

MySpace.com uses SQL Server in a big way. On Tuesday night MySpace Chief Data Architect Christa Stelzmuller spoke to the Silicon Valley SQL Server User Group in Mountain View. We had a record turnout. This was a rare opportunity to learn how a high profile company is using SQL Server to manage very large data.  And I mean large – think 130 million active users a month!

It’s pretty well known that MySpace.com started out as a two-tier system. They used ColdFusion on the front-end, and SQL Server at the back-end.  Traffic grew radically, and the technical team scrambled to adapt. Over the years, the technology has matured, but we’re talking about big data, heavy traffic, and continued rapid growth. 

Christa Stelzmuller and me at Microsoft

Now ColdFusion is gone, replaced by C# and ASP.NET. They added a middle tier, and are running mainly on SQL Server 2005, Standard Edition, with a few instances of Enterprise where required.  They have about 4 petabytes of disk space, spread across 17,000+ disks.  You can read more about the specifics in this MySpace Microsoft Case Study.

That volume of data pushes the database hard – and in some cases, beyond what SQL Server can handle out of the box.  Load during replication was so high that they had to write their own replication mechanism.  Likewise for many other processes. The load also impacts the development, testing, release, and backup routines. According to Christa, they literally invented their own processes and tools, as they are in uncharted territory.

Despite continued growth, MySpace is making real technical progress. For instance, when Christa joined the team from Yahoo 2.5 years ago, they were experiencing more than 2 million data integrity errors per day. Now that’s down to about 100,000 per day. My hat goes off to the MySpace engineering team!

The audience was so engaged that an extended Q&A that broke out in the middle of the presentation. Christa fielded dozens of questions, ranging from hardware configurations to backup strategies, and then finished off her presentation. You can check out Christa’s slides here.

Christa will speak to the San Francisco SQL Server User Group on October 14, 2009 when her topic will be Service Dispatcher: The MySpace Implementation of Service Broker, and I expect we’ll see another record turnout.

 Send to Twitter