SharePoint Data Storage: Beam Me Up Scotty
By default, when you upload a document or any other large file to SharePoint, it gets stored as a Binary Large OBject (BLOB) in the content database in SQL Server. As revisions are made, each version of that file also gets stored (not just the differences). The amount of BLOB data grows significantly faster than associated metadata, causing SharePoint to consume large amounts of expensive SQL Storage space. Burzin talked about externalizing BLOB storage, as well as options for storing infrequently used BLOBs in the Cloud. These approaches can help ease the backup and storage cost problems content-heavy SharePoint sites encounter.
Burzin’s SharePoint Storage Best Practices talk also covered Configuration, Maintenance, and Performance Tuning. He explained some of the unusual stresses SharePoint puts on SQL Server, and offered suggestions on how to avoid degraded performance. If you’re planning a significant SharePoint implementation, you’ll want to take a close look at his specific recommendations regarding recommended I/O Capacities, Database configuration and sizing, processors and memory.
Given the headaches SharePoint BLOBs cause in many organizations, it makes sense that StorSimple has a complete solution to externalize them. Their storage-on-demand appliance provides tiered storage for SharePoint with the option to secure and store infrequently updated BLOBs to the cloud to achieve substantial cost savings. According to Ursheet Parikh, StorSimple’s Founder and CEO, Burzin’s extensive SQL Server and SharePoint experience make him a key member of the StorSimple team.
I’ll write about StorSimple’s product in an upcoming post, and will follow that with a case study once DesignMind has had a chance to implement StoreSimple’s Cloud Storage Solution for one of our clients. For data storage, Space is the Final Frontier.
Sphere: Related Content1 comment
Solid State Drives – You’ve Come a Long Way Baby
At the November 2009 PASS Summit in Seattle, one of the outstanding keynote presentations was by Dr. Dave DeWitt, Microsoft Fellow, and leader of the Microsoft Jim Gray Systems Lab, in Madison, WI. I received a copy of his slide deck from PASS Headquarters, which you can see below.
Dr. DeWitt is working on releases 1 and 2 of SQL Server Parallel Database Warehouse. In his keynote he reviewed the 30 year history of CPU, memory, and disk performance. Variations in performance gains across these subsystems, with disk performance lagging badly, have major impacts on database system performance.
Disk performance gains have been made in three areas, Capacity, Transfer Rate, and Average Seek Time. However, the gains over the last 30 years have not been uniform.
Capacity of high performance disk drives has increased by a factor of 10,000. Transfer rates have increased by a factor of 65. The average seek time has only increased by a factor of 10. Dr. DeWitt talked about the impact of these discrepancies on OLTP and Data Warehouse applications.
One of his conclusions is that some problems can be fixed through smarter software, but that “SSDs provide the only real help.”
We learned more about SSD’s during the Fusion-io presentation to the Silicon Valley SQL Server User Group. The DesignMind team has also been evaluating SSDs to determine situations where we can provide our clients with the most leverage. Plus here’s a terrific video which shows SSD’s in action.
Sphere: Related Content
No comments
Holiday Gadgets 2009
Geek extraordinaire Fred Davis shared the coolest, wackiest gadgets of 2009 at SofTech’s 5th Annual Gadget Night. Fred is Co-Founder and CTO of Grabbit, and an award-winning tech journalist, entrepreneur, and gadget enthusiast. His past lives included stints at Ziff-Davis, Wired, and CNET. Fred’s annual presentations to SofTech are among my favorites. He was accompanied by his fiancé, Lisa Padilla, host of Lisacast.com.
Joining me during last night’s introductions were Suzanne Skyvara, of Women in Consulting’s North Bay Chapter, and Harry Chapman of the Bay Area Consultants Network, as both of these organizations helped host the event.
Fred talked about cell phones (iPhone, Palm Pre, and Motorola Droid), eBook Readers (Kindle 2 and DX, and rumors of Apple’s upcoming entry into this space), netbooks, HD TVs and associated products, digital still and video cameras, and other miscellaneous gadgets.
The Duracell Instant Power Charger, at $15.99, would be a great stocking stuffer. If you have a lot of hungry mouths to feed, ask Santa for a Chefstack Automatic Pancake Maker, which can crank out 200 pancakes per hour. Just the thing for those 6th grade sleepovers!
And for the tech fan who has (almost) everything, there’s also the Husqvarna Automower (a Roomba on steroids!)
Sphere: Related ContentNo comments
Solid State Storage on Steriods – See It For Yourself
I just had to share this terrific demo of the ioDrive in action. Here’s Father Robert Ballecer interviewing Fusion-io President and CTO David Flynn.
Father Robert is a Jesuit Priest. The video was produced by Tech Stop at the Center for Apostolic Technology, headquartered in San Jose, California. How cool is that?
Sumeet Bansal of Fusion-io will speak to the San Francisco SQL Server User Group on November 11, 2009. We’ll be at the Microsoft office on Market Street in downtown San Francisco. Please consider joining us.
You can see more from Father Robert on the Gadget You Tube channel.
Sphere: Related Content2 comments
Solid State Drives: SQL Server 2008 at the Speed of Light
Perhaps you’ve heard this astonishing figure. At Amazon.com, every 100 ms of latency costs the company 1% in sales. Sumeet Bansal, Principal Solutions Architect at Fusion-io, referenced the Amazon study as he separated myth from reality about Solid State Storage and its role in the modern Database enterprise system.
Fusion-io is a producer of enterprise-class SSDs. Their silicon-based storage architecture known as ioMemory applies flash memory to large-scale enterprise storage products like Storage Area Networks.
Sumeet’s presentation to the Silicon Valley SQL Server User Group on Tuesday included a variety of reasons why SQL Server DBAs, as well as other technologists, should be excited about SSD’s. He stressed that SSD’s are ready for the enterprise today. His description of the things to look out for when purchasing SSD’s was quite useful – kind of like the Consumer Reports of SSD’s.
This list of differentiators is probably part of why Steve Wozniak joined Fusion-io as Chief Scientist – “the Woz” wouldn’t put his energies into anything but the most promising technologies. And neither would Sumeet, who came over to Fusion-io from Wine.com, where he was VP of IT at the San Francisco company.
David Leston walked away happy, and probably stayed up very late installing his blazingly fast 320 GB Fusion-io Solid State Drive (SSD). He won the coveted door prize, which was generously donated by Fusion-io.
I hadn’t met David before tonight, but he was on the same wavelength as our speaker. Sumeet’s discussion of SSD’s pointed out that you don’t purchase SSD’s based on cost per GB, but rather by the value of the performance gains and reliability.
It was particularly interesting to hear comments from the audience about how Microsoft and other vendors will start optimizing performance based on SSD-equipped systems, in addition to conventional drives. Right now there’s an assumption of significant latency when going to the drive. As the operating systems see great reductions in latency, additional optimizations will add to the performance gains of this breakthrough technology.
Sphere: Related Content1 comment
MySpace: SQL Server at its Best
Christa Stelzmuller, Chief Data Architect at MySpace.com, spoke Wednesday night to the San Francisco SQL Server User Group about the MySpace Service Broker. Last summer, Christa spoke to the Silicon Valley SQL Server User Group about the MySpace Data Architecture. MySpace is an amazing example of what can be done with SQL Server.
Christa started her presentation with a description of Service Broker, and the challenges they faced creating it. She then covered basic features, advanced features, and the major use cases. She concluded with a roadmap of their continuing development plans, and some fun examples of how their developers have sometimes used Service Broker to solve their problems in somewhat misguided ways.
Keep an eye out on CodePlex, where her team will be posting their work. We’ll get a chance to speak more with Christa in early November at the PASS Community Summit in Seattle.
Sphere: Related ContentNo comments
Making Software Development Hum
Ron Lichty spoke to SofTech recently. His presentation, Making Your Software Development Hum, contained a lot of useful information for software professionals. Since I manage DesignMind’s custom software development and database teams, I found this topic particularly interesting and valuable.
Ron learned his trade at companies like Apple, Schwab, Berkeley Systems, and Avenue A/Razorfish. He’s Co-chair of SDForum’s Emerging Technology and Software Development Best Practices Special Interest Group. I’ve known Ron since the dot-com boom, and over the years have seen him leverage his large development community following to raise a lot of money for charity, an effort I’ve always admired.
Software development is difficult to be good at individually, and even harder to perfect as a team. This is a discipline where you have to deliver something that works, and if you don’t, it’s obvious. What’s not so obvious, though, is whether you got the job done as well as you could have. It’s worthwhile to look over Ron’s slides and see his suggestions for making your software development efforts more productive and enjoyable.
If you missed Ron’s presentation to SofTech, he’ll be presenting to the East Bay Innovation Group on October 7, 2009.
You can learn more about SofTech’s upcoming meetings here. SofTech is a San Francisco Bay Area business and technology networking group.
Sphere: Related ContentNo comments
Think Like a Hacker
Think Like a Hacker really got me thinking! Sudha Iyer of LogLogic and Slavik Markovich of Sentrigo spoke to the Silicon Valley SQL Server User Group on how hackers attack databases, and what can be done to reduce their likelihood of success. Sudha gave a good overview of the threats, and about some unfortunate organizations who were vulnerable. For instance, Heartland Payment Systems had a breach exposing 130 million credit and debit cards! How was it done? SQL Injection Attacks.
There are some obvious best practices that should be implemented. Remember the SQL-Slammer worm? Microsoft had closed that vulnerability, but many thousands of servers had not been properly patched.
Slavik talked about basic hacking techniques, ranging from brute force password cracking, and to SQL Injection. He walked us through different forms of SQL Injection attacks, culminating with complete control of an admittedly vulnerable server (as many are). Slavik talked about best practices for securing SQL Server, many of which apply to Oracle, DB2, MySQL, and other databases.
The amount of database talent we have here in the San Francisco Bay Area is remarkable and I can guarantee we’ll be thinking more like hackers at DesignMind. Thanks again to Sudha, Director of Product Management at Loglogic, and Slavik, CTO at Sentrigo, for a fantastic presentation. You can follow Slavik’s Database Security Blog here.
No comments
Fusion-io: Can you say Super Speedy Database?
Fusion-io is bringing a whole new meaning to SSD – Super Speedy Database.
Last week I had lunch with Sumeet Bansal, Fusion-io’s Principal Solutions Architect. Fusion-io is a producer of enterprise-class SSDs (Solid State Drives, as opposed to HDDs – Hard Disk Drives). Fusion-io applies flash memory to large-scale enterprise storage products like Storage Area Networks. I first met Sumeet when he was VP of IT at Wine.com.
Part of our discussion was about their products, and how they differ from the competition. SSDs can be very fast. They can also be pretty rugged, as they have no moving parts. They are more expensive per GB than conventional hard drives – at least for now. SSDs eventually wear out – each spot on the NAND Flash Chip can be exercised a finite number of times before it stops working.
Key points regarding Fusion-io’s ioDrive:
Performance of NAND Flash Chips is extremely fast. Read latency is an order of magnitude faster than conventional HDDs. But the Fusion-io ioDrive removes the IO bottleneck and makes the cpu utilization more efficient. Fusion-io decided to adopt a PCIe architecture rather than dealing with the multiple layers of protocols necessary to implement a SATA interface. I reviewed performance results at HotHardWare, and it’s clear that they are getting a big performance advantage.
They have the wear under control. Their controller spreads the “wear” across the entire drive, giving them excellent life (1.2 million hours MTBF!)
The rest of our discussion was around how the ioDrive could be used most effectively on different platforms we work with a lot at DesignMind. There is work to be done regarding Best Practices for implementing SSD technology for SQL Server, SharePoint, Exchange, and other key Microsoft Server tools. We look forward to working with Fusion-io to help them develop Best Practices, Performance Benchmarks, and Case Studies.
Sphere: Related Content6 comments
SharePoint: The Ultimate Business Timesaver

Montreal, Quebec
I wish I were attending the SharePoint Summit conference in Montreal this week. We use it at DesignMind. It really enhances our collaboration and content management processes. It’s no wonder that Forbes Magazine calls SharePoint the “Ultimate Business Timesaver”.
According to InfoWorld, MOSS (Microsoft Office SharePoint Server) 2007 is the fastest growing product in Microsoft’s history and has “as many uses as a Swiss Army knife”. Its six focus areas are collaboration, portal, search, ECM (enterprise content management), business process management, and business intelligence.
Ford Motor uses SharePoint for its dealer portal, and the Marines have deployed collaborative applications to aid their efforts in Iraq. In Scotland, the entire K-12 education community is connected via SharePoint.
The great thing about SharePoint is that it offers a single environment for all your information and collaboration tasks and it’s easy to learn, use, and personalize. It has a single platform and well-integrated set of technologies to manage and lower user support requirements. Can you tell I’m a big fan?
I’m also a fan of Formula One racing, so the following bit of news made me particularly happy. If you like fast cars check out the new Ferrari.com site which was built with MOSS 2007.
Sphere: Related ContentNo comments
SQL Server 2008 Migration Workshop – The Buzz
We had a full house at last week’s SQL Server 2008 Migration Workshop with 55 folks in attendance. The event was at the Microsoft office in San Francisco where Alex Viera was our host. John Hanson of DesignMind and well-known SQL Server author, and Chief Data Architect at Autodesk, Paul Bertucci led the 3-hour workshop, which included a walk through the steps of a successful migration.
The crowd at the workshop included folks from AXA Rosenberg, Chevron, Clicktime, Intuit, Ninth Circuit Court of Appeals, Onyx Pharmaceuticals, Opentable.com, Symantec, Shop.com, Sun Maid Growers, UCSF Medical Center, Varian, and Visa.
You can take a look at the slides from John and Paul’s presentations on SlideShare.
- Migrating to SQL Server 2008 (John R. Hanson)
- Migrating to 2008 (Paul Bertucci)
Some of the areas John and Paul covered were:
-
How to reduce database costs through consolidation
-
Virtualizing servers
-
Compressing data
-
Allocating resources more effectively
-
Effective policy management
According to Jeff Van Vliet , “the most valuable thing I learned was information on deprecated key words and existence of lists and support from Microsoft on the deprecated items for each new release”. Also, “I liked the fact that they spent time talking about what also worked in 2005, as most of my clients are still using SQL 2005 (one major client just migrated from SQL 2000 to SQL 2005). If my clients are indicative of average enterprise level clients, they lag by a version or so, typically.”
Rick Griest of Systron liked, “the complexity of searching through legacy code looking for all SQL calls. It became obvious that it would be easy to overlook an entire block of calls if they were in an unusual format. Also, the talk time during intermission was pretty exciting for me and I wouldn’t have minded possibly a little more”. Rick also suggested that Anchor Steam would be a welcome addition to any future workshops…
Sphere: Related ContentNo comments
MySpace Data Architecture: Hello Large Data
MySpace.com uses SQL Server in a big way. On Tuesday night MySpace Chief Data Architect Christa Stelzmuller spoke to the Silicon Valley SQL Server User Group in Mountain View. We had a record turnout. This was a rare opportunity to learn how a high profile company is using SQL Server to manage very large data. And I mean large – think 130 million active users a month!
It’s pretty well known that MySpace.com started out as a two-tier system. They used ColdFusion on the front-end, and SQL Server at the back-end. Traffic grew radically, and the technical team scrambled to adapt. Over the years, the technology has matured, but we’re talking about big data, heavy traffic, and continued rapid growth.
Now ColdFusion is gone, replaced by C# and ASP.NET. They added a middle tier, and are running mainly on SQL Server 2005, Standard Edition, with a few instances of Enterprise where required. They have about 4 petabytes of disk space, spread across 17,000+ disks. You can read more about the specifics in this MySpace Microsoft Case Study.
That volume of data pushes the database hard – and in some cases, beyond what SQL Server can handle out of the box. Load during replication was so high that they had to write their own replication mechanism. Likewise for many other processes. The load also impacts the development, testing, release, and backup routines. According to Christa, they literally invented their own processes and tools, as they are in uncharted territory.
Despite continued growth, MySpace is making real technical progress. For instance, when Christa joined the team from Yahoo 2.5 years ago, they were experiencing more than 2 million data integrity errors per day. Now that’s down to about 100,000 per day. My hat goes off to the MySpace engineering team!
The audience was so engaged that an extended Q&A that broke out in the middle of the presentation. Christa fielded dozens of questions, ranging from hardware configurations to backup strategies, and then finished off her presentation. You can check out Christa’s slides here.
Christa will speak to the San Francisco SQL Server User Group on October 14, 2009 when her topic will be Service Dispatcher: The MySpace Implementation of Service Broker, and I expect we’ll see another record turnout.
Sphere: Related Content2 comments
SharePoint 2007 Feature Comparison
A prospective DesignMind client in San Francisco is making modest use of SharePoint 2003, and is considering a move to MOSS 2007. The reasoning is good. They can do an incremental rollout, and gain immediate value from SharePoint’s ability to index documents, including PDFs.
In preparation for the meeting, I had to refresh my memory in two very important areas. First, I needed a Comparison of the features of SharePoint 2003 with those of MOSS 2007.
The second major question to answer was which features are included in each version of SharePoint?
Here’s another useful site Comparing WSS and MOSS.
The last gem I uncovered was an excellent PowerPoint file covering the Functionality in MOSS 2007 and Office 2007. This is a great resource for those thinking about leveraging the integration Microsoft has delivered in these two major toolsets. It will help us make the right recommendations to our clients when it comes to purchasing Standard vs. Enterprise Client Access Licenses.
No doubt there’s a ton of information to sift through on MOSS. Next I’ll look forward to reviewing the comparisons between MOSS 2007 and SharePoint 2010. I must say I’m pleased that Microsoft is making our lives easier by by shifting back to the name SharePoint rather than MOSS…read my June 3rd Ginneblog post for more on the rollout of SharePoint 2010.
Sphere: Related Content2 comments
Balsamiq: Can You See It Now?
Wouldn’t it be great to sit down next to your client and sketch out ideas? Now there’s an online, user-friendly tool called Balsamiq Mockups that allows you to do that remotely.
Balsamiq Mockups helps you and your team or clients iterate on wireframes as early in the process as possible, when it’s cheapest to do so. Not surprisingly, it was developed by a former Macromedia/Adobe software engineer.
Balsamiq Mockups hits the bulls-eye for firms like DesignMind. We can use this inexpensive tool to rapidly create mock-ups of both desktop and web applications. It’s got a lot of power, and is easy to learn. One of the really great ideas in this tool is that the prototypes look hand-drawn. This informal appearance makes stakeholders more comfortable about giving the developers early feedback, which is so incredibly valuable.
I learned about Balsamiq Mockups last week when Don Robins spoke to the San Francisco .NET User Group. (Don, a principal at Outformations, gave a great overview of SQL Server Compact Edition.)
Software Application Developers have long struggled with ways to help their clients visualize what an application will look like, early in the project. These days developers communicate their user interface designs in many ways, all the way from whiteboard sketches, Visio diagrams, Excel, and at the high end, complex software visualization systems such as iRise.
If you aren’t doing prototyping, or if you’re using the paper, Visio, or Excel approach, you can elevate your development process easily with this product. Thanks to Don for spending some of his presentation demonstrating the Balsamiq tool.
Sphere: Related Content
1 comment














