Northern California CMG

Voting with our Feet

November, 2008
by Denise P. Kalm, Secretary, NCCMG

About the Author
Denise P. Kalm, CA, Inc. formerly Cybermation

Denise Kalm has 30 years experience in IT including application programming, enterprise systems management and performance management/capacity planning at Pacific Telephone and Bank of America. She moved to vendor land in 2000, spending 5 ½ years with BMC on the EPA product line, then recently became the senior product marketing manager for enterprise job scheduling products at CA, Inc., formerly Cybermation. She is a regional officer of CMG, has held many volunteer positions within that organization and is a frequent contributing author. Prior to entering the IT profession, she was a biochemical geneticist. Her hobbies include flying, Jazzercise, writing and scuba diving. Her book, Lifestorm, on the Oakland Hills fire, is available on Amazon. She is an executive and personal coach as well, offering phone and in-person coaching.

On Election Day, many of our NCCMG members gathered at the Compuware Corporation site in Pleasanton to vote in person on the critical election of NCCMG officers.

After a delicious breakfast provided by Compuware, Tom Halinski, Compuware, started our day with "Business Service Management - the End User, the Business and IT - Visualized." Tom is a frequent presenter, opting to enjoy the nicer weather of the Bay Area than the colder climes of Michigan. He is a great supporter of CMG, having served this year as SAC for CMG 2008. He describes his background as a major in mainframe performance, now extending to distributed systems, with a minor in DB2. Starting the talk with some levity, Tom noted that when he was transitioning from teaching to IT, he wondered if there was any future in computers.

The focus was on how we are doing application performance management (APM) now. He looked at how we get information on a problem, whether we got it before the user learned about it and then, what did we do to resolve it. Silo monitoring doesn't work - the cumulative effect of small problems in different silos can result in serious impact to the end user. What is needed is:

  1. Visibility into true service quality. For this you want to see both synthetic agents and agent-less probes, the former to give proactive indication of problems and the latter to show what is happening now.
  2. A service model so you can prioritize based on business impact. The service model defines the business and IT layers and how they map to each other. Tom showed a dashboard to help us see how this would look like. You need a view for executives, service management and operations. He also talked about Apdex specifications for service quality and what mainframe monitoring should do.
  3. Efficient troubleshooting and resolution
  4. Continuous improvement in quality

The 2007 Michelson Award winner, Adrian Cockcroft, Netflix, shared his upcoming 3-hour workshop material, "Performance Management with Free and Bundled Tools," in an information-packed hour. He created this material with Mario Jauvin. He presented a large list of tools to handle load generation, modeling, application, database and network monitoring, offering suggestions on the pros and cons with each. The challenges are: a changing infrastructure, too few people looking at performance management, scaled commodity systems, per node software costs too high, there are too many tools, there are too many agents and too much data to analyze. In some cases, like for virtualization, tools are lacking. And of course, as has been from the beginning, the challenge of projecting performance when the data is non-linear and scalability is non-intuitive. In an ideal world, all tools would be the same, producing data that everyone uses, but in practice, each area has its own tools and this can mean that the data doesn't even match.

Adrian also reviewed his new CHP - the Cockcroft Headroom Plot - an R-based plotting tool. Learn more at www.perfcap.blogspot.com. He showed a lot of response time versus throughput plots - this kind of plotting can highlight resource issues in a new way.

Lunch was provided by Compuware and we had time to network and enjoy a brief presentation by Compuware on their new branding and direction. Then, our business meeting was marked by election. A unanimous vote was entered to re-elect Keith McAndrews, Treasurer and Shennon Shen, Vice-Chairman. Returning Directors are: Bill Jouris, Mel Boksenbaum.

Bruce Spencer, IBM, a new NCCMGer, kicked off the after-lunch sessions with "IT Green Strategies or How You Can Save the World from Global Warming Without Trying Too Hard." Bruce has been at IBM for 28 years, starting out as an Engineer, then moved to the dark side of sales. He is now a Systems Architect, specializing in AIX tuning and performance. He noted that IT is very energy-intensive, so it is a great target when you are trying to reduce energy. In 2007, 180 B kw-hrs were used - this is expected to double in 4 years. By 2011, the cost of electrical will equal the hardware cost, making this a very important issue as companies strive to reduce their costs. But the costs are not just the power - you also have the environmentals, hardware and software - all of which can be reduced by power-reducing strategies. And in fact, it isn't just about cost - several places such as NYC and London have been told that they cannot increase their power demands - there just isn't anything to give.

It has been estimated that $1 saved on energy translates to $6-8 in operational savings. You can save on new construction, software licenses and also increase your business resiliency and agility. Bruce then moved onto strategies:

  1. Prevention
    • Use quality hardware designed for efficiency (especially power supplies). You need to look at throughput/watt to ensure you get the best hardware
    • Increase efficiency thru virtualization - raise % utilization of your resources
    • Eliminate redundancy - data deduplication can help a lot as can MAID (massive arrays of inactive disks) - a technology that can turn off disks when not in use.
    • Automation
  2. Mitigation
    • Concentrate heat
    • Cool at the source
    • Increase room temperature - it is now safe to do so
    • Recycle heat
    • Efficient management - such as turning off servers when not needed and exploiting tools like VMMotion
    • Centralize IT purchases to ensure that virtualization is favored

He also talked about virtualizing storage to reduce the number of disk drives and the amount of data redundancy. Also, relocating data to cost-effective storage units, like tape, can also help. Thin provisioning is a new strategy - you only allocate part of what is requested and then only provide what is actually needed.

Hugh Smith, BEC Consultants, presented his studies on "A Load Balance Index - the Missing Metric." He feels that there currently isn't anything that works to really assess how well load balancing tools are working for you. Is focus was on the parallel and homogeneous resources and he looked at hardware queuing. The problems with load balancing are:

  • Sub-optimum performance
  • Unavailable capacity
  • Lack of automation

Imbalance of utilization leads to poor response time averages. He then defined a calculation to help people look at the quality of their load balancing and its impact on response time. His metric results in a value that tells you what response time could have been if all had been balanced. This calculation can apply to networks, servers of all kinds and storage. He is just beginning this journey and is looking for data and validation from all of us.

Our next meeting is February 3, 2009 at Compuware. Save the date! And come see us at CMG 2008 in Las Vegas. Any questions or comments, please contact Cathy Nolan at ( or ).

Denise P. Kalm
Secretary, NCCMG