How To Choose Application Performance Management Tools

Customers frequently ask us, “What tools do I need for effective application performance management?”

The right set of tools can be a catalyst to increase your staff’s productivity and reduce the time required for problem identification and resolution. The wrong tools can waste your time and money and potentially lead you down an unproductive path.

Before you shop, you should consider the following points.

One Size Does Not Fit All

Clearly define the problem set that you are trying to address and identify any application performance products, which you already have. This will help you understand whether additional tools are warranted, and whether a niche solution or a product suite is most appropriate for your needs.

Be cognizant that product suites contain a variety of tools, some of which will be significantly better for your purposes than others.

Although these suites are often attractive offerings because the tools appear to work together seamlessly, they frequently are made up of acquired software that has been integrated in name only. Often, it is better to purchase a niche product even if you have a competing tool within a product suite. In most cases, a robust product suite plus a few best-in-class niche solutions provides the most cost-effective and usable solution.

Be Honest About Your Organization’s Maturity Level

All households make good use of just a few basic tools: a hammer, a set of screwdrivers, a pair of pliers. A minor “splurge” of a few dollars can get you a couple of productivity-enhancing gadgets such as a power drill or an electric screwdriver. Unless you are building furniture, however, a US$4,000 lathe will be a waste of money.

It is important to understand your existing tool inventory. Ask yourself these questions:

• Are you using your existing tools to your best advantage?
• Are you missing basic tools and toolsets but have expensive, complex tools?
• Are you having problems for which your current toolset can’t provide insight?

If you answered yes to any of these questions, you probably have the wrong set of tools for the maturity of your organization. Often, it is to your detriment to purchase a sophisticated solution if you do not have the staff or expertise to make full use of the tool.

Use Your Performance Management Products

A tool left idle is more damaging to your company than no tool at all. We find expensive unused software products in every shop in which we work.

The hard dollars spent on software acquisition, implementation and training are substantial; not only could that capital be used on other projects, but the manpower effort to evaluate, select and fund the software has a real cost as well.

Frequently, a product is installed and reports are generated but no one is interpreting the reports. Remember, buying a tool is a waste of money if the learning curve is too long and you do not have experienced staff to use it and a well-defined problem to be addressed.

Beware of Performance Management Product Misuse

A tool misused can be even more damaging than one unused. Misuse will result in inaccurate predictions and ineffectual recommendations.

Some more sophisticated performance management products require broad performance expertise, specific skills in the tool and intimate knowledge of the application.

Performance management modeling packages require the analyst to build a framework that is a simplified representation of the application.

If a key piece of application is missing from the model, the model still can be calibrated and produce what appears to be highly accurate results — which in reality are totally erroneous.

Different Architectures Require Different Tools

It is important to understand your environment, including all tiers and layers of the infrastructure and application. This includes the software infrastructure, whether it is Apache or IIS, WebLogic or WebSphere, Oracle or Sybase.

Do not assume all tools will provide the same value or be equally effective for all environments. Browser-based architectures require different products than legacy client/server apps. It is often necessary to use two different products to accomplish the same function on two different architectures.

One of Keynote’s customers, for example, has a group of Web sites supported by a content management application. Although they use synthetic transactions to monitor their Web sites, it is not practical to use them to represent the user experience on the content management site, since they do not want to change the content. In this case, finding a real user monitor is required to accomplish the same function.

Retire Tools When They No Longer Provide Significant Value

Most tools have a limited lifespan. Either the vendor fails to keep the software in step with the changing technology world, or the problem set it was initially designed to address is no longer a significant issue.

I have a client that has a server monitoring tool that merely five years ago was the hot product but is now considered outdated. The tool’s vendor failed to address its cumbersome agent management issues, but still its maintenance fees remain costly. So my client is now replacing this monitor with newer, less expensive, and easier-to-manage solutions that perform additional functions.

Ultimate Tool Is the Analyst

Great tools do not make great carpenters. Problem-solving requires procedures, experience, facility and action. A performance management tool will do nothing by itself; you must supply the analyst.

The analyst is the human component, which correlates data between tools and ties an event to a root cause.