Article 1:
Survey Effectiveness Index Construct

Article 2:
The PDi Survey Process

Article 3:
Taking Surveys to the Next Level

Article 5:
Employee Benefits

by Robert A. Page, Jr., Ph.D., Mark Spence, Ph.D. and Edward Tamson, Ph.D.

Contact PDi about these publications

(copyright 2002 by PDi. All rights reserved. Please do not modify, copy or distribute this article -- see website terms and conditions of use.)

Executive Summary

In March of 2002, PDi conducted a customer satisfaction survey of customers of the Cisco 6500 series router system. 65 Local Area Network and IT managers in firms with over 10 thousand employees participated. 45% of the respondents had purchased their 6500 series routers within the past 18 months.

Of the 65 survey respondents, 13 had experienced a Cisco 6500 router failure within the last six months. 23 of the 805 routers installed had hardware failures (25 incidents), software failures (2 incidents), or both, representing a failure rate of 2.9%. Given this figure only reflects the last six months, and does not take into account failures which occurred before that time limit, the overall failure rate is somewhat higher.

Two factors were identified in the statistical factor analysis: a Quality Category including questions on convenience, reliability, ease of integration and competent support staff, and a Service Category including questions on problem resolution, warranty programs, and speed of support. Customers gave high ratings (over 4.0 on a 5 point agreement scale) to all of the questions in both categories, regardless of whether or not they had experienced problems with Cisco routers.

As expected, Pearson Correlation Matrices revealed that both the quality and service categories had strong positive, statistically significant relationships with perceptions of overall value (.46, .47 respectively) and overall satisfaction (.61, .58 respectively). This means that when respondent ratings of quality and service increase, their ratings of overall value and satisfaction also tend to increase.

Surprisingly, customers who experienced problems with router performance rated their satisfaction higher than customers who never had a problem in two areas: warranty programs and the problem resolution process. Also, experiencing router failure did not affect customer loyalty, with 62 of 65 respondents indicating that if they needed to buy another router system, they would return to Cisco. Written comments indicated that decisions to leave were based on cost and features, not on product reliability considerations.

Written comments suggest that customer satisfaction and loyalty remain high because Cisco exceeds industry norms and customer expectations for router quality and support. When problems happen, Cisco's warranty and problem resolution processes also exceed industry norms and customer expectations, leaving customers pleasantly surprised. Ironically, unless they are experienced, excellent warranty programs and problem resolution processes will be undervalued, judged instead by general industry expectations and reputation. Extremely positive problem resolution experiences offset the frustration of product failure, leaving customer loyalty unaffected, or even strengthened.

Introduction

It is far less expensive to retain an existing customer than to attract new customers (Brown 2000). Dissatisfied customers can cause two deleterious effects: a lost customer with no further 'lifetime value'; and the spreading of negative word-of-mouth. Compared to firm-sponsored promotion media, consumers weight personal recommendations highly. A negative experience, if left unchecked, can create a negative chain reaction. TARP, an international consulting firm, estimates that 2% of customers who hear negative word-of-mouth are lost (Management Decision 1994), and Brown (2000) reports that dissatisfied customers tell 10 to 20 people of their experience, snowballing the impact of their dissatisfaction.

For this reason, the concept of 'defensive marketing' (Clark et al. 1992), and "service recovery and customer retention programs' (Lehman and Winer 2002) have gained popularity. Properly handled complaints mitigate negative word-of-mouth (Singh 1990). Further, it has been shown that quickly responding to, and satisfactorily resolving, customer complaints can increase loyalty (Gary et al. 1992; Lipton 2000) and generate positive returns (Management Decision 1994; Brown 2000). In a study commissioned by Avis involving 11 companies (Management Decision 1994), processing a complaint cost $26.85, yet the return on investment was 184 percent. Gary et al. (1992) report evidence suggesting that it costs $20 to keep a customer satisfied, while a lost customer reduces profit by $118.

Most of the research in this area examines how companies respond to customer complaints in business-to-consumer contexts (Peter and Shea 1997; Gary et al. 1992). However, many companies operate in a business-to-business environment, an environment characterized as having a fewer number of more sophisticated buyers. Further, resultant transactions are for reuse (material to which one will add value) or for on-going operations (such as office supplies or information technology). Insights gleaned from consumer contexts on how to respond to complaints, such as offering redeemable coupons, upgrades or discounts, may not apply. This paper explores the impact of complaint resolution in a specific business-to-business context: local area networks.

In a local area network (LAN) the most critical hardware components are routers, which allow users to communicate with other external computer networks. The dominant manufacturer in this product class is Cisco. This study explores customer satisfaction of LAN managers with one of Cisco's flagship products, the 6500 series router. Some of the LAN managers experienced product failures with the 6500, thereby allowing us to report the effect of product failure/no failure on customer perceptions of product quality, service support and ultimately loyalty. Our starting supposition is to guardedly assume that conclusions reached within the business-to-consumer market apply to business-to-business contexts, namely: those experiencing product failures will be less satisfied with the product, yet proper resolution of the problem/complaint can result in higher levels of satisfaction with the service support staff. Taken together, given superior problem resolution processes, there should be little if any negative effect on customer loyalty.

Sample

In March of 2002, a sample of convenience was obtained from a market research firm, which identified local area network (LAN) managers in firms with over 10 thousand employees in the United States who used Cisco router systems. 485 of these managers were contacted by phone, and 65 participated, completing the survey. This 13 percent response rate would have been considerably higher if the target sample had been expanded to include all classes of Cisco router systems, instead of focusing exclusively on the high-end 6500. Willing potential respondents were screened out if they did not meet this demanding criterion.

Respondents included some of the largest organizations in the US, including:

Private Sector	Public Sector
· Honeywell International · Warner Elektra Atlantic · Lockheed Martin · GMAC Insurance · Ernst and Young · State Farm Insurance · Kaiser Permanente · Transamerica · First Union National Bank · Bank One · Ford Motor Company · TRW · Time Warner Cable	· U.S. Postal Service · Environmental Protection Agency · Federal Aviation Administration · University of Chicago · State colleges and universities in Minnesota, Georgia, California, Iowa, and Kentucky · Government agencies in South Dakota, Colorado, Michigan, Montana, Minnesota, Mississippi, Idaho, Iowa, Maryland, Louisiana, New York, Kansas, Florida, and Utah

Of the respondents, 80% had installed less than 10 routers on their systems. Almost all of the public sector respondents are included in this group. Of the remaining respondents, most installed 10 to 50 routers on their systems. At the high end, one respondent had over 200 routers:

Together the 65 survey respondents used a total of 805 Cisco 6500 routers in their organizations. 13 organizations reported a total of 23 router failures in the last six months. Open-ended questions revealed these failures usually involved hardware problems (25 incidents) instead of software problems (2 incidents). Problems were usually resolved by adjusting or replacing a card or chassis (16 incidents). Re-engineering solutions involving software upgrades (7incidents) and workaround solutions involving reseating cards or cables and rebooting the device (6 incidents) were used less frequently.

Survey Methodology

Excluding contact information, this customer satisfaction survey used a total of 22 questions to explore perceptions of Cisco routers. All interval questions were adapted from pre-validated customer satisfaction questions provided by PDi, LLC. The interval scale featured 5 rating points ranging from strongly disagree to strongly agree, and included 2 non-response options. 5 open-ended questions gathered qualitative commentary on the quantitative ratings. Overall interval scale results are included in the appendix, along with the survey script, in its entirety. For the purpose of analysis, the respondent set is split into two groups: those who experienced router failure, and those who had not:

Cagtegories

Exploratory factor analysis was used to identify categories within the data. This involved a principal components analysis using an orthogonal (VARIMAX) rotation. Following generally accepted statistical standards, factors were retained if they had an Eigen value greater than 1.0. All questions neatly loaded on to two separate factors, with loads greater than .40. (Complete factor analysis printouts are available upon request). The two factors were retained because they were conceptually consistent. The quality category included questions on convenience, reliability, ease of integration and competent support staff, and the service category included questions on problem resolution, warranty programs, and speed of support.

Cronbach Alpha coefficients established the reliability of these categories. Both featured Alphas (.68, .73) over the reliability threshold of .60. (Reliability estimation printouts are available upon request).

Customer Satisfaction Results

Ratings of customer satisfaction were uniformly high - over 4.0 on a five point agreement scale for all questions and categories. Overall, customers agreed that Cisco 6500 routers provide superior value and strongly agreed that they were satisfied with their 6500 system. To Cisco's credit, ratings remained high even when respondents had experienced router failures within the last 6 months.

As expected, Pearson correlation coefficients revealed that both the quality and service categories had strong positive, statistically significant relationships with each other (.58), and with both outcome variables -- perceptions of overall value (.46, .47 respectively) and overall satisfaction (.61, .58 respectively). When respondent ratings of quality and service increase, ratings of overall value and satisfaction also tend to increase.

Failure Experiences and Customer Loyalty

As predicted, when router systems fail, those who experience the problems rate perceived quality, overall satisfaction, and overall value somewhat lower than those who have not experienced any problems. In a two-way analysis of variance, the perceived quality of Cisco routers significantly declined when respondents experienced router failure (F= 4.15; .046 significance; printout available upon request)

However, there was an unexpected trend in the findings. The service category scores did not decrease, as expected, they actually increased. Customers who experienced problems rated their satisfaction higher than customers who never had a problem. Examining the individual questions in this category, ratings increased on warranty programs (means of 4.25 versus 4.05) and the problem resolution process (means of 4.62 versus 4.36). While this difference is not statistically significant, this trend requires explanation. Why would respondents whose router systems failed give higher customer satisfaction ratings than those who experienced no problems?

Further, these failures seemed to have no effect on customer loyalty. 62 of 65 respondents indicated that if they needed to buy another router system, they would return to Cisco. Of the three respondents who indicated they might, or would, go elsewhere, only one of them experienced router failure. When they were asked why they were thinking of leaving Cisco, system failure was not a consideration. Their verbatim comments indicated that cost considerations and application limitations prompted their exit strategies.

The verbatim comments help to interpret these findings. Most of those who gave verbal comments (31 respondents) could not think of anything that needed to be improved with the 6500, except for the price. Of those who experienced problems, most noted that Cisco's warranty and problem resolution systems resolved the issue very quickly - so quickly and so thoroughly they were very impressed. One manager gave this representative comment: "Had a failure when first installed because of faulty software but they replaced it immediately, and it fixed the problem. Haven't had any other failures."

A few respondents had an unfavorable experience. When problems are complex and non-routine, the service process can break down. The customer support system sometimes allows sophisticated users to fall through the cracks, where they complain of inadequate service, arrogance, indifference, and useless or inaccurate information. One such respondent had a lot on his mind, and his comments are included verbatim:

There are a couple of things that are unfortunate about the design. The fact is that really the box was designed as a switch and not a router, until they came up with a consolidated IOS. It is two separate layers and that is confusing. The fact that layer 2 switching functions were developed by another company shows. It is not fully consistent with traditional Cisco product quality because of the disjoint between switching and routing functions. If you are just switching it is not an issue, if you need routing there may be a problem. Also, the person who wanted to use the 6500 also bought the intrusion detection module. That module is very poorly integrated into the system and software dependencies never successfully functioned together, despite trying to work with Cisco on it. The IDM, although purchased, is sitting unused, not plugged into 6500. They need to try to come up with a valid initial configuration (combination of hardware and software). Subsequent upgrades to software can be difficult. To make sure you have a valid configuration requires some time.

Implications

For both respondents who had experienced problems, and those who had not, there seems to be a mediating variable which colored their assessment of router system service - Industry Norms. Respondents who had not encountered problems were not rating their experiences with Cisco, because they never had occasion to use the warranty program, or the problem resolution process. Their ratings reflect their expectations of what service would have been like if they had used it. These ratings are based on industry norms of customer service, Cisco's reputation in relation to those norms, vicarious experiences of IT colleagues with Cisco, comparisons of warranty and service offerings with competitors, and other idiosyncratic criteria. Cisco receives high ratings because their reputation as an industry leader - verbatim comments indicate the 6500 is a top-of-the-line product, and everyone knows it.

Respondents who have encountered problems give lower ratings on overall satisfaction and value, and significantly lower ratings on perceived quality. However, because the 6500 still exceeds industry norms - problems notwithstanding - and because Cisco has a service and support program which exceeds industry norms for service and support, most customers remain loyal. These relationships are modeled in figure 1:

Conclusion

This research supports the supposition that conclusions reached within the business-to-consumer market apply to business-to-business contexts. LAN managers who experience product failures will be less satisfied with the 6500 router series, yet superior resolution of the problem/complaint can result in higher levels of satisfaction with the service support staff. Taken together, customer loyalty remains strong. Further, so long as service standards exceed industry norms, customers will tend to underestimate their satisfaction with problem resolution and warranty processes, until they are pleasantly surprised when they experience them.

References

Brown, Steven W., 2000, Practicing best-in-class Service Recovery, Marketing Management, summer.

Clark, Gary L., P.F. Kaminski and D.R. Rink, 1992, Consumer Complaints: Advice on How Companies Should Respond Based on an Empirical Study, Journal of Consumer Marketing, 9(3).

Lehman, Donald R., and Russell S. Winer, Product Management, 3rd edition, 2002, McGraw-Hill Irwin.

Lipton, David, 2000, Customer Complaints are Not Bad if Viewed as Business-Building Occasions, Nation's Restaurant News, 28.

Making Complaints Pay, Management Decision, 1994, 32(5).

Manickas, Peter A., and Linda J. Shea, 1997, Hotel Complaint Behavior and Resolution: A Content Analysis, Journal of Travel Research, 36(2).

Singh, Jagdip, 1990, Voice, Exit, and Negative Word-of-Mouth Behaviors: An Investigation Across Three Service Categories, Journal of Academy of Marketing Sciences, 18(1).

Contact PDi about these publications

(copyright 2002 by PDi. All rights reserved. Please do not modify, copy or distribute this article -- see website terms and conditions of use.)