典型的測試錯誤有哪些？(20)

　　困難的問題是很難知道你是否圓滿地完成了測試工作?？赡苤挥挟敭a品已交付后才能真正知道?？梢岳斫獾氖?，這使得經理們不舒服。有時候你會發現他們熱心采用代碼覆蓋率，認為只有那些簡單的數字可以鼓舞士氣。候測試員也變得傾心于覆蓋率，雖然他們的興趣沒有那么高，而且結束得也快。

　　What is code coverage? It is any of a number of measures of how thoroughly code is exercised. One common measure counts how many statements have been executed by any test. The appeal of such coverage is twofold:

　　什么是代碼覆蓋率?它是代碼是否全面執行的數字衡量。一個常見的衡量是計算所有測試共執行了多少條語句。對這種覆蓋率的呼吁有兩方面：

　　1. If you've never exercised a line of code, you surely can't have found any of its bugs. So you should design tests to exercise every line of code.

　　如果你從未執行過某一行代碼，你當然不能找出它的任何 bug 。所以應當設計一個可以執行每一行代碼的測試。

　　2. Test suites are often too big, so you should throw out any test that doesn't add value. A test that adds no new coverage adds no value.

　　測試套件常常很大，所以應該拋棄任何不能增值的測試。一個不增加新覆蓋率的測試不能增加任何價值。

　　Only the first sentences in (1) and (2) are true. I'll illustrate with this picture, where the irregular splotches indicate bugs:

　　句子(1)和(2)中，只有第一句是正確的。我將用下面的圖說明，其中的不規則黑點指示的是 bug ：

　　If you write only the tests needed to satisfy coverage, you'll find bugs. You're guaranteed to find the code that always fails, no matter how it's executed. But most bugs depend on how a line of code is executed. For example, code with an off-by-one error fails only when you exercise a boundary. Code with a divide-by-zero error fails only if you divide by zero. Coverage-adequate tests will find some of these bugs, by sheer dumb luck, but not enough of them. To find enough bugs, you have to write additional tests that "redundantly" execute the code.

　　如果你僅編寫需要滿足覆蓋率的測試，你會發現 bug 。那些總是失敗的代碼不論怎樣執行，你都肯定能發現它們。但是大多數的 bug 取決于如何執行某一行代碼。例如，對于“大小差一”(off-by-one)錯誤的代碼，只有當你執行邊界測試時才會失敗。只有在被零除的時候，代碼才會發生被零除的錯誤。覆蓋率足夠的測試會發現這些 bug 中的一部分，全靠運氣，但發現得還不夠多。要發現足夠多的 bug ，你必須編寫其他的測試“冗余地”執行代碼。

　　For the same reason, removing tests from a regression test suite just because they don't add coverage is dangerous. The point is not to cover the code; it's to have tests that can discover enough of the bugs that are likely to be caused when the code is changed. Unless the tests are ineptly designed, removing tests will just remove power. If they are ineptly designed, using coverage converts a big and lousy test suite to a small and lousy test suite. That's progress, I suppose, but it's addressing the wrong problem.

　　同樣的原因，因為有些測試不能增加覆蓋率而將它們從回歸測試套件中去掉也是危險的。關鍵不是覆蓋代碼;而是測試那些當代碼更改時容易被發現的 bug 。除非測試用例是不熟練的設計，否則去掉測試用例就是去除作用力。如果它們是不熟練的設計，可以使用覆蓋率將一個大而粗劣測試用例套件轉化成一個小而粗劣的測試用例套件。我想這是進步，但是與這個問題無關。

　　A grave danger of code coverage is that it is concrete, objective, and easy to measure. Many managers today are using coverage as a performance goal for testers. Unfortunately, a cardinal rule of management applies here: "Tell me how a person is evaluated, and I'll tell you how he behaves." If a person is evaluated by how much coverage is achieved in a given time (or in how little time it takes to reach a particular coverage goal), that person will tend to write tests to achieve high coverage in the fastest way possible. Unfortunately, that means shortchanging careful test design that targets bugs, and it certainly means avoiding in-depth, repetitive testing of "already covered" code.

　　代碼覆蓋率的一個重大危險是它是具體、主觀而易于衡量的。今天的許多經理都使用覆蓋率作為測試員的績效目標。不幸的是，一個重要的管理規則適用于這里： “告訴我如何評價一個人，然后我才能告訴你他的表現。”如果一個人是通過在給定的時間內覆蓋了多少代碼(或者是在多么少的時間內達到了特定覆蓋目標)來評估的，那么那個人將傾向于以盡可能快的方式達到高覆蓋率的測試。不幸的是，這將意味對以發現 bug 為目的的仔細測試設計的偷工減料，這當然也意味著避開了深層次、重復地測試“已經覆蓋”的代碼。

原文轉自：http://www.uml.org.cn/Test/200709289.asp