March 2005

Developers Testing Software, Take 2

Charles Adams quantum6013ATcox.net had the following comments about developers testing software.

In regard to "March 17, 2005 Blog Entry -- Can Developers Test Software?" For me, your observations go back to the dynamic between verification and validation.

Verification

In my experience the developers have the verification mindset, "Does it do the job?" Most everyone who uses the software has the mindset, "Does it allow me to do my job?" And that is the essential difference. The job or requirements are dependent on what pair of glasses one wears. Developer, project manager, line manager, procurement official, program manager, and user all have different ideas about the software.

Validation

I hold the opinion that "doing the right job", or validation, is much more important than "doing the job right", or verification. Validation ultimately includes verification, but not the other way around. Unfortunately verification seems to be the tail that wags the dog, at least in government procurements and way too often in commercial procurements.

To me it seems that software craftsmen are not the only ones who do not look to the user. Jerry Weinberg in his QSM Vol 4, "Anticipating Change" quotes Witold Rybczynski:

"What did 'getting it right' mean in practice? To the classically trained architect it meant, first of all, pleasing the client or, in a broader sense, the user of the building (not always the same person). This unassuming, and to most persons obvious, requirement needs emphasizing in a period when architectural design has become a self-expressive pastime. The great Chef Carême said, 'In matters of cookery there are not a number of principles, there is only one and that is to satisfy the person you are serving.' If I were to quote his advice to my students, they would find it a hopelessly old-fashioned and intolerable imposition."

Your comment, "A quick review of the most project dynamics indicates that everything can slip as far as delivery except the final ship date." is only the beginning. I usually see final ship dates turn into the first of many unplanned ship dates, which begs the old question, "Why do we have time to do it over, but never the time to do it right the first time?"

Appreciations

I appreciate Charles for taking the time to read and respond!

Can Developers Test Software?

The Pacific Northwest AYE Community members gathered on 3/6 in Seattle. The group, split between developers and testers discussed the recent phenomenon of using developers to do testing.

It Only Makes Sense

After all, developers have always done some testing.

• Does the code compile?
• Does the project still build?
• Does the new code do what it should?

Great! What's next on the list of things to do?

Agile methods promote test driven development. The testing plan gets written before the code does. Management is simply proposing the logical result. Fire the testers since the developers are now doing the testing! But does this really make sense?

You Mean There's More To Testing?

Yes, Virginia, there's more to testing. I'm a developer, not a tester. I know only a little bit about testing, but I DO know that testing methodologies move just about as fast as development methodologies do. It's no longer just about white boxes and black boxes. Check out Quardev and Satisfice for some great ideas in testing. Developers are lucky if they have the resources (time and training) to stay current with new development concepts. What are the odds management will think to include resources for learning how to test?

The View From Over Here

Another point to consider: Developers and testers have different local and global view points. The goal of development is to create and prove software works. Testing tries to discover those conditions where the software doesn't work. Two very different ways of looking at the software. Shannon Severance does both developing and testing, but never on the same project. Once you get a particular view point on a piece of software, it's hard to switch to another viewpoint. The difference in view points leads to better software.

Slip Sliding Away

Given that developers like to develop, not test, how does having developers do testing impact testing time? A quick review of the most project dynamics indicates that everthing can slip as far as delivery except the final ship date. Software development requires discovering what features should be coded, figuring out how to code them, coding them, testing the code and delivering the software. Testing time gets compressed in most projects to the time left (if any) after the major development is completed and the scheduled delivery date. With developers doing the testing, time (if any) will naturally shift to development to squeeze in "one more feature." After all, "I've been testing as I go along."

A Tester, By Any Other Name

Any significant software project involves multiple development teams. Who tests the overall results? Designate a developer to handle this release's integration testing? Doesn't that make them a defacto tester? And an unhappy one at that. Developers don't have the mindset, training, and knowledge to do a good testing job. Why force this on them? Good software needs multiple viewpoints (after all, pair programming is basically instantaneous code reviews). Why not get the view point of some one who wants to do the job?

Let's Think About This Some More

Firing the testers and using developers to test is a Siren's song. Do it long enough, and you'll hear sirens again as the software crashes and burns on your former customers computers.

Stabilizing Systems

Engineers make the darndest assumptions. I made one such assumption in "Change and Stable Systems". The unstated assumption involved starting with a stable system. But what do you do if your system (as in team, project, company) is unstable?

Stable by Existence

It was a fair assumption. After all, the topic was "Change and Stable Systems." Unstable systems don't exist very long. Like fire flies on June nights, they flash briefly, then go black.

First Things First

If the system is unstable, the first change to make is get stable. Being unstable violates the Law of Continued Existence. You have three intervention points to create stability: gain, energy, and degrees of freedom.

Gain

Technically system output divided by input. The higher the gain, the more amplification of the input. A common example is putting a microphone in front of a speaker connected to the microphone, and turning up the volume. In organizations this tends to be the "rumor mill". Information, real or imagined enters the rumor mill and gets amplified and re-transmitted. This makes it difficult to tell fact from fiction. To adjust the gain have meetings where you can learn what fictions are circulating. Then share the facts as simply as possible.

Energy

Energy comes in two forms:

• Potential energy - the ability to get work done.
• Kinetic energy - doing the work.

If the system is unstable, try changing the energy balance. Is everyone running around in panic mode? Slow things down and convert some kinetic energy to potential energy. The opposite situation, too little kinetic energy creates sluggish responses (if any), and prevents change from occuring.

Degrees of Freedom

Degrees of Freedom means flexibilty of action. Normally we want this. However if we're dealing with an unstable system, too much freedom creates a "loose cannon on a rolling deck", and the ship we sink, might be our own. Reigning the troops in won't be popular, but it will reduce the number of ways to go wrong. It will also change the system gain and energy.

The Only Hard and Fast Rule

Unfortunately is, "There are no other hard and fast rules." I can't tell you which parameter to work with. But by manipulating gain, energy, and degrees of freedom, you can restore stability to your team, project, or organziation.