Whose Quality Is It Anyway?

I’ve always been a lover of nature. I grew up surrounded by nature and learned how to explore as a child in the woods close to our house and a forest only a half hour hike away. As soon as I was old enough to make reasonable decisions on my own and remember my home phone number, I was allowed to explore the natural world around me without much adult supervision. Thanks to my parents, I knew that littering was a problem and I grew up with an awareness of leaving the land better than I found it. In today’s terms, I was what some would describe as a “free-range” child.

As an adult, my love of nature and exploration in all forms were the primary reasons I found my home in the Burning Man community. The community has an established Leave No Trace (LNT) ethos, one of ten guiding principles of Burning Man.

Burners refer to anything that is not “originally of the land” as Matter Out Of Place (MOOP) and any MOOP found or created is to be disposed of in an environmentally responsible manner. Burning Man takes LNT very seriously and in addition to every citizen doing their part to keep the playa clean, the event itself has volunteers dedicated to ensuring the desert is restored post-event to its original pristine state. Side note: Any tester will find the MOOP Map is a fascinating way to visualize Leave No Trace coverage in Black Rock City and measure progress from year to year.

I am a LNT enthusiast and my standards for quality are very high when it comes to MOOP. As a tester, my standards for evaluating product quality are also quite high.

This brings me to some important questions:

What exactly do I mean by “quality”? And how much do my standards matter?

In his book, Quality Software Management: Systems Thinking, Jerry Weinberg brings up a few foundational points about quality we should consider to answer these questions:

Quality is relative

“…what is adequate quality to one person may be inadequate quality to another.” (p. 5, Weinberg, 1992)

During my adventures at Burning Man, the heuristic I apply is “If you see it, pick it up” – when I see MOOP, I stop and pick it up to dispose of it properly later on. I do not feel good if I see MOOP and do nothing about it. I feel the very presence of it threatens the value of the natural environment we all have the privilege to enjoy.

This heuristic does not apply well in the “default” (that’s what we burners call the world outside of our community) because the larger society does not generally cultivate a common ethos around LNT, so there is a lot of MOOP everywhere. Many people treat the environment like their own personal trash can or ashtray, throwing trash and cigarette butts anywhere they please. In the default world, it is not possible to stop and pick up every piece of MOOP one sees.

The presence of trash in the natural environment negatively impacts my enjoyment of those spaces. The value of my experience is lessened when I see a water bottle gently floating down a stream. The presence of trash also proves that there are people out there – many people – who are not negatively impacted by the presence of trash at all. Or at least not enough to do anything meaningful about it in the moment.

Many people are content to just walk past that crushed beer can sitting on the side of the trail. Perhaps people do not feel it is their responsibility to clean up after others or maybe they do not want to touch trash on their afternoon walk when they do encounter it.

Whatever the reasons:

• people have different perceptions of value,
• and these perceptions affect how they perceive quality,
• which ultimately influences their decision on whether to do something about it.

In software development teams, testers will often find their notions of quality at odds with developers, product managers, designers, and other stakeholders. Much of the time, our standard for quality will be much higher and our level of scrutiny of the product much deeper. Our craft requires that we evaluate the product from multiple angles, so testers are in a great position to know what the product risks may be, what problems the team may run into, where the known bugs exist, and so on.

Over time, we become repositories of product knowledge as we build our credibility within our teams. When influencing decisions about quality, we can advocate for what we think makes sense or sound the alarm bells when something about the product isn’t making sense (and hopefully have some evidence or data to support our arguments).

There is a person behind every statement about quality

“Quality does not exist in a nonhuman vacuum. Every statement about quality is a statement about some person(s).” (p. 5, Weinberg, 1992)

Burning Man’s adherence to the Leave No Trace principle is a statement about quality. Defining anything not “originally of the land” as MOOP and setting the expectation that all MOOP be removed raises the collective bar for the LNT effort in the Burning Man community. Every responsible burner participates and does their part in keeping the playa clean.

In addition, the Burning Man organization must meet strict LNT requirements for use of the public land to be able to hold the event every year, so the community and the Burning Man organization work together to create accountability around LNT. Guidelines are established as to what constitutes MOOP and how to deal with it. The community has a clear vision of what our quality story looks like from year to year.

In software development organizations, we often hear the following statement about quality: “Quality is everybody’s responsibility.” Often this statement comes from somebody in a leadership position, or it may be heard among frustrated testers who feel like the burden of product quality falls only on them. The problem is that this statement is very vague and usually doesn’t go into enough detail to be useful in practice. And if we do not know the specifics of what various individuals and departments do to improve quality and what it means to them in their contexts, then it is nearly impossible to instill the shared ethos of “Quality is everybody’s responsibility.”

Most organizations do not have a clear picture of what quality means to the various stakeholders in and outside the organization. Often leaders within the organization have disparate opinions on what’s important. Without effective leadership to steer the quality ship, teams end up building products that are incongruent with the company mission or consumer expectations, departments fight for budgets and position leading to bad blood between teams, and the wrong projects are prioritized.

A clear vision of what quality means to an organization is key and being specific about it is even more important. What does the organization value? Where does it want to go? What value does the organization want to bring to its customers? What about all the other stakeholders? What do they value? What has to improve now?

Opinions and decisions about quality are political and emotional

“More quality for one person may mean less quality for another.” (p.6, Weinberg, 1992)

The existence of trash cans in parks and natural spaces is a perceived necessity because, as a society, we have not made any efforts to cultivate a culture of LNT. One of the tenets of LNT is to pack out your own trash. This means that once you’re done drinking that soda down at the beach, you take the can home with you and dispose of it.

There are no trash cans or dumpsters for public use at Burning Man. By LNT standards, trash cans actually lower the value of the experience and disincentivize some participants from practicing LNT.

Parks officials make decisions to provide trash cans in certain places, yet MOOP still ends up thrown in the bushes, or maddeningly, next to the park trash can. The existence of trash cans does not necessarily improve the cleanliness of the park or the value of the park experience. Trash cans may alleviate the existing problem from becoming or looking worse but they do nothing to change the underlying offending behavior.

Worse than seeing trash scattered randomly around a park is seeing a park trash can overflowing with it.

Trash cans are a band aid; not a solution.

When evaluating software and assessing quality, it makes sense to pay attention to the trash cans and what’s in them. I’ve been a part of many product builds over the course of my 22-year tech career. During every one of those builds, certain design or technical shortcuts in the product were made. Reasons for this vary. Often, the team is under a tight deadline and there is simply no time to dedicate to a better solution. It becomes problematic when every decision starts to feel like a shortcut and then the whole product experience starts to feel more like trash, and less like something a customer would value. It’s good to stop once in a while and assess the landscape, as a team. Ask some questions:

How many of those trash cans exist in the product? How many are dumpsters? Are any of them an active dumpster fire? Are we dealing with the accumulated trash?

There are times when it may be ok to let trash pile up. For example, during the build stages of a feature, we may let some bugs sit around until we fix other more important stuff and then come back to those bugs later. Eventually we should strive to take out the trash or risk an even bigger mess.

A practical definition of quality

“Quality is value to some person” (p.7, Weinberg, 1992)

Jerry Weinberg’s definition resonates because it takes into account the political and emotional motivations behind decisions about quality. He goes on to say, “By ‘value’, I mean, What are people willing to pay (or do) to have their requirements met?”

I’ve talked at length about value in the above paragraphs. Our emotions and political motivations drive how we assess value, how we form opinions about quality, and ultimately what we do about it, if anything. I take a trash picker tool and a trash bag on most hikes and walks now, so that I can do something about it, knowing the job will never be complete and there will always be more trash to pick up.

“Whose opinion of quality is to count when making decisions?” (p. 7, Weinberg, 1992)

It matters whose opinion counts when making decisions about quality.

Park officials making decisions about placing trash cans in the parks may not care about the opinions of persons like me who practice Leave No Trace anyway. Their concern is with those who would not take their trash home with them and instead litter it about the park. Their decision to provide trash cans around the parks is a quality decision that makes sense, given the context.

Do I like that this decision has to be made? No. Do I accept it? Yes.

This happens in testing, as well. Testers are generally not the ones making the final decisions about product quality.

The stakeholders who do have the power to make those decisions are continually weighing whose influence and opinion matters as they make those calls. Realize that it may not be your opinion that matters, depending on the context. It’s our job as testers to support these stakeholders with evidence and data about the product and the risks so that they are empowered to make informed decisions.

We get to influence those decisions, and that’s good enough. Many times, the final decisions will mirror some aspects of our influence, but there are also times when no matter what we say, a decision we do not agree with will be made. Those times that the decisions made do not reflect our advocacy, it’s time to accept it and move on.

There is so much more to test and it’s not personal.

References:
Weinberg, Gerald. Quality Software Management: Systems Thinking. 1992. p. 5-7.