Feedback to "A Question of Quality"- Murphy's Law,
Read the original artilce at A
Question Of Quality
return to Murphy's Law
I think I struck a chord with a few readers on this one. Some
of these e-mails are longer than the original article! The first
comments here agree with what I had to say but see Stephen
Hanka's e-mail for some disagreement. Bruce
Reynolds gives an account of how ISO damaged a number of companies.
Finally Larry Bannister, a TickIT auditor
gives a robust defence of the ISO 9000 standards. Bob
Schaefer also defends the standard.
Ron Bauerle quotes
his favorite bits from the article.
I could cut and paste
a dozen excerpts, but will let you read it for yourselves...
Actually, here are
the two best:
"If no one uses a document
to produce something of value to the company, it should not be
written. Auditing alone is not reason enough to create documents.
the ISO 9000 industry
can't produce any measurements to show that applying ISO 9000
has improved a company in any way. Something - productivity, share
price, or customer satisfaction - should increase if the movement's
claims are true, but no measurements exist to prove that this
is the case. "
Too bad this didn't
come out 5 years ago when it was getting shoved down our throats...
Your article in EMBEDDED SYSTEMS magazine warmed my heart!
I have over 25 years of experience in managing manufacturing
for a number of companies, such as Motorola and Harris, Corp..
I also have 15 years experience in software management, I written
applications and embedded code. I also have software patents for
wireless data. Therefore my comments may carry some weight in
this arena of ISO 900X.
In every successful enterprise there comes a moment when someone
suggests that the only way to gurantee continued success is to
'capture' the best of the current activities into some codified
process. This is accepted as a compliment to those whose efforts
led to this success. A feel-good experience akin to a religous
feeling motivates those involved. As in religous experiences,
if a little 'grace' is good, a lot more must be better (again
my experience...I am also a Catholic Deacon). This effort, in
conjunction with ISO 9000, leads to process Police. Process is
more important then outcome! What started out as a 'simple' effort
to capture the best practices becomes, as you so well point out,
a restraint to creativity or individual efforts to improve. The
best process is simple, flexible, and rational. Good management,
good workers (almost) always produce good results!
End of rant.
Just finished your
article in Feb ESP. Excellent points. I used to work for a company
who promoted the person who got the ISO 9000 certification to
be vice president, and later on the company went bankrupt, why?
You right on the point of documentation. If a company has ISO9000
certification does not mean it produces quality products. If more
than a quarter of a million companies are ISO9000 registered and
if they are all buying and selling products to eachother, can
you imagine the quality of the products based on that certification?
You are also right about auditors. I am sure there are bogus auditing
companies, too. And, who audits these auditors? and what standard
do they follow? The ISO9000 sounded like a fad to me when it spread
all over the world and waiting for some company to come and proclaim
that they do not need ISO certification, that their process and
products beat the ISO documentation requirements.
Hopefully, enough people
will voice their opionion about this useless standard that does
not have any effect on the quality and everybody will start saving
money that they have spend on audits and keeping the certificaiton
alive. Keep up the good work.
I read your article
for embedded sytems regarding ISO-9000.
I thought it
Please continue the flow of great ideas.
Gus S Calabrese
You've touched ---
no, cut --- a nerve with your insightful comments about the bloated,
self-ingratiating bureaucracy that has grown up around ISO. What
started as, I believe, a thinly-veiled form of protectionism,
has become entrenched in modern business. The sad thing is, everyone
participating KNOWS that the ISO badge is meaningless, and yet
we keep playing along. The system adds nothing to productivity,
is prohibitively expensive to smaller companies, and, in spite
of its ostensible aim, leads to mediocrity becoming entrenched
when the paperwork becomes easier than the real work of progress.
I believe in continuous,
demonstrable improvement, but ISO is not the answer. Thanks for
the courage to speak out.
Vince Owens Ohio, USA
Your February column in Embedded Systems Programming was interesting
and entertaining. Particularly the point about the rarity of a
company failing an audit.
I recently left a company that was ISO 9000 certified, and quite
proud of that fact. Or at least the executives and the ISO office
were. The engineers knew it to be the joke that it was.
About a year before I left I went through the project files for
all of the projects on which I had worked while employed there.
I was looking for information on our peer reviews, and the ISO
process required that information to be in the project files.
What I found was an utter mess.
I doubt that 10% of the reviews for which there was any evidence
had all the evidence required by the process. I would guess that
no more than half the reviews required by the process had ever
taken place. The real gem was the paperwork for two reviews, which
was duplicated with different names. The single sheet of paper
said, in essence, we know this review took place, it was about
six months ago, but we have no record of what was reviewed, who
reviewed it, or what was found. We do know that the process requires
something in the project file for each review, so this is it.
Trust us, the review really was done.
This company, of course, never failed an ISO audit.
I made a brief study of quality methods while being retreaded
in graduate school, and this was followed by employment in the
telecom industry in an ISO 9000 shop. I can only say that Niall
Murphy was, if anything, too kind to ISO in his February column.
The major fallacy of today's "quality" efforts in large electronics/software
shops is the assumption that engineering is a production activity.
Engineering is by definition the solving of problems, of application
of new principles, or new application of principles. Engineering
activities are by definition either done once, or unnecessarily
duplicated. Process efforts such as TQM are aimed at improving
quality by reducing variance in PRODUCTION activities. The idea
is to bring a process into statistical control by removing special
causes of variance, and then to reduce the variance. If the product
then is unsuitable, it is at least conformant to design, so the
design can then be adjusted. This all makes perfect sense, and
reduces costs as well, thus supporting Philip Crosby's assertion
that "quality is free".
ISO 9000 attempts a much lower bar: that of documenting whatever
process is to be used. The biggest REAL value a firm can get from
ISO 9000 is frequently simply to record and know what it is actually
doing, as some firms don't even know THAT. The idea is that the
process may be bad, but at least it's written down so it can be
reviewed and analyzed. And, presumably, has measurements specified
so the benefits/losses of any change may be quantified. The best
engineering managers, when forced to do ISO 9000, will use the
opportunity to learn more, and to modify practices to better accommodate
their engineers. This is too infrequent, of course. In all cases,
ISO 9000 imposes overhead - IMHO, about 20%.
The biggest COMMERCIAL value a firm gets from ISO 9000, however,
is the claim that it is certified. This allows it access to business
from customers who wrongly believe that the certification proves
something about the quality of product or service. Now we're doen
to the nitty-gritty. The goal is certification. Who cares what
we do, or even how we do it, as long as we're certified? And we
know all about the audits (we buy them from a vendor), so we can
usually fake a process sufficiently to get the certification.
This whole approach works against an effective engineering organization,
because it is required to document processes that probably shouldn't
exist in the first place. Now, granted, there are certain "engineering"
activities (perhaps, e.g., the archiving of blueprints) that admit
of a high degree of repeatability, and these might be a proper
subject of certain ISO procedures. Similarly, perhaps, a checklist
of certain regulatory requirements. But the general case is that
the important stuff either isn't repeatable, or doesn't need to
Bruce Reynolds gives the most dramatic
accounts of how much damage ISO can do...
Mr. Murphy: I most
resonantly enjoyed your ISO9000 diatribe published in the February
2K2 Embedded Systems Programming. My poor wife has been subjected
to almost a decade of my rantings on the subject; she is convinced
I am seeing plots. Perhaps there is one: an insidious plan to
prevent creative people from competing with slugs and nitwits.
In nature, remember, the reproductive capacity of the nitwit exceeds
the survival skills of the clever.
My first exposure
to this insipid scam came without warning. Being employee number
three of a subsidiary which grew to over 300 in five years, I
was most astonished and affronted when two persons whom I had
never before encountered sauntered into my lab and announced in
a huff that I had entirely too many data books on my shelves,
and must remove all but 12 by the end of the day. It seems our
new ISO9000 program (started as a '"quality improvement" enhancement
by the owner-president's idiot brother, who, because he looked
good in a polo shirt, wound up running our division) included
interior decour. My several dozen data books were in violation
of the new ISO documentation, which also dictated plants and carpets.
Within 3 months every project was put on hold pending ISO evaluation;
a year later, we still hadn't been able to work on anything, and
were roundly scolded for not getting projects done. Any attempt
to actually do anything was punished. Failure to achieve deadlines
was punished. By the end of a year, every competent engineer had
quit. I beat the crowd by 6 months.
Some time later (after
an episode of actually enjoying my job, an infraction for which
engineers are inevitably punished) I found myself in the service
of a well known environmental instrumentation manufacturer. For
the first month I had nothing to do, as my in-processing paperwork
was trapped in the ISO9000 mill, and I was not allowed to do more
than count my (and I am NOT making this up) seven -count them,
seven- ISO process mandated paperclips. There I sat, making almost
a hundred large per year, without a phone and counting paperclips.
My intranet connection would of course not be activated until
the paperwork was ground into pulp.
Now the nutty thing
about this company's ISO nonsense was that there were several
levels of documentation. Persons on lower levels were not allowed
(for security reasons! God, they measure poop... what security
reasons?) to view the ISO9000 documents for higher levels. Yet
the lower levels had to FOLLOW the higher levels' documentation.
But we could not know what the documents said. It was up to the
department managers to 'guide' the workers; workers were not allowed
to know what the documents required, but one was 'informed' when
one violated a higher-level procedure. Then one was forced to
guess what the proper procedure was. Wrong guesses were punished.
Once I was official,
I still had nothing to do, as the project I was hired to work
on was stuck in another ISO-process-required committee trying
to decide (again, I'm not making this up) whether the device would
have four or seven buttons. Now, I could have been working on
the innards of the thing, but nooo... engineering work was forbidden
until all specifications were signed off. That took almost 8 months
from the date I was hired. Eventually, marketing's gestation period
ended and I was presented with a final spec which made no sense,
defied the laws of physics, and granted engineering 4 months.
They began the button argument 6 months before I hired on. A full
14 months were spent in ISO-process mandated wrangling over the
number of buttons on the damned thing, and then they wanted it
finished in 4 months. I insisted this time table was absurd, and
was informed that our ISO documentation called for an 18 month
development cycle, there were still 4 months left, and it would
be engineering's fault if it didn't get done on time. During the
proceedings five other engineers quit, leaving 2 of us to pull
this off. This did, of course, not affect the schedule. The Nazi
accountant who ran the place viewed the reduction in salaries
paid out to engineers as a GOOD THING. No replacements were recruited.
The vacated engineering space was taken over by marketing.
But I digress... After
counting my allotted paperclips often enough to ensure myself
that there were, in fact, seven, I amused myself by looking at
the schematics for a new process monitoring probe they had recently
introduced. After some pondering, I came to the conclusion that
the thing simply couldn't work. This device was supposed to measure
dissolved oxygen in water. The basic sensor was ingenious and
the system worked great on the bench. Unfortunately, my analysis
indicated that submersion in a real world earthed tank would produce
a ground loop of such biblical proportions that smoke could well
ensue. I pointed this out, and was summarily punished.
Two months later,
there were rumblings from MARKETING (I must use full caps as these
entities consider themselves dieties and will broach no disrespect)
that customers were complaining, and these complaints were not
being submitted in our ISO9000 certified method. Indeed, instead
of filling out the proper forms, they were (gasp) bitching...
Bitching, I say... that the sensors worked OK in a bucket but
went nuts in the process tank. Following several weeks of hand-wringing,
a contingent of managers descended upon the corner-of-the-hall
which was my office and demanded that I explain why I thought
the sensor was flawed. I outlined the current design's deviations
from accepted engineering practice. I was informed that the sensor
had passed all its ISO9000 tests, and was punished.
A month later, as
the complaints mounted and as I continued to await the final specification
necessary before beginning work on my project, I was confronted
by the director of engineering, the principal engineer, and the
head of marketing. They all wanted to know why I had been speaking
(internally, of course) negatively regarding the aforementioned
sensor. I was shown the result of the recent ISO9000 audit, which
found no problems with the suspect sensor's development, and I
was roundly chastised for my bad attitude. At this point I had
already set up, in the lab, one of the sensors; mounted on a chemists'
tripod; it could be immersed in either an isolated beaker or a
grounded metal sink. Suffering some derision for setting up an
experiment which was not predicated upon established processes,
I convinced my antagonists to accompany me to the lab. In short
order, I demonstrated that whilst the sensor performed splendidly
when bobbing in an isolated beaker, it went berserk when immersed
in the grounded lab sink. Upon some discussion, it was acknowledged
that 100 percent of the sensors shipped had been returned as non-functional.
We, of course, were merrily shipping non-functional replacements.
I was given 2 weeks to correct any design deficiencies, and admonished
to follow the ISO procedures in future.
At the end of two
weeks, I had corrected the design flaw, prototyped the solution
(I got in trouble for that too... prototypes were not part of
the design process) and proved, via grounded sink, that the solution
worked. Another month was spent in getting the changes thru the
Eventually I was ensconced
in the project for which I was hired. I will not go into the frustration
involved; suffice it so say that gin was the only respite.
Five months later,
I got a call from one of the few cogent marketing guys, who was
attempting to resolve an installation issue. (This company doesn't
do field technicians... because their ISO process is so tight,
they have convinced themselves that they don't have field failures...all
problems are due to the customers' ineptness, and if the customer
is truly pissed, they dispatch a marketing guy to buy more gin.)
This guy had been sent out to a site which had about a dozen of
these sensors; they all worked in a bucket, but went nuts when
put in the grounded tank. [wasn't this where I came in?] I was
sure my correction was valid, but the guy in the field assured
me he had picked up a new batch of sensors from manufacturing
before heading out; these were fresh off the press, and they were
failing on contact with the grounded process water. After about
an hour of over-the-phone diagnosing, I asked him for the model
number embossed on the side of those new sensors. Sure enough,
it was the model number of the non-functional version. I stomped
out to manufacturing and examined the latest crop.... as predicted,
they were continuing to produce the non-functional version. Turns
out that our ISO9000 process requires us to use up all existing
circuit board stock before employing any new designs. Never mind
that the existing boards didn't work. ISO is ISO, and God forbid
we would not follow the ordained path.
Irritated beyond normal
levels, I hijacked a lift and tossed the crate of defective boards
into the dumpster. Now, this is wierd... I got in a world of trouble
for taking the defective boards 'out of the process'; but once
they WERE out of the process, we couldn't use them. They stayed
in the dumpster, and the corrected design (in the real world a
normal ECO would have sufficed) was used on subsequent production
runs. No problems since. I was reprimand for finding a problem.
I was reprimanded for finding a solution. I quit a few months
later, when (again, I'm not making this up) I was chastised for
pointing out that their new humidity sensor would only work in
I took interviews
at several concerns, but generally lounged about the house. Eventually
the wife forbade further remodeling activities, so I accepted
the position of lead hardware engineer at a power conversion concern.
I already had some 'alone time' scheduled, for which they were
most accommodating. After a few weeks spent wandering in the desert
(Death Valley and Mojave... OK... I'm enough American Indian to
getcha into the woods; not enuf to getcha out...) I reported for
work. I was astonished to find that the company I was now working
for was not the same company with which I had, on several occasions,
interviewed. In the interim they had gone ISO.
Regardless of the
fact that the company had just pulled off a 200 million dollar
IPO, my cube was sans chair. I had a computer but no monitor.
Two weeks later, the chair appeared; I could figure out how to
sit in it myself, but the monitor was another story. I could not
'use' the monitor until signed off on it via ISO paperwork, after
which the monitor may be scheduled for delivery. Now, this company
was not yet fully into the ISO thing; they were just trying to
function in an "ISO FASHION" in preparation to the actual event.
Oh-oh. I sensed trouble brewing.
I was tasked with
developing CPLD code for a new power converter. The specification
required us to use a particular tool in a specified manner using
ISO documented methods. Unfortunately we had not actually bought
that tool, so, as specified in another document, we were not permitted
to even think about HOW we would use it if in fact we had access
to it. I spent months judiciously not using the required tool
to work on the project we were not authorized to work on which,
according to the schedule we were not allowed to see, we were
I began to suspect
that a John Cleese character was running the place. Total Industrialized
Schizophrenia. Corporate dementia.
On a Monday our group
was told to have a subsystem prototype working by Friday. On Wednesday
the design was done and the parts reqs were submitted, with a
note that the parts had to be in next-day. We immediately got
a memo from purchasing that the procurement paperwork took 7 to
10 days to process, after which the actual parts may be ordered;
no overnight shipments would be processed without the express
permission of the GM, and everyone should please go away. I asked
the GM for overnight authorization, but he said the ISO process
didn't require that, and refused.
Of course on Friday
there was nothing to show. The GM went ballistic; "where's the
prototype?" "No parts, I answered" "I don't want any excuses"
he scolded. "We need parts NOW if we're going to prototype NOW"
I retorted. "I'm tired of excuses" he snorted, and stormed out
of the room. I received a memo on the importance of learning to
follow the ISO procedures. (We were not ISO yet, but we apparently
needed to practice...) and another memo on the importance of meeting
Still not quite ISO,
but practicing, some really nifty procedures were put in place.
One of them mandated that any samples from vendors be processed
thru purchasing, that an internal part number be assigned and
that part number be ordered from the vender using a zero-price
PO, that the part be entered into the inventory control system
and assigned a stock location. Receiving could not receive any
parts without an accompanying PO number stamped on the box, and
once received under the null price PO that part would not be delivered
to an engineer without a Bill of Materials number, which wasn't
assigned until the engineer actually designed the part into something
and the formal design review had been completed. Which pretty
much shoots the concept of a 'sample'. So we took to requesting
sample parts be delivered to our homes. Then they came out with
a rule that no parts may be used (even for prototyping) which
didn't go thru the null-value purchasing process. No parts were
allowed in the lab unless they had a Part Number. Now you just
try giving a null-value PO to something like Maxim's web-sample
site. They can't process such a thing. No such a field as 'null
value PO". So we couldn't get samples from MAXIM, or TI, or DALLAS,
or anyone. And even if they did come in, we couldn't get them
out of the inventory system unless they were going into a product.
It cost about 50 dollars per PO to process the paperwork, so not
ordering samples for prototyping saved money. Recognise please
that this new company didn't actually HAVE a product yet. They
are still planning for the day when they have one.
So there we are, in
the normal monday morning beating, being berated because the tests
on the prototype we don't have parts for yet are not on schedule.
After three hours, the powers that be allow us the remainder of
the week to finish the tests; these runs are vital, as they will
be presented to the board (which comprises doctors and lawyers
and famous astronauts) who are expecting splendid results. I am
allowed, as a special dispensation, to bring in parts from my
private stash in order to commence these tests. This is a once-only
allowance, and is being granted only after I have begged forgiveness
for not having specified the components two months before being
appraised of the design requirements. I am most grateful for the
leniency, and we engineers grovel our way out of the meeting intent
on performing this suite of tests.
Upon entering the
lab, we are amazed to find that every single bit of test equipment
is missing. Every 'scope, every analyzer, every multimeter...
nowhere to be found. After some hunting, we learn that everything
has been sent out for calibration... some stuff will be back by
friday; some in another week.
Our (not that we're
certified yet) ISO9000 plan, which no engineer has ever seen,
calls for all instruments to be calibrated this week. Doesn't
matter what other schedules are paramount. So in I go to the management
gods to try to make sense of this; "We don't want any excuses...
these tests must be successfully completed by the end of the week",
says they; "All the test equipment is gone, says I"; "We're tired
of your excuses", says they.
Late Friday some of
the equipment returned. I spent Saturday and Sunday trying to
figure out why my CPLD code was not producing the waveforms I
had designed with the obsolete free demo version of the package
we were required to use but not allowed to purchase. Late Sunday
evening I began to suspect the freshly calibrated analyzer. I
broke protocol, drove 30 miles home and fetched my own (The company
had others back from cal, but I was not authorized to use them;
they were locked away, and only the QC guy could release them.
We were not allowed to use our own stuff, but I was desperate.)
and found that the freshly calibrated analyzer had holes in its
trace buffer which made it look like my logic was flawed. I was
maximally pissed. I'd spent 20 weekend hours figuring out that
our freshly calibrated instrument was defective. I resolved to
stop being so nice and really have a fit.
Monday morning, before
I could vent, we engineers were informed via corporate lecture
that anyone entering the labs without an approved antistatic foot
strap would be summarily dismissed. This was the first edict of
our new corporate ISO quality wonk, and was presented with much
fanfare. Recognize please that this company didn't actually have
a product yet... but it was important that if ever we did manage
to develop a product, everyone involved would be static proof.
Engineering was then chastised for not completing the aforementioned
So we engineers approached
the labs, and were greeted by a security person who demanded to
know where our antistatic foot straps where. I said that this
was all new to us, and we would be glad to put them on. Where
could we get them? "Oh, we don't have any right now, they're still
on order", he responded. We were not allowed into the lab. Off
to the GM I stormed. "Now let me get this straight... we have
to have antistatic foot straps to enter the lab, we will be fired
if we don't have them, but they're still on order? We can't finish
these tests without test equipment, and we certainly can't do
them without going into the lab but we'll get fired if we go into
the lab to test circuits we don't have parts for using test equipment
we don't have access to without the antistatic straps we don't
have... that makes no sense". "All I ever get from engineers is
excuses", he huffed. "I don't understand what you have against
ISO", he barked, and stormed out. My chest began to hurt, and
I was hauled away on a gurney.
Enough is enough.
It's now been several months, and I have no intention of going
back. If ever I return to engineering, I shall avoid any company
which is or makes noises about becoming ISO. I shall negotiate
a clause into my contract stipulating that should the company
choose in future to go ISO, I get a year's pay severance. But
I probably won't go back to engineering. I'll just putter around
my gardens; if I keep the hedges high enough, maybe the ISO squads
won't find me.
Stephen Hanka suggests that ISO 9000 needs
to be looked at as not just about certification
I read your ISO 9000 article in Embedded Systems Programming
with some amusement and chagrin. You are correct in stating that
in many places the way the standard is implemented is ridiculous
and serves no apparent purpose except to create paper. However,
this is due to the fact that it is often mandated for purposes
of certification instead of for its intended purpose which is
to provide a framework for quality improvement. I have worked
in more than one place where the ISO 9000 process was viewed as
A problem with quality systems and processes in general is that
they have to be embraced by management as a quality system not
as another government regulation. The idea of a quality system
is to establish a baseline from which the quality of products
can be measured and improved, not to establish an audit trail
for high priced audit firms. The audit trail should be a secondary
artifact of a well managed process. I would speculate that most
companies embrace ISO 9000 because it is forced upon them, that
they have no personnel trained in quality assurance, measurement,
or improvement, and that they perceive no incentive to do more
than the bare minimum required to pass an audit.
As you mentioned in your article, the standard is very vague
and it is difficult to eliminate flawed variations. What you failed
to state was that it is deliberately vague so that organizations
can tailor it to their own ways of doing things. It is understood
that there is no single correct way to build all classes of products
and the standard is written so that almost any complete quality
system can fulfill its requirements provided that the system is
documented and monitored. Indeed, in the design controls section
which is most applicable to software development, the standard
allows for modification of the process on a project by project
basis, provided that the changes are planned and that the minimal
documents required by the standard are produced and maintained.
In the case of software development processes, any of the models
currently employed such as the waterfall model, the spiral model,
or even Extreme Programming, can easily fit the design control
sections of the standard. All of the models include the required
steps for planning, requirements definition, design, implementation,
and test. The standard only requires that there be artifacts from
each of the stages and that they be reviewed and verified, and
that changes be approved. The documentation need not be useless
and voluminous; it only has to exist so that it can be audited
periodically. If an organization can make the quality part of
the system work then it should easily pass an audit.
The problem, at least as I see it, is one of sales and marketing.
We aren't selling quality improvement to corporate management,
we are pushing ISO 9000. It isn't viewed as an opportunity or
method for making cheaper and more competitive products; it is
viewed as another expense. If a company's management isn't interested
in making a commitment to quality improvement and doesn't support
it then you are right, the ISO 9000 certification is largely a
waste of time and money.
Steve Hanka, Qwest
Larry Bannister, a TickIT auditor
gives a robust defence of the ISO 9000 standard
I could not help but
respond to your diatribe against ISO 9000. Although there are
some points in your article I would heartily agree with there
are also some statements that I strongly disagree with. I have
been a Lead TickIT auditor and worked for Lloyd's Register Quality
Assurance (LRQA) from August 1995 to January 2001. I have audited
hundreds of companies: small companies, large companies, privately
held enterprises and DOD subcontractors and everything in between.
Everything from shrink-wrap products, to database apps, to sophisticated
embedded systems have come under my review. And yes, I have audited
things far afield from software like machine shops, in vitro medical
products, chemical plants, etc. I have a degree in Chemistry,
have done grad work in Electrical Engineering and Bioengineering,
worked as an electrical engineer, but most of my career has been
taken up doing software development and software test (20+ years).
I only bring up my qualifications so you know that the background
of a TickIT auditor is typically broad and significant experience
in software development and/or software test is required. (I can't
speak for the typical background of a non-TickIT auditor).
1) I will start at
the end of your article and hit what I think is the most important
point: incorrect implementation. This is indeed the reason for
the kind of irritating and counter-productive practices you have
cited. However, to answer your question of "how to know when it
(ISO 9000) is being used the wrong way" I think I can offer some
guidance. The answer is really the answer to this question: "What
do you need to produce a good quality product?" I would suggest
this "minimum set" of activities: a plan/schedule, a requirements
statement, some design work, decent coding practices, decent reviewing
practices for your designs, adequately designed tests, tracking
of test results, a good source control system and some way to
keep track of the most recent versions of your documents. In the
area of design, ISO 9000 doesn't ask you to do too much more than
this. I don't think any of these are too much and I have been
on many projects where the absence of one or more of these led
to disaster. If you can think of some way to efficiently encapsulate
the way you will use these things then you have a good set of
practices. If you can think of a way that will make these hard
to use or creates a nightmare of overhead then you have a bad
set of practices, i.e. an incorrect implementation.
I hear many objections
whenever I bring up this rebuttal. One is: "the procedures don't
allow us to do so-and-so the way we want to, we are stuck with
this stupid way of doing things". Sometimes this is truly the
case, in which case I think that someone in the quality department
has created some kingdom that he/she is protecting and won't allow
constructive change to occur. The majority of the time, though,
I find that the developers themselves have not been involved in
the process of creating the practices/procedures. Another objection
is: "when we do those things the project just bogs down in paperwork."
If the system is that hard to manipulate (like your example of
the ponderous method of applying for a document number and the
duplication of effort that followed) then the system needs to
have some fat trimmed from it. However, more often than not, I
get this from people that don't like to document anything at all,
that just want to "get the job done." I find it hard to work with
people that don't document things or keep up their documentation
because, in a team environment, inevitably I will write some code
that is supposed to interface with theirs and do so depending
on information from their document, only to find out that they
have changed their code but not updated their document. Their
"getting the job done" has impeded my progress and slowed down
the team as a whole.
2) You state that
"the motivation behind ISO 9000 is to produce documentation for
audit purposes." I would counter that documents are the products
of those "minimum set" of activities that I identified in my first
point above. If you don't have a written set of requirements then
you don't know what you are producing and you have no basis for
test. If you don't document your design you have insufficient
detail to proceed to coding. I could say similar things about
the documents produced for each of the elements in that "minimum
set". If you are producing documentation beyond what you need
to do your job (refer to your comment on "who reads it all"),
then you might have a case. However, if you think there is too
much documentation required in your system then you should change
your system. The ISO 9000 requirement for activities to be auditable
(i.e. produce documentation) dovetails quite nicely with an expectation
on the part of many "stakeholders" in the company that developers
will take the time to practice due diligence to adequately document
their design and development activities.
3) You state that
"audits performed after the fact based on paperwork generated
for auditing purposes achieves little" and that "processes should
be audited by watching what actually happens". This is patently
untrue and a rather fanciful wish and, ultimately, a cheap shot.
If I come into a company and review a particular development program
and I find that there is no schedule available, no requirements
available, no source code control system in use, no tests being
written, no test results available, etc, I am not going to think
that these people are efficient and super-productive; I am going
to wonder what the hell these people are doing. You HAVE TO produce
documents of the type that are auditable in the normal course
of running a project. If you don't then the developers and testers
don't know what "truth" is on the project and the whole thing
spins out of control, especially in a team environment. For instance,
in one of the worst projects that I was ever on we had multiple
versions of the software requirements statement and never did
produce half of the originally planned design documents. This
was so because there were a large number of developers (and some
management) that just wanted to "get the job done." You can imagine
what kind of chaos existed on this team and, of course, the project
ended in failure. On the other hand, given that you run a project
well, then you will be producing the documentation of the type
that ISO 9000 requires anyway, so what's the big deal? As to "watching
what actually happens" if this isn't what "actually happens" then
I don't know what is.
4) You state that
"If an auditor flunks a company, they lose a customer. So it's
not surprising that few companies ever lose their certification."
Maybe this might be true of other certification bodies, but it
wasn't true at LRQA. Personally, I "flunked" one out of every
three or four companies on their first certification audit. I
raised major non-compliances at certified companies a number of
times, and these are threats to keeping a certification. I know
of more than one certified company that LRQA had to begin proceedings
for withdrawal of certification because they did not resolve these
Now, as for what I
5) Companies do indeed
create ISO 9000 quality systems that are administrative monsters;
unwieldy and ultimately counterproductive turkeys that are millstones
around everyone's neck. Why they do so is beyond me because it
is not required by the standard, they cost too much to maintain,
they don't contribute to quality and, (from a personal point of
view) they are harder to audit. Every time I saw a procedure that
was outdated, a form that was unnecessary or a process step that
did not add much value I would at least recommend that they get
rid of the offender and sometimes I'd raise a non-compliance.
It is quality systems of this type that generate all the myths
of ISO 9000 being inflexible and stifling of innovation.
6) Companies do indeed
miss the point often, i.e. "by setting certification as their
goal, businesses neglect more important objectives." I can't tell
you how many companies that I was forced to recommend for certification
because they operated a quality system that was compliant to the
ISO 9000 standard but did nothing to improve quality, much less
improve their bottom line. In my opinion these companies wasted
their money AND missed a real opportunity for improvement. These
companies don't really care if there is any kind of quality system
in place, they just want a piece of paper on the wall.
7) Quality staff that
understand ISO 9000 but don't understand software/technology only
add a superficial layer of quality. To really contribute to the
quality of a software project you must understand the processes
that are used in the development of software. This ultimately
means that you must have a software professional that is trained
in quality for these positions, not a quality professional that
is trained in software.
8) Although you are
a bit more cynical than I am about this, I will have to agree
that ISO 9000 has become big business. I think this has led to
slight increase in overall quality in other industries, mainly
because of the emphasis on process. It has not led to an increase
in overall quality in the software world mainly because of reasons
cited in (6) and (7) above. Two illustrations of this: a) Companies
will shop around and find the consultant that will write their
quality systems for them (rather than write it themselves) at
the lowest price and will find a certification body that has requirements
lax enough to allow them to pass with the least amount of hassle.
How can you say that a company like this has changed anything,
much less improved their quality by being ISO 9000 certified?
b) The companies that I know of that were at risk of losing their
certification by LRQA if they didn't address their non-compliances
responded by simply moving to another certification body. This
is a failing of the ISO 9000 world which you did not point out
and probably didn't know about.
Well, I could write
a lot more on this subject but I'll let you go with this: the
problem is really not ISO 9000, it is the way that we deal with
software quality and process. No one wants to pay the real price
of quality, which is to change the way we do things. Consider
how many quality processes, systems, approaches and gurus we have
against the truly rare occurrence of high quality software.
Bob Scheafer also defends the ISO process,
or at least some aspects of it.
I liked your article
and have some comments, but I don't want to look like a fool or
for you to take me as an idiot, so please try to bear with me.
I feel that your problems
with ISO 9000 are more with how it has been implemented, and the
management employee relationship rather than with the intent of
ISO. A "learning organization" (I use quotes around learning organization
because I read the term somewhere and can't remember where) is
one that is supposed to listen to its employees and learn from
its mistakes. That is, if a process is stupid, then the process
should change. Often, management believes it does that when in
fact the relationship is backwards, that is, if the boss makes
the rules, then the rule cannot be stupid. That could be what
you meant when said when you wrote that if a document does not
add quality, then the document should not be written. Aha! If
the process says the document should be written, then automatically
and logically by the boss's definition, that document MUST add
Am I brilliant or
idiotic? (I hope somewhere in-between) I don't believe that ISO
9000 says that stupid documents should be written. Finding out
what's stupid is the job of the person who documents the process
(oops, that could be your boss).
I am curious as to
what documents that you dislike exactly are the ones you wrote
I will also point
out (you can't stop me unless you delete this right now) that
your example of writing test cases points to stupidities in the
process, not that the idea of documenting the process itself is
stupid. My rationalization is that there is nothing wrong for
one document to point to another rather than to need to write
the same table, paragraph, whatever twice -- just like using an
html pointer if the documents are on-line. The process could have
been modified to say this. Also, the random number? Anyone could
have provided a tool that created random numbers, and the process
could say "Run the Random number generator, and apply the data
to the test input". The problem is not the process, but the person
who writes the steps in the process. Similarly, applying for unique
document ID's can be automated. If your company does not listen
to its workers as to the best way for work to be done, then the
company deserves what it gets! (And that explains an awful lot
of crap going on, doesn't it)
I also think you are
missing the big picture, that if something fails in the field
miserably, and someone takes the effort to diagnose the problem,
and it points to testing, the tests can be exactly rerun to see
if the problem is in improper testing, or manufacture, or whatever,
but only if there is enough documentation to rerun the test. If
no one will ever need to re-test in the future, or will ever want
proof that something was ever tested, then why not throw out testing
I hope this wasn't
all a dreadful waste of electrons. I am a programmer, not a boss,
and I have to work under process, and some of it yes, sucks, but
I have also seen quality go up when done right. But noooo, there
are no good metrics that measure it. We are not manufacturers
we are artisans and everything we do is unique. Think of process
as the net and court lines in tennis.
I like my letter so
much, I think I will send myself a copy.