"Performance-Based Assessments: External, Internal, and Self-Assessment Tools
for Total Quality Management" by Paul F. Wilson and Richard D. Pearson,
published by ASQC Quality Press (611 East Wisconsin Avenue, Milwaukee,
Wisconsin 53202), 1995, 202 pages, $33 list, $30 ASQC members
 
A Book Review by Norman C. Frank, PE, CQE, CQA
CER Corporation, Washington, DC
 
"Performance-Based Assessments" provides a process for assessing processes
that goes beyond looking for compliance and beyond looking for effectiveness. 
Although "performance-based" has become one of the latest buzzwords in the
field of assessments (includes assessment, audit, appraisal, and
surveillance), the fundamental concept is good and has much to offer to
improve the processes of your company.  Considerable effort is made to define
compliance and effectiveness assessments, yet there is no obvious definition
of performance-based assessments.
 
The basic premise is that performance-based assessments should not be
conducted unless at least one hoped-for positive outcome is identified.  The
ultimate success of the effort will be judged by a simple criterion: Did we
find some way to do things better?
 
Performance-based assessments can be classified broadly as process, product,
or system and are concerned with those attributes of products or services
reflecting customer needs and expectations.
 
The performance-based approach to assessments:
 
         requires direct observation of activities
         focuses on processes or activities most directly impacting
          operations, safety, reliability, or maintainability
         includes an evaluation of not only whether, but to what extent,
          product or service goals are being achieved
         requires technical expertise or knowledge of operations to evaluate
          the selected component/system, activity, or program
         adds the step in which the significance of any recommendation or
          finding is analyzed and documented
         adds the process whereby the underlying or apparent cause(s) of
          less-than-adequate performance is analyzed
 
Unfortunately, sometimes it is not possible to directly observe the process
or activity.  Then the assessor must resort to review of past data.  "Lines
of inquiry" replace the more common checklists to allow for a broader look at
a process or activity.
 
There were several weaknesses in the book:
 
    The book tries to cover too much in the space allowed.  As a consequence
     it gives little help in how to perform the assessment, although it does
     give some help in how to organize the results of the assessment.  There
     are few examples to clarify the concept or approach.
 
    Missing from the book is a clear description of who does what part of
     the process.  Even a section on the options available would add to the
     book.
 
    The book assumes that you have a broad base knowledge to draw from
     (e.g., the book mentions a Type II error in management decisions and
     does not elaborate).
 
    There is no clear training to be given to assessors performing
     performance-based assessments, yet the authors state "performance-based
     training is a highly effective means of ensuring that personnel receive
     proper training to conduct their work safely and efficiently."  There is
     a fair amount on "performance-based training", but nothing on the
     content of training for performance-based assessors.  The only item I
     looked up in the index, "trained auditors and surveillors, 52", did not
     have anything on page 52 about trained auditors and surveillors.
 
    This book presents a process that is different from the process used by
     other parts of the DOE to perform performance-based assessments.
 
    There is considerable discussion on creating the best organization and
     culture.  Again there is not enough information to allow you to take
     action, plus there is only one supporting reference for the chapter on
     "management".
 
    A process of "pulse point auditing" as a tool for cutting the time of
     performance is suspiciously similar to "key point auditing" that has
     been around for years.
 
    For some reason, the authors coin the word "surveillors" without apology
     yet apologize for using the word "doability".
 
The authors' concept of performance-based assessments includes getting to the
causes; solving problems; generating alternatives; evaluating risk or
importance; considering cost and schedule; making change happen; and
implementing the recommendations.
 
This book gives you one group's concept of what a performance-based
assessment consists of.  This book is good for an overview of the
performance-based assessment concept.  Recommended for someone not familiar
with performance-based assessments who would like to get a feel of what it
entails.
----------------
Mr. Frank has over 25 years experience in the field of quality, in the areas
of nuclear quality assurance, research and development, and consulting.  He
is currently in Washington, D.C., with CER Corporation out of Las Vegas,
Nevada.