00.H1149.FM(i-viii).qxd:00.H1149.FM(i-viii).qxd

5/31/11

10:08 AM

Page i

Concepts for R&R Studies Second Edition

00.H1149.FM(i-viii).qxd:00.H1149.FM(i-viii).qxd

5/31/11

10:08 AM

Page ii

Also available from ASQ Quality Press:

If Concepts for R&R Studies, Second Edition was of interest to you these publications may also be of value: An Introduction to Design of Experiments: A Simplified Approach Larry B. Barrentine Reliability Methods for Engineers K. S. Krishnamoorthi HALT, HASS & HASA Explained: Accelerated Reliability Techniques Harry W. McLean The Handbook of Applied Acceptance Sampling: Plans, Procedures, and Principles Kenneth S. Stephens Statistical Quality Control Using Excel (book plus software) Steven M. Zimmerman, PhD, and Marjorie L. Icenogle, PhD The Desk Reference of Statistical Quality Methods Mark L. Crossley Statistical Process Control Methods for Long and Short Runs, Second Edition Gary K. Griffith Glossary and Tables for Statistical Quality Control, Third Edition ASQ Statistics Division

To request a complimentary catalog of ASQ Quality Press publications, call 800-248-1946, or visit our Web site at http://qualitypress.asq.org .

00.H1149.FM(i-viii).qxd:00.H1149.FM(i-viii).qxd

5/31/11

10:08 AM

Page iii

Concepts for R&R Studies Second Edition Larry B. Barrentine

ASQ Quality Press Milwaukee, Wisconsin

00.H1149.FM(i-viii).qxd:00.H1149.FM(i-viii).qxd

5/31/11

10:08 AM

Page iv

Concepts for R&R Studies, Second Edition Larry B. Barrentine Library of Congress Catag-in-Publication Data Barrentine, Larry B., 1938– Concepts for R&R studies / Larry B. Barrentine.—2nd ed. p. cm. Includes bibliographical references and index. ISBN 0-87389-557-6 (pbk. saddle stitch : alk. paper) 1. Quality control—Statistical methods. 2. Process control—Statistical methods. 3. Mensuration. I. Title: Concepts for R&R studies. II. Title: Concepts for R and R studies. III. Title. TS156 .B43 2002 658.5'62'015195—dc21

2002010554

© 2003 by ASQ All rights reserved. No part of this book may be reproduced in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher. 10 9 8 7 6 5 4 3 2 1 ISBN 0-87389-557-6 Publisher: William A. Tony Acquisitions Editor: Annemieke Koudstaal Project Editor: Craig S. Powell Production : Gretchen Trautman Special Marketing Representative: Denise M. Cawley ASQ Mission: The American Society for Quality advances individual and organizational performance excellence worldwide by providing opportunities for learning, quality improvement, and knowledge exchange. Attention Bookstores, Wholesalers, Schools, and Corporations: ASQ Quality Press books, videotapes, audiotapes, and software are available at quantity discounts with bulk purchases for business, educational, or instructional use. For information, please ASQ Quality Press at 800-248-1946, or write to ASQ Quality Press, P.O. Box 3005, Milwaukee, WI 53201-3005. To place orders or to request a free copy of the ASQ Quality Press Publications Catalog, including ASQ hip information, call 800-248-1946. Visit our Web site at www.asq.org or http://qualitypress.asq.org . Printed in the United States of America Printed on acid-free paper

Quality Press 600 N. Plankinton Avenue Milwaukee, Wisconsin 53203 Call toll free 800-248-1946 Fax 414-272-1734 www.asq.org http://qualitypress.asq.org http://standardsgroup.asq.org E-mail: [email protected]

00.H1149.FM(i-viii).qxd:00.H1149.FM(i-viii).qxd

5/31/11

10:08 AM

Page v

Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

vii

Chapter 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1

Chapter 2 Procedures for Performing an R&R Study . . . . . . . . . . . . . . . . . .

5

Chapter 3 Measurement Capability Indices

. . . . . . . . . . . . . . . . . . . . . . .

11

Chapter 4 Planning an R&R Study . . . . . . . . . . . . . . . . . . . . . . . . . . . .

17

Chapter 5 What Do We Do If the Results Are Unacceptable?

. . . . . . . . . . . . .

23

Chapter 6 How Do I Compare R&R Results to the Device Manufacturer’s Specs? . .

25

Chapter 7 Evaluation by Analysis of Variance . . . . . . . . . . . . . . . . . . . . . .

27

Chapter 8 What Are the Relationships of the Measurement Capability Indices and Process Capability? . . . . . . . . . . . . . . . . . . . . . . . . . . . .

33

Chapter 9 Examples of Problems and Procedures . . . . . . . . . . . . . X-Ray Gage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Electronic Width Gage . . . . . . . . . . . . . . . . . . . . . . . . . . Electronic Temperature Equipment . . . . . . . . . . . . . . . . . . . . Chemicals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sheet Flatness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Physical Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Profilometers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Micrometers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scales . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bore Gage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nuclear Moisture Gage . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . .

37 38 38 39 40 40 41 42 43 44 45 46

Chapter 10 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

49

Index to Appendixes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix A The Standard Procedure . . . . . . . . . . . . . . . . . . Appendix B Factors for Varying Sample Sizes . . . . . . . . . . . . . Appendix C Derivation of Repeatability and K1 and K2 Factors . . . . Appendix D Derivation of the Calculation of Reproducibility . . . . . Appendix E How to Address Variation within a Sample . . . . . . . .

51 52 53 54 57 60

v

. . . . . . . . . . . .

. . . . . .

. . . . . . . . . . . .

. . . . . .

. . . . . . . . . . . .

. . . . . .

. . . . . . . . . . . .

. . . . . .

. . . . . . . . . . . .

. . . . . .

. . . . . . . . . . . .

. . . . . .

. . . . . .

00.H1149.FM(i-viii).qxd:00.H1149.FM(i-viii).qxd

vi

5/31/11

10:08 AM

Page vi

Contents

Appendix F Confidence Levels in Estimating Standard Deviations Appendix G Use of Control Charts . . . . . . . . . . . . . . . . . Appendix H Relationship of Process Capability and Measurement Capability Indices . . . . . . . . . . . . . . . . . . . . . . . . . Appendix I The Effect of R&R on Process Capability . . . . . .

. . . . . . . . . . . . . . . . . .

64 66

. . . . . . . . . . . . . . . . . .

68 70

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

73

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

75

00.H1149.FM(i-viii).qxd:00.H1149.FM(i-viii).qxd

5/31/11

10:08 AM

Page vii

Preface

T

he intent of this manual is to provide background on the origins of the R&R methodology; help for those who work with devices that don’t fit the usual approach; and ideas for measurement devices that require innovation to assess their performance under off-line, static conditions. The initial utilization of the R&R format is only briefly described; the reader can learn more about how to apply the procedures in 10 minutes of trying them than from several pages of instruction. R&R analysis is not a complete measurement assessment. However, it is often easier to start with the limited scope addressed by R&R. Once the R&R is assessed, the need to study further measurement issues can be evaluated. The ultimate objective is to determine how best to improve the control and performance of a process. The reader is assumed to be familiar with basic control charting methodology since assessment of statistical control of the measurement process is important. The material is presented at multiple levels. One reader may concentrate on the “how-to” sections; another may wish to consider the derivations; a third may be interested in the interrelationships of measurement and process capability. Those who are interested in a more rigorous approach should review the appendices and references. Since this manual was first published, several changes and options have evolved in the way R&R studies are analyzed. This revision updates the material in these areas: 1. The original format and procedures were based on a stand-alone form, the “GM long form.” Current availability of computer makes the use of a prepared form unnecessary. Spreadsheets are faster and easier, providing an easily filed and maintained record. The forms that were originally so useful now simply provide a sample format for spreadsheets. That format is referred to here as the “standard procedure.”

vii

00.H1149.FM(i-viii).qxd:00.H1149.FM(i-viii).qxd

viii

5/31/11

10:08 AM

Page viii

Preface

2. The original R&R assessment criterion was made against the product tolerance. Today the assessment is typically calculated against the estimate of process variation. That measure is independent of the specifications and the criteria are more statistically defensible. Since these two assessment techniques answer very different questions, both are presented and the advantages of each are discussed. Preference is given to the assessment against the process variation. 3. The simplest analysis method uses ranges, and that classic approach remains preferred in this manual. Statistical packages are available, however, that can do the equivalent analysis using analysis of variance (ANOVA). Such packages provide a subtle option in the model used, involving the interaction of operator and sample. The application of ANOVA techniques is discussed with an example of typical output. 4. The interrelationship of the two criteria for R&R with process capability, represented by , is shown to better illustrate the broad view needed to keep R&R priorities in perspective. It is the author’s contention that decision making on and evaluation of measurement systems should be done in the context of process capability. The particular criterion used for measurement capability is less important than the full context of measurement and process variation. Hopefully some of the points introduced here will help in process improvement efforts. Larry B. Barrentine 8724 Warm Springs Way Knoxville, TN 37923 (865) 692-9950

1 Introduction

J

ust what is an “R&R” study, anyway? Well, it isn’t a study of “rest and relaxation!” R&R—repeatability and reproducibility—studies analyze the variation of measurements of a gage (repeatability) and the variation of measurements by operators (reproducibility). To understand why this is so important, recall that the goal of process control is reduction of variation in the process and ultimately the product. To address actual process variability, the variation due to the measurement system must be identified and separated from that of the process. Studies of measurement variation are a waste of time and money unless they lead to action to reduce process variation and improve process control. Since you can’t address something that cannot be measured precisely, the assessment of the gage becomes an early priority. Before we can continue discussing R&R, we have to define “gage.” When asked to name a gage, people typically think of micrometers, pressure gages, temperature gages, and so on. However, the term “gage” actually refers to any device used for making measurements. In this manual, the “gage” and “device” are used interchangeably and refer to any device or equipment for making a measurement. Every observation of a process contains both actual process variation and measurement variation (Figure 1.1). In the case of measurement systems, the sources are: 1. The gage 2. The operator 3. The variation within the sample Gage variability can be broken into additional components, such as: 1. Calibration (Is the gage accurate?) 2. Stability (Does the gage change over time?)

1

2

Concepts for R&R Studies

3. Repeatability (Is there variation of the gage when used by one operator in a brief time interval?) 4. Linearity (Is the gage more accurate at low values than at high values or vice versa?) Variation within a sample is a part of process variation that is often mixed with measurement variation. As noted, R&R studies assess reproducibility (operator variation) and repeatability (gage variation). Repeatability is the variation observed when an operator measures the same sample with the same gage several times. Reproducibility is the additional variation observed when several operators use the same gage to measure the same sample. The combination of both sources of variation is referred to as R&R (see Figure 1.2). Note that R&R does not address the total measurement system but is narrowly defined and is gage specific. The exclusion of the other potential sources of measurement variation does not imply that calibration, stability, or linearity are unimportant; it is just that those sources are ordinarily less significant in their impact. For that reason, R&R are often studied and quantified first. In order to improve them, you must address the key measurement process variables via procedures, standards, training, and appropriate studies. We plan and execute R&R studies in a fashion designed to avoid confusion with sources of variation other than repeatability (gage) and reproducibility (operator). While this manual describes how to perform R&R studies, you cannot ignore the other sources of variation for long. In particular, the actual process variation is the ultimate subject to be addressed. Customers require both R&R studies and process capability. Process capability includes both process variation and measurement variation. Consequently, R&R studies should be accompanied or quickly followed by evaluations of calibration, variation within the sample, and any other relevant source of variability. Variation within the sample being measured is often difficult to exclude from the R&R study. While not attributable to measurement, this source is extremely important and Observed process variation

Actual process variation

Long-term process variation

Short-term process variation

Repeatability

Measurement variation

Variation within a sample

Calibration

Figure 1.1 Possible sources of process variation.

Variation due to operators

Stability

Variation due to gage

Linearity

Chapter One: Introduction

Repeatability

3

distribution of repeated measurements on the same part by one operator with the same gage

+ Reproducibility distribution of the averages of many operators using the same gage

= R&R

the combined effect of gage variation among operators

Figure 1.2 Repeatability, reproducibility, and R&R.

should always be pursued with diligence. It not only has relevance to understanding R&Rs but also provides vital information on how to gain process capability improvements. A specific example of variation within the sample is apparent in measurements of surface texture by a profilometer. The test piece itself is sufficiently variable that if the measurement is made at a random position, the variation within the sample will inflate the estimate of repeatability. It is necessary to identify and measure this variability within the sample; this is not the role of an R&R study alone. Appendix E discusses this issue in more depth. The key point is to make certain that process variability within the sample does not intrude on an R&R study if it can be avoided. Determination of an unsatisfactory R&R should always lead to an evaluation of whether variation within the sample is part of the problem. The impact of any environmental conditions needs to be evaluated. That is more appropriately addressed by designed experiments. In R&R studies an effort is made to block out such sources of variation. It is necessary to introduce the mathematical version of Figure 1.2 since this relationship is used repeatedly. To add distributions, one must add the variances or σ2’s of the distributions being added. If the distributions or spread due to repeatability and reproducibility can be characterized by their respective sigmas (σrepeat and σrepro), then combining these distributions as in Figure 1.2 results in the following distribution for R&R: σ 2R& R = σ 2repeat + σ 2repro The sigma for R&R (σR&R) is the square root of this expression. This same Pythagorean relationship will be used in the appendices to relate the observed process variation to the measurement variation and the actual process variation; that is, σ 2observed _ process = σ 2actual _ process + σ 2measurement

02.H1149.ch02(005-010).qxd:02.H1149.ch02(005-010).qxd

5/31/11

10:12 AM

Page 5

2 Procedures for Performing an R&R Study

T

he standard manner of performing an R&R study follows procedures that were presented originally as the “General Motors long form.” This method is referred to in this manual as the “standard procedure.” A minimum of two operators and 10 samples are used for the study. Each operator measures each sample at least twice, all using the same device. Each round of measurements on all the samples by an operator is called a “trial.” This approach allows the variability inherent in the device (repeatability) to be separated from the additional variability contributed by the variation among operators (reproducibility). The derivations of the factors used in the analysis appear in appendixes C and D. Other sources of measurement variability (calibration, stability, linearity, and variation within the sample) are not addressed in R&R studies. This approach commonly defines R&R in of an interval that contains 99 percent (5.15σ) of the theoretical distribution. Some organizations use different definitions. Some use a 99.7 percent (6σ) interval; others use a 95 percent (4σ) interval for defining intervals for repeatability, reproducibility, and R&R. Software will ask for confirmation of the interval definition. To avoid confusion, clearly note which interval is used. The original analysis techniques use ranges to estimate the results and that is still the primary analytical approach taken here. This approach uses the parameters of statistical quality control to calculate estimates of various standard deviations. R&R can also be analyzed as a designed experiment with components of variation estimated from an analysis of variance. Since many practitioners do not possess that type of statistical sophistication, the range techniques are given preference in this manual. The results are equivalent. An example of the use of the ANOVA approach is presented in chapter 7 to illustrate the options and interpretations of that approach. The criteria for assessment originally described the R&R results as a percentage of the process specifications. It has become more common to describe the results as a percentage of the process variation. The latter approach has the advantage of not being dependent

5

02.H1149.ch02(005-010).qxd:02.H1149.ch02(005-010).qxd

6

5/31/11

10:12 AM

Page 6

Concepts for R&R Studies

on specifications that may be arbitrary. However, both approaches have value and while the process variation approach is given preference, both are presented. Each adds to the knowledge of the measurement situation. The “standard procedure” format used in this manual is shown in appendix A and illustrated in Figure 2.1. 1. The procedure handles two to four operators. More than four can be used, although the study might become unwieldy. Anyone needing to use more than four operators should review appendix C to obtain the correct factors. The recommended number of operators is three. (Three operators provide better estimates of reproducibility than two while more than that become a logistics problem.) 2. Appendix B provides a modification of the factors when (# of operators) × (# of samples) is less than 16. This permits an R&R study to use fewer than 10 parts or to use a single operator by assuring that the appropriate statistical factors are used. Appendix B also provides recommendations for the minimum number of trials for the planned number of operator–sample combinations. The technical basis of these guidelines is provided in appendix C. 3. The procedures generate estimates of the standard deviations (σ) of repeatability, reproducibility, and R&R. This provides measures of variability that can indicate improvement over time, regardless of changes in specifications or process performance. In cases with one-sided specifications, only the standard deviations can be used. In addition, the 99 percent intervals are estimated for repeatability, reproducibility, and R&R. The repeatability interval is referred to as EV for equipment variation; the reproducibility interval is referred to as AV for appraiser variation. 4. There is a choice of two assessment criteria: one may use specifications, the process variation, or both as a reference. As will be discussed later, there are advantages to either approach. Their relationship with will also be defined. If a single criterion is to be used, the ratio to process variation is recommended. Using Figure 2.1 as an example, the procedure for a typical R&R study is: 1. Calibrate the gage, or assure that it has been calibrated. 2. Have the first operator measure all the samples once in random order. 3. Have the second operator measure all the samples once in random order. 4. Continue until all operators have measured the samples once (trial #1). 5. Repeat steps 2–4 for the required number of trials, making sure that previous results are not known to the operators. (Note that it is not always possible to have the operators take turns. Sometimes the operators may need to run all their trials consecutively. If so, continue, but note the warning about calibration on page 17.)

02.H1149.ch02(005-010).qxd:02.H1149.ch02(005-010).qxd

5/31/11

10:12 AM

Page 7

Chapter Two: Procedures for Performing an R&R Study

Repeatability and Reproducibility Report Aluminum analysis

Product Characteristic

Date

Upper Specification Limit

650

Performed By

Lower Specification Limit

350

Gage

50

Process Sigma

Gage Number

2

Number of Operators Oper.

A

B

Sample # 1st Trial 2nd Trial 3rd Trial 1

Range

471 484 742 765 328 326 446 433 456 454 443 450 557 552 477 479 509 508 371 384 4831 4804

2 3 4 5 6 7 8 9 10 Totals

SumA – XA – RA – RB – RC – RD

13 485 778 23 2 328 13 455 2 470 7 460 551 5 2 499 513 1 13 390 81 4929 8.1

480 807 314 450 461 455 547 492 540 385 4931

SumB – XB

9860 493.0

– RA

9635 481.8 8.1 11

19.1 9.5

Sum – – R

C

1st Trial 2nd Trial 3rd Trial

Range

Range

1st Trial 2nd Trial 3rd Trial

Range

5 29 14 5 9 5 4 7 27 5 110 11 – RB

– RC

SumC – XC

# Trials

D4

d2

– R=

2

3.27

1.13

D4 =

3

2.58

1.69

*UCLR

4

2.28

2.06

– – *UCLR = D4 × R

Repeatability

D

1st Trial 2nd Trial 3rd Trial

– RD

SumD – XD

– Maximum X = 493.0 – Minimum X = 481.8 – X DIFF = 11.2

9.5 3.27 31.1

*Limit of individual Rs. Identify the cause and correct for those beyond the limit. Either a) repeat those readings using the same operator and gage, or b) discard those values and recompute _ R and UCL from the remaining observations. Reproducibility Appraiser variation (AV) = 99% range

Equipment variation (EV) = 99% range – EV = R × K1 = 9.5

AV =

× 4.56 = 43.3

(X

diff

×K2

) − ⎡⎣⎢(EV ) / (n × r )⎤⎦⎥ = 2

n = # of samples r = # of trials

2

(11.2 × 3.65)2 − ⎡⎣⎢(43.3)2 / (10 × 2)⎤⎦⎥

– σEV = R /d2 = EV/5.15 = 8.4

A.V. = 39.7 * *

σ AV =

AV = 7.7 5.15

# Trials

2

3

# of operators

2

3

4

K1

4.56

3.05

K2

3.65

2.7

2.3

**If a negative value is calculated under the square root or if there is only one operator, AV = 0.

Repeatability and Reproducibility

R &R =

(EV ) + (AV )

(43.3)2 + (39.7)2 σ R &R

σ EV σt = 100 × 8.4/50 = 16.8% σ % Reproducibility = 100 × AV σt = 100 × 7.7/50 = 15.4% σ % R & R = 100 × R &R σt = 100 × 11.4/50 = 22.8% % Repeatability = 100 ×

R&R = 99% range 2

Measurement Capability Index 1 As a Percentage of Process Variation ( σ t )

2

=

= 58.8

R &R = = 11.4 5.15

Figure 2.1 Repeatability and reproducibility format.

% R&R Acceptance Criteria MCI-1 Good Marginal Unacceptable

>0 – <20% >20%; <30% >30%

7

02.H1149.ch02(005-010).qxd:02.H1149.ch02(005-010).qxd

8

5/31/11

10:12 AM

Page 8

Concepts for R&R Studies

6. Calculate the ranges of the R&R study: • Compute the ranges of all repeated measurements by each operator. • Compute the average range. • Calculate the upper control limit (UCL) for the ranges and check for lack of statistical control. Ranges exceeding the UCL should be repeated or dropped and the calculations redone. (Reasons for such “special causes” must be pursued.) • Calculate the average results for each operator and compute the range of these averages. 7. Calculate the performance statistics of the R&R study (the origin of the K1 and K2 factors is shown in appendix C): • Repeatability—also called equipment variation (EV). This is a 5.15σ range, estimating the spread that covers 99 percent of the measurement variation due solely to the gages. • Reproducibility—also called appraiser variation (AV). This is a 5.15σ range, estimating the spread that covers 99 percent of the measurement variation due solely to the operator. • Repeatability and Reproducibility—also called R&R. This is a 5.15σ range, estimating the spread that covers 99 percent of the variation due to the combination of these two sources. • σEV , σAV , σR&R—These are the estimates of the respective standard deviations for repeatability, reproducibility, and R&R. 8. Compute the measurement capability index. Measurement capability indices indicate how “good” the R&R estimates are with respect to reference criteria. The index in Figure 2.1 shows the R&R variation as a percentage of the production process variation. This is the recommended index. Chapter 3 discusses the options and advantages of the two commonly used indices. 9. Analyze the results and develop follow-up action as necessary. Review the results of this example in Figure 2.1.

Measurement Capability Index 1—Reference to Process Variation • The combined effect (R&R) has a 99 percent range of 58.8 units. The sigma for R&R is 22.8 percent of the process sigma. This means that the distribution of measurements, based on R&R, covers 22.8 percent of the distribution of the process.

02.H1149.ch02(005-010).qxd:02.H1149.ch02(005-010).qxd

5/31/11

10:12 AM

Page 9

Chapter Two: Procedures for Performing an R&R Study

9

• The repeatability of the measurement device is estimated by a 99 percent range (EV) that is 43.3 units. The sigma of repeatability is 16.8 percent of that of the process. • The reproducibility (operator effect) is estimated by a 99 percent range (AV) that is 39.7 units. The sigma of reproducibility is 15.4 percent of the sigma of the process. The decision criteria for index 1 are defined in chapter 3. Any result above 30 percent is considered unsatisfactory and needs improvement.

RELATION OF THE PROCEDURE TO CONTROL CHARTS Although appendixes C and D show the derivations of the procedure, let’s make sure that the relation to control charts is clear; for example, the ranges of each operator from Figure 2.1 are plotted in Figure 2.2 as a control chart. Those ranges measure the repeatability of each measurement by the same operator. This control chart allows estimation of the standard deviation of the measurements (repeatability) for each operator as well as a visual comparison of operator performance for precision. Thus: σ repeatability =

R 9.5 = = 8.4 d2 1.13

EV is defined as 5.15 σ repeatability . Then EV = 5.15 × 8.4 = 43.3

80 70

Ranges

60 50 40 UCL = 31

30 20

_ R = 9.5

10 2

4 6 8 Operator A

Figure 2.2 Range chart for Figure 2.1.

10

2

4 6 8 Operator B

10

02.H1149.ch02(005-010).qxd:02.H1149.ch02(005-010).qxd

10

5/31/11

10:12 AM

Page 10

Concepts for R&R Studies

Compare these results for repeatability to Figure 2.1. It should be obvious that the procedure there is set up to facilitate these calculations without a control chart. While the procedure streamlines the calculations, the control chart is still very helpful in detecting special causes or “lack of control” in the measurement process. In the format of Figure 2.1, the upper control limit for the ranges is computed to at least allow a quick check. When a range shows to be a “special cause,” those observations should be retested or dropped from the calculations per standard SPC procedures. The causes of such lack of control should be explored as in any control charting application. Refer to appendix G for a more detailed example. – Question: Why was an X-chart not also shown? Because the samples were selected to be different. Most of the averages for the 10 samples should be outside the control limits of – the X-chart. If they were not, there would be a serious problem of lack of measurement discrimination. In other words, the measuring device would be unable to tell the samples apart! For an excellent discussion of this problem, refer to Wheeler and Lyday (1989).

03.H1149.ch03(011-016).qxd:03.H1149.ch03(011-016).qxd

5/31/11

10:11 AM

Page 11

3 Measurement Capability Indices

O

nce the data from an R&R study have been analyzed, the results must be evaluated to determine the adequacy of the measurement system. This is done by computing a measurement capability index (MCI). There are at least four different ways of calculating capability indices for measurement systems, whether R&R or general measurement. The two most commonly used will be discussed and compared here. A single index is not sufficient to provide a complete assessment of the measurement situation. The measurement capability index (MCI) should be evaluated in conjunction with the process capability.

1. MCI1—Measurement Capability Index as a Percentage of Process Variation MCI1 = 100 ×

σ R& R where σ t is the estimate of the process variation. σt

It is important to note that σt should be estimated from a process control range chart, using σt =

R d2

Some references use the variation in the samples of the study to estimate this variation. Such a procedure is not recommended since it is difficult to appropriately represent process variation with the small samples used in a measurement study. Samples are better chosen to represent the range of measurements anticipated, rather than trying to select them to define a large population. MCI1 is the most commonly used index since it indicates, as shown in Figure 3.1, the comparative size of the distribution of the measurement system to that of the process being

11

03.H1149.ch03(011-016).qxd:03.H1149.ch03(011-016).qxd

12

5/31/11

10:11 AM

Page 12

Concepts for R&R Studies

6σR&R 6σt

Figure 3.1 R&R as proportion of process variation.

measured. Since the measured process variation is a combination of the true process variation and the measurement variation, this index indicates the degree of distortion in reporting the process variation due to measurement. This index is independent of the specifications and indicates the clarity with which the process can be viewed. When a one-sided specification is used, it is the only index that can be used without artificially creating a specification. MCI1 has the disadvantage of the component values not being additive. That is, the index for repeatability plus that of reproducibility does not equal the index for R&R. (This is a result of using the sigmas rather than the squares of the sigmas.) Despite this drawback, this index is clearly communicated and provides a good indicator of the adequacy of the measurement system. The index is most meaningful for R&R, the combined gage–operator variation. Obviously, if R&R is adequate, the separate calculations for repeatability and reproducibility are of limited interest. However, the calculations can be made for the separate components (repeatability and reproducibility) to indicate the comparative influence of those two sources. This provides direction for improvement activities when the R&R index is not adequate. The criteria for assessment of this index are customarily: ≤ 20%

good

> 20%, ≤ 30 %

marginal

> 30 %

unacceptable

While these criteria are the “customary” guidelines, evaluation of their discrimination ratio provides an acceptable value (approximately 4.6 for a criterion of 30 percent). Discrimination ratio (refer to Wheeler and Lyday [1989]) describes the number of categories that a measurement system can determine. The ratio needs to be in excess of 4.0 to be effective.

03.H1149.ch03(011-016).qxd:03.H1149.ch03(011-016).qxd

5/31/11

10:11 AM

Page 13

Chapter Three: Measurement Capability Indices

13

An “unacceptable” rating implies that measurement distorts the apparent process variation and there is a need to improve the measurement system before one can address the production process. This result should also be viewed in conjunction with the process capability index as represented by the process , the indicator of the adequacy of process spread. An “unacceptable” MCI is an opportunity to improve; however, if the is adequate, improving measurement may not be an immediate priority. A poor index means that improved measurement performance will improve the observed process capability; a good index means there is little benefit to further reducing the measurement variation. This is the preferred index since it is not dependent on specifications and its criteria can be defined statistically.

2. MCI2—Measurement Capability Index As a Percentage of Process Specifications MCI2 = 100 ×

5.15σ R& R , where the specifications are process or product specifications. USL − LSL

This index, illustrated in Figure 3.2, shows the size of the measurement variation compared to the tolerance (specification) range. (The first edition of this manual used this index exclusively since that was the common practice at that time.) This index remains the easiest to explain and still has value in indicating the status of the measurement variation with respect to the process specifications being applied. The problem with this index is that specifications are not necessarily set reasonably nor are they consistent from one application to another. Specifications that are either too generous or too tight may lead to misleading interpretations and comparisons of measurement systems. This index shows the ability to classify product against the specifications, while index 1 is indicating the distortion in the process variation due to measurement. These are two different questions. The utility of MCI2 is illustrated in Figure 3.3. LSL

USL

5.15 σR&R

Figure 3.2 R&R as proportion of tolerance range.

03.H1149.ch03(011-016).qxd:03.H1149.ch03(011-016).qxd

14

5/31/11

10:11 AM

Page 14

Concepts for R&R Studies

Spec

Low R&R

Spec

Spec

High R&R

Spec

Wrong conclusion

Wrong conclusion

Figure 3.3 Errors caused by R&R.

Figure 3.3 illustrates the range of possible values that might be reported for an actual value occurring near a spec limit. The larger the R&R, the greater is the risk of misclassification. The customary criteria for assessment of the index are similar to MCI1: ≤ 20%

good

>20, ≤ 30%

marginal

>30%

unacceptable

These criteria, while common practice, cannot be statistically defined. While the index is a very descriptive indicator, its meaning is dependent upon the specifications. As with MCI1, the index is applied to R&R initially and to the components only if the R&R is not acceptable. The index should be interpreted in conjunction with the process capability index ( ) to better establish the priority for measurement improvement. If the variable being measured has only a single specification, the tolerance cannot be defined and this index cannot be used. The tolerance is based on product or process specifications. If the tolerance used is the manufacturer’s specification for the measurement device, the interpretation of the R&R results is quite different! Refer to chapter 6 on this subject.

03.H1149.ch03(011-016).qxd:03.H1149.ch03(011-016).qxd

5/31/11

10:11 AM

Page 15

Chapter Three: Measurement Capability Indices

15

COMPARISON OF MEASUREMENT CAPABILITY INDICES So—which index should be used? If only one index is used, MCI1 is preferred. It is more meaningful statistically, is consistent in its interpretation, and is independent of the specifications. However, that each index is answering a different question. MCI1 looks at how measurement contributes to the observed process variation. MCI2 looks to see how well one can interpret specifications for given measurement variation. Chapter 8 and appendix H show that the combination of both indices will reveal the process . Note that a single indicator is not sufficient to completely describe the status of the measurement system as part of the production process. It would be preferable to include the process with a measurement index to put the measurement process in proper context for asg priority. ( is used since it refers to the spread capability of the production process. The k includes the targeting of the process average, something outside the description of variation.) If the is good, a poor measurement process needs improvement but may not be an immediate priority.

04.H1149.ch04(017-022).qxd:04.H1149.ch04(017-022).qxd

5/31/11

10:15 AM

Page 17

4 Planning an R&R Study

T

he most important part of a study occurs prior to making any measurements. In this planning phase, the decisions made or ignored will determine the validity of the study. The plan must address the following:

1. How and when will the device be calibrated? Calibration can affect the R&R study. Ideally, the device should be calibrated before the study begins and not recalibrated until the study has ended. If this is not possible, variation due to calibration may appear in the study. a. If it is necessary to recalibrate after each measurement, then repeatability and calibration variability are combined. b. If each operator must recalibrate before each trial (set of sample tests), repeatability and calibration variation are combined. c. Sometimes scheduling of operators requires each operator to run all of his trials consecutively; that is, operator A runs trial 1 in random order, then immediately runs trial 2 in a different random order, and so on. In such cases, if each operator only calibrates before he begins, calibration variation will be combined with reproducibility. All operators must clearly understand and consistently apply the calibration procedure. It is not the purpose of this text to discuss calibration procedures. Please note, however, that any indication of significant variation due to calibration requires correction. 2. How many operators will participate? Give careful thought to the population represented by the operator(s). Some devices do not have an operator—for example, automatic gages. Treat such cases as if there were a single operator. If a single

17

04.H1149.ch04(017-022).qxd:04.H1149.ch04(017-022).qxd

18

5/31/11

10:15 AM

Page 18

Concepts for R&R Studies

operator performs all the measurements with the subject gage, then one cannot determine an operator effect and there should be no attempt to create one artificially. Where multiple operators do use such gages, the minimum number required in the study is two. For greater confidence in the estimates, the recommended number of operators in the study would be either three or four, if at least that many are in the operator population. The recommended number is three. Using more than that number is preferable statistically but generally makes the study too cumbersome. When a representative sample of operators is to be used in the study, they should be selected randomly. Ongoing studies are planned to eventually cover all operators. Operators are assumed to be proficient at their task; the operators are assumed to have equivalent skill levels. Never have poorly trained or new operators mixed in the study with skilled operators. The intent is to assess skilled operators. Problems of differing proficiencies will be apparent on the control chart of ranges. 3. How many samples will be measured? If practical, use 10 samples. However, select enough samples so that (# of samples) × (# of operators) is greater than 15. If this is not possible or practical, increase the number of trials, referring to appendix B. 4. How should the samples be selected? Parts or samples should cover the range of measurements expected. If necessary, break a measurement variable into homogeneous groupings within which the variability should be consistent and independent of the differences in samples. For example, the precision of chemistry measurements for low-carbon steel may differ from that of high-carbon. It may be necessary to form at least two groups of carbon ranges for study. Each such category should be the subject of a separate R&R study. Such categories can usually be identified by noting changes in the specified tolerances (for example, gage ranges, voltage ranges, and so on). Note that the samples do not represent the process and are not intended to. Since it is not feasible to represent the process with such a small set of samples, it is better to represent the measurement range. This means that the sample variation should not be used in the assessment procedures. 5. How many trials should be run? A trial is one measurement on all the samples by each operator. The numbers of samples and operators in the study determine the minimum number of trials. Appendix B indicates the minimum number of trials for each operator and sample combination. Two trials are usually sufficient. 6. How do I minimize variation within the sample in the R&R study? Studies of some devices may not be able to exclude variation within the sample from the measure of repeatability. In the case of destructive tests, it is often possible to minimize within-sample effects by selecting test pieces from each sample from a homogeneous and compact area of a master sample. Other cases, such as surface

04.H1149.ch04(017-022).qxd:04.H1149.ch04(017-022).qxd

5/31/11

10:15 AM

Page 19

Chapter Four: Planning an R&R Study

19

texture measurements with a profilometer, may require directing the operators to test at exactly the same point, as long as the test doesn’t change the characteristic being measured. If you believe variation within the sample has intruded or will intrude upon the study, define special studies to estimate that component of variation. This may not always be possible; if not, at least identify this source as a contributor to the R&R assessment. (Refer to appendix E.) Presence of variation within the sample will force two R&R studies to be run. The order is determined by what is known: a. If you know or strongly suspect variation is present within the sample, run the study on special standards first to avoid such variation, and follow up with a study using production samples. b. Otherwise, run the first study on production samples; follow up suspicious or unsatisfactory results with a study using special samples. 7. Do I use individual measurements, averages, or what? Generate the measurement in the same manner that it is produced in the standard operating procedure for product evaluation. If a single measurement typically is used, then do that in the R&R study. If more than one measurement is made and averaged for the official measurement, then do that for the R&R. Make sure all operators report the measurement in the same way! However, where averages are reported, record the individual measurements. The individual measurements may be useful for diagnostics. 8. How do I analyze the results? The easiest approach to analysis is to use the range procedures shown in Figure 2.1 and in appendix A. This approach is easy but is statistically adequate. The calculations are easily done with spreadsheet programs. Standard statistical packages will also perform the calculations, as well as a number of special purpose programs. With software there is an option to use either range techniques or analysis of variance. Refer to chapter 7 for a more detailed example and discussion of the comparative merits of the analysis of variance procedures. Check how the measurement capability indices are computed to assure that the sample variation is not used. The danger in using computers for the analyses is that people tend not to look at the data. Poor results should prompt a careful manual review of the data and results. The advantage of software is that the calculations are performed and filed automatically. More importantly, software will facilitate recordkeeping on devices and operations, calibration status, R&R schedules, and so on. This is a major consideration when there are a large number of devices to maintain. A graphical technique for evaluating the results is to make a control chart of the ranges of the trials for each operator–sample combination. Plot by operator and interpret as you would any other range chart. An example of this technique is illustrated in appendix G.

04.H1149.ch04(017-022).qxd:04.H1149.ch04(017-022).qxd

20

5/31/11

10:15 AM

Page 20

Concepts for R&R Studies

9. What do we do about one-sided tolerances? Some properties have a one-sided tolerance. In such cases, it is not possible to do the tolerance analysis (MCI2 ). There are two possible alternatives to apply with one-sided specifications: a. Use MCI1, using the process variation. b. Use an appropriate internal two-sided spec to calculate the tolerance and then use MCI2. Document such a criterion and explain its origin. A common error when only an upper spec is given is to assume that the lower spec is zero. While this may be meaningful in some cases, do not use this approach unless it is considered “reasonable” for that application. That is, is zero achievable? Is near-zero truly a desirable value? Does use of zero define a tolerance that is realistic with respect to the process capabilities being procured? Note that “lack of a spec” and “zero” are not the same thing. 10. How can we tell if there is inadequate measurement discrimination? “Measurement discrimination” is the ability to measure small differences reliably. If the round-off in measurement exceeds the actual measurement variation (repeatability), artificially low estimates of measurement variation (repeatability) will result. For example, if a chemical analysis is reported to the nearest hundredth and the σ of repeatability is less than .01, then the round-off error may be larger than the measurement variation. Calculation of the ranges of repeated measurements will contain a lot of zeroes. This forces the estimate of the measurement variation to be lower than it actually is, making the results look better than they are. Wheeler and Lyday (1989) provide two rules to determine if there is a problem of discrimination: a. The ranges calculated on the R&R form have three or fewer possible values less than the upper control limit b. More than 25 percent of the ranges are zero If either of these situations occurs, the measurement discrimination is inadequate and the estimates generated are suspect (probably as an understatement of repeatability). 11. What can be done about inadequate measurement discrimination? Only three options are available: a. Measure and report to as many decimal places as the measurement device permits. Limit calculations, however, to one decimal place more than the measured values. b. Seek a better measurement device, one that can measure smaller units. c. If there is nothing else to be done in the near term, you must live with it. Be sure to note that this situation exists. Its impact depends on the process capability of the variable being measured.

04.H1149.ch04(017-022).qxd:04.H1149.ch04(017-022).qxd

5/31/11

10:15 AM

Page 21

Chapter Four: Planning an R&R Study

21

12. Can an R&R study ever include more than one gage? Normally an R&R study is gage specific; that is, each individual device requires its own R&R study. Some device types, such as micrometers, have a very large number of units to be evaluated. For such cases, a shortcut procedure is described under “micrometers” in chapter 9 that reduces the amount of testing. 13. How can I identify problems that create special causes in my R&R study? Keep very good notes of the circumstances and details of the study. A journal or log would be helpful. Be present to observe! Detail any environmental conditions or note any uncontrolled factors that might create distortion in the results.

05.H1149.ch05(023-024).qxd:05.H1149.ch05(023-024).qxd

6/1/11

8:34 AM

Page 23

5 What Do We Do If the Results Are Unacceptable?

I

f the R&R results are unacceptable, by whatever criterion, it is necessary to know the source(s) of these results. Is the excessive measurement variation due primarily to repeatability (device)? Is it due primarily to reproducibility (operators)? Or is it the combination of both? Knowing the answers to these questions allows you to evaluate which of several options to pursue: 1. The unacceptable results may be telling you what you need to know: either the device, the operators, or both are not doing the job that is required. Define and perform corrective action to assure that the device and operators are doing the best that is possible. 2. Further analysis and/or discussion with the device manufacturer may lead to the conclusion that the device is state-of-the-art and improvement is not possible. If so, document that conclusion and go to item 3. 3. If the dominant source of variation is repeatability, one way to improve precision is to average more than one test for the reported measurement. Averaging reduces the repeatability portion of the R&R by the square root of the number of measurements averaged. If such precision is truly needed and no other improvements are possible, the additional testing may be the only way to achieve that precision. This approach should not be used unless the testing procedures for production material are changed to apply this same technique to routine measurements. Is poor repeatability due to poor performance by all operators? Again, review training and procedures. 4. When operators (reproducibility) are the major source of variability, address training and standardization of procedures.

23

05.H1149.ch05(023-024).qxd:05.H1149.ch05(023-024).qxd

24

6/1/11

8:34 AM

Page 24

Concepts for R&R Studies

5. The specifications may be unrealistic. Does the customer really need them that tight? Ask. 6. Was there indication of, or reason to suspect, additional variation from incorrect calibration or variation within the sample? If so, it is now necessary to know if the unsatisfactory results are the result of those components. Design appropriate studies to isolate them. See appendix E for help. 7. If the process capability is adequate (the > 1.33) despite the lack of precision, the inadequate R&R performance is not currently hindering the process and may not be an immediate priority. As process improvements are made, that situation will not last. Develop a time line to start working toward measurement improvements. If the process capability is borderline (1.0 ≤ ≤ 1.33), unacceptable R&R variation may make the difference in whether the observed process is reported as capable. If the process capability is clearly inadequate ( < 1.0), both measurement and process must be addressed. Measurement should be given priority in order to facilitate addressing the process. 8. A single index is not sufficient to clearly indicate the measurement and process status. It will always be helpful and desirable to use multiple indicators to fully comprehend the situation and priority. The two indices used for R&R assessments answer two different questions. Both questions may be of interest. The process capability measures also help understand the priority of measurement in the general improvement activity. Appendix H shows the relationship of the R&R indices and .

06.H1149.ch06(025-026).qxd:06.H1149.ch06(025-026).qxd

5/31/11

10:19 AM

Page 25

6 How Do I Compare R&R Results to the Device Manufacturer’s Specs?

S

ometimes it may be desirable or necessary to compare the results of an R&R study with the device manufacturer’s specifications. It is essential to recognize such situations, since the interpretation of the results will change. Notice, first of all, that these “specs” are not customer targets on the product or process. That is, the tolerance calculated from the manufacturer’s specs on the device is not at all the same as a customer tolerance on the product or process. Therefore, the procedure for repeatability will be the same, but the interpretation will be quite different. Using MCI2 (assessing against specifications), % Repeatability =

5.15σ EV × 100 tolerance

If the tolerance is the device spec, then the criterion of acceptability is no longer 0 to 30 percent. Instead, a ratio of 100 percent in such an instance means that the device has demonstrated what the vendor promised. A ratio of less than 100 percent implies that the device did better than promised; greater than 100 percent is worse than promised. The key to recognizing this situation is to determine what the specs mean. Typical R&R studies assume that the tolerance is based on a customer spec on the product or process. If it is not, the interpretation is drastically different. If there is indication that the device is not meeting the manufacturer’s promises, the investigation should include: a. Calibration program

d. Environmental control

b. Maintenance schedule

e. Possible overhaul by the vendor

c. Maintenance program

f. The manufacturer’s recommendations

25

07.H1149.ch07(027-032).qxd:07.H1149.ch07(027-032).qxd

5/31/11

10:20 AM

Page 27

7 Evaluation by Analysis of Variance

T

here are several statistical packages available that will perform the analyses of R&R studies. Such packages generally provide an option to use either range methods or the more formal computations of an analysis of variance (ANOVA). The results will not be exactly the same but will be equivalent. Range estimates for small subgroups are nearly as powerful as the computations from ANOVA and much easier to explain or interpret. ANOVA offers a check on operator–sample interaction that is not available with the range method and requires more statistical knowledge to interpret and explain. Due to the simplicity of the approach, the range methods have been preferred here. The following is presented to those who prefer, or only have available, the ANOVA software. There are two major differences in the two approaches: 1. Depending on the specific software, the ANOVA method may not require use of a range chart to evaluate the presence of special causes during the R&R study even though very good graphics are available. The R-chart serves to validate the stability of the measurement process and indicate problems that should be addressed immediately, before further analysis. Such occurrences are screened from subsequent calculations to better estimate the performance of a stable measurement process. 2. The ANOVA method computes the interaction between operators and samples and adds this source of variation to the reproducibility source. As noted in appendix D, range methods assume that there is no interaction between well-trained operators and the samples. In other words, all operators are assumed able to test each sample with equivalent precision. The use of well-trained operators generally is sufficient to avoid the presence of a significant interaction. The review of the control chart of ranges is also a weak check on the assumption of no interaction. Use of the interaction term provides a better estimate of reproducibility if there is an interaction. If the ANOVA procedure uses the estimate of the interaction component regardless of

27

07.H1149.ch07(027-032).qxd:07.H1149.ch07(027-032).qxd

28

5/31/11

10:20 AM

Page 28

Concepts for R&R Studies

whether that term is statistically significant, the assessment of reproducibility will be slightly inflated. If the interaction term is not reported separately due to being insignificant, some software adds that term to the assessment of repeatability. In reality, these differences in methods generally have minor impact on the evaluation of a stable measurement process. The ANOVA software may or may not provide an option on the interaction assumption. One must be aware of the particular approach of the software being used. If an option to assume no interaction is provided, the ANOVA results will be very similar to those using the range methods. If the software computes the interaction automatically or you prefer to use that option, the results for reproducibility and total R&R will be larger than the estimates of the range methods by the addition of this interaction component of variance. Note that presence of a significant interaction requires investigation to determine why it occurred! Most outputs will give you an option to have assessments using MCI1 (process standard deviation) and/or MCI2 (tolerances). These are the standard assessments discussed above. Some output formats also include an assessment against the total variation from the study. This is not recommended since this procedure assumes that the process variation can be represented accurately with the samples in the study. This is a very poor assumption for such a small sample size. The process control chart provides a much better estimate of the process variation for reference. Computer output formats vary by the software vendor, usually accompanied by good graphics. The following study is used to illustrate the typical output with the ANOVA approach.

EXAMPLE OF AN ANOVA The supplier of metal stampings is concerned about his ability to measure the depth of the marked lines in order to adequately assess his product against his customer specifications. Five samples were tested three times by each of three operators. (Note that the ANOVA method easily handles any number of samples, operators, and trials.) The data and analyses are shown in Figure 7.1, using Minitab, with a tolerance range of 250 and a process standard deviation (σt ) = 35. Operator →

Bob

Sample ↓

test 1

test 2

test 3

test 1

test 2

test 3

test 1

test 2

test 3

1 2 3 4 5

343 288 357 280 247

359 325 353 253 271

376 334 326 265 236

366 347 317 296 266

402 322 327 257 250

377 326 340 285 264

369 315 349 264 274

348 318 360 294 269

384 330 385 289 251

Figure 7.1 Data depth.

Jack

Ann

07.H1149.ch07(027-032).qxd:07.H1149.ch07(027-032).qxd

5/31/11

10:20 AM

Page 29

Chapter Seven: Evaluation by Analysis of Variance

29

The data input format will vary according to the specific software. In most cases, input is by columns. That is, there may be one column for the sample identification, a second column for operator designation, and the third column for the data. There may be a choice of output formats. Initially, the case with interaction has been selected. The results are displayed in the following ANOVA table. Two-Way ANOVA Table with Interaction Source

DF

SS

MS

F

P

Sample Operator Operator*sample Repeatability Total

4 2 8 30 44

78080.8 1210.8 2707.0 7846.7 89845.2

19520.2 605.4 338.4 261.6

57.6885 1.7892 1.2937

0.00001 0.22792 0.28403

The highlighted column shows the alpha risk. For 95 percent confidence, these values will be less than .05. Note that neither operator nor operator–sample interaction is significant. The sample variation is significant but of no interest to us. These results are repeated without the interaction since the interaction is not significant. Two-Way ANOVA Table without Interaction Source

DF

SS

MS

F

P

Sample Operator Repeatability Total

4 2 38 44

78080.8 1210.8 10553.6 89845.2

19520.2 605.4 277.7

70.2854 2.1798

0.00000 0.12700

Note that the nonsignificant sums of squares for interaction were included in the sums of squares for repeatability. This will slightly inflate the estimate of repeatability. This table is then used to generate the R&R statistics by solving the expected mean squares for the estimates of the variance components (if that statement mystifies you, refer to the text by Hicks in the reference section):

Source

StdDev (SD)

Study Var (5.15*SD)

%Tolerance (SV/Toler)

%Process (SV/Proc)

Total Gage R&R Repeatability Reproducibility Operator Part-to-Part Total Variation

17.3082 16.6652 4.6738 4.6738 46.2391 49.3723

89.137 85.826 24.070 24.070 238.131 254.267

35.65 34.33 9.63 9.63 95.25 101.71

49.45 47.61 13.35 13.35 132.11 141.06

07.H1149.ch07(027-032).qxd:07.H1149.ch07(027-032).qxd

30

5/31/11

10:20 AM

Page 30

Concepts for R&R Studies

1. The estimates of the standard deviations for R&R, repeatability, and reproducibility are shown in the second column. 2. The third column multiplies each standard deviation by 5.15 to estimate the 99 percent interval for R&R, equipment variation (EV or repeatability) and appraiser variation (AV or reproducibility). 3. Both measurement capability indices are provided: a. The highlighted column labeled “% Process” is MCI1, showing that the sigma for R&R is 49.45 percent of the process standard deviation. (This is equivalent to saying that the 99 percent range for R&R covers 49.45 percent of the 99 percent range for process variation.) This clearly exceeds the maximum acceptable criterion of 30 percent and indicates that measurement significantly adds to the true process variation. b. The column labeled “% Tolerance” reveals that the 99 percent interval for measurement covers 35.65 percent of the specification range. This indicates that decision making with respect to the specifications is not very good. 4. Comparison of the repeatability and reproducibility contributions reveals that repeatability (the gage or measuring equipment) is the dominant source of the R&R variation. Operator performance has limited potential for improvement. Substantial improvement must come from studying the performance of the gage, how it is used, maintained, and so on. 5. Note that the estimate for reproducibility is repeated for each operator. Had the operator–sample interaction been significant, reproducibility would have been split into two sources: operator and interaction. This example is interesting if we go further in our assessment. Although both indices are unacceptable (measurement variation distorts our view of the process and weakens our decision making with respect to the specifications), we need to check the of the process. (We don’t check k since it is concerned with the process target or average, in addition to spread.) Given the data above (tolerance = 250; σt = 35), = 1.19, a marginal value but not too bad. With such unanimously poor R&R assessments, how are we this good? There are two reasons: 1. The specifications are generous. This eases the priority of this measurement process for the near term. 2. The R&R variation is a major contributor to the apparent process variation. The actual process variation is substantially less than the observed process variation and is relatively small. (This means that the actual is quite good.) This unusual result is characteristic of a stamping operation. Once the operation is set, it is extremely repetitive in its performance within the limits of tool-wear. Most processes could not tolerate such measurement noise added to the natural process noise.

07.H1149.ch07(027-032).qxd:07.H1149.ch07(027-032).qxd

5/31/11

10:20 AM

Page 31

Chapter Seven: Evaluation by Analysis of Variance

31

Here we have an opportunity to improve the observed process variation and increase our ability to make decisions relative to the specifications by improving the measurement process. The urgency or priority for doing so must be balanced with the awareness that the production process capability is not in desperate condition. The point is that management requires more than a single index to establish the priority warranted in this situation. Various other subtle issues are being glossed over. The operators participating in an R&R study are assumed to be a random sample of a very large pool of possible operators. (Technically, this has to do with whether the operator term in the analysis of variance is “random” or “fixed.”) Violation of this assumption does not have major consequence in most studies. The use of confidence intervals to more statistically define significance of the R&R estimates is possible with some software. (The software from NCSS provides very detailed and extensive information on this. It is a very -friendly software.) The estimates of repeatability are much stronger than those of reproducibility. Most such debates are minor compared with the need to do R&R studies on a routine basis. R&R is intended to be an ongoing, periodic assessment or monitoring tool. It is not a “do it once and we’re done” technique! Use the method with which you are most comfortable and can explain to others. Those who wish to delve further into the details of the analysis of variance are referred to Hick’s text in the reference section.

08.H1149.ch08(033-036).qxd:08.H1149.ch08(033-036).qxd

5/31/11

10:21 AM

Page 33

8 What Are the Relationships of the Measurement Capability Indices and Process Capability?

P

rocess capability is defined by comparing the 6σ interval of a statistically stable process to the specification range or tolerance. An index of this capability is where: =

USL − LSL 6σ t

USL and LSL refer to the specification limits and σt refers to the process standard deviation. This is an indicator of what the process could do if properly centered and assesses the variation observed in the process. A similar and more pertinent index, k, indicates the capability of the process as it is currently centered. Since we are concerned with the spread in the process and not the average, is the index to compare. Recall that: σ t2 = σ 2p + σ R2 & R where σt2 is the observed process variation, σp2 is the actual process variation, and σ2R&R is the R&R variation.

WHAT IS THE EFFECT OF R&R ON PROCESS CAPABILITY? Obviously any improvement (reduction) in the R&R variation will also reduce the observed process variation, increasing the process capability index and reducing both measurement capability indices. Figure 8.1 shows the degree of distortion of the actual process due to the amount of measurement variation shown by MCI1, using the process sigma. A similar graph can be generated for MCI2, using specification as the reference.

33

08.H1149.ch08(033-036).qxd:08.H1149.ch08(033-036).qxd

34

5/31/11

10:21 AM

Page 34

Concepts for R&R Studies

Several observations are apparent from a study of the graphs in Figures 8.1 and 8.2: 1. Impact of R&R variation in either case is minor if the index is less than .3 or 30 percent. 2. The MCI1 index is more robust than MCI2. 3. At first, it may be surprising that the R&R results have a negligible effect on actual at the low values of observed . This indicates that the overwhelming problem in such cases is improvement of the process. Improvement to the measurement system, while necessary, will not correct an obviously incapable process. 4. R&R becomes more influential as the observed process capability increases to at least marginal levels. The derivations of the above graphs are shown in appendix I. 2.0 1.8

Actual

1.6 1.4 1.2 1.0 0.8 0.6 0.6

0.8

1.0

1.2 1.4 Observed

0.6

MCI1

0.5

0.4

1.6

1.8

0.3

0.2

Figure 8.1 Distortion of for MCI1. 2.0 1.8

Actual

1.6 1.4 1.2 1.0 0.8 0.6 0.6

0.8

1.0

1.2

1.4

Observed MCI2

0.6

Figure 8.2 Distortion of for MCI2.

0.5

0.4

0.3

0.2

08.H1149.ch08(033-036).qxd:08.H1149.ch08(033-036).qxd

5/31/11

10:21 AM

Page 35

Chapter Eight: Measurement Capability Indices and Process Capability

35

HOW DO THE INDICES RELATE TO ONE ANOTHER AND TO ? The three indices (MCI1, MCI2 and ) are completely linked. Any two of these will completely specify the third. This s the point that a single measurement capability index is not sufficient to fully define the measurement/process situation while any two of these provides a complete picture of measurement and process. The derivation of the relationship is shown in appendix H. Figure 8.3 displays the interrelation of the three indices. Review of Figure 8.3 reveals: 1. The two measurement capability indices used together define process capability. One index and the will define the second index. 2. A full appreciation of the process status is best achieved by a measurement index and . 3. A good can only be achieved with poor measurement capability (using MCI1) when the actual process variation is relatively small and/or the specifications are wide (reflected by a small MCI2 index). 4. The two measurement capability indices described here answer two different questions about the measurement process. Neither is sufficient to indicate the priority and impact of measurement on a specific production process. Full comprehension requires an MCI with the . 5. MCI1 will usually (careful—there are no absolutes here!) be a more stringent requirement than MCI2. That is, in the range of marginal or better, the criteria for MCI2 are more forgiving than those of MCI1.

0.70 MCI2 = 5.15 Sigma R&R/Tolerance

= 0.75

0.60 = 0.5 = 1.25

0.50

= 1.0

0.40 = 1.5

0.30 0.20 0.10 0.00 0.1

0.2 0.3 0.4 0.5 0.6 MCI1 = Sigma of R&R/Sigma Process

Figure 8.3 Relationship of and measurement capability indices.

0.7

09.H1149.ch09(037-048).qxd:09.H1149.ch09(037-048).qxd

5/31/11

10:21 AM

Page 37

9 Examples of Problems and Procedures

U

nfortunately, there are many measurement situations in industry in which standard procedures do not fit without modifications. This requires modifications to the procedures and/or calculations. Some examples are:

1. Significant variability within the sample being measured; for example, surface roundness 2. Instances in which there is no operator effect; for example, X-ray gage for thickness 3. Instruments that cannot repeat measurements on the same part and therefore must be assessed off-line; for example, width gage on a high-speed mill 4. Destructive measurements that cannot be repeated on the same item; for example, tensile tests There is always more than one way to accomplish an R&R study. The most important requirement is thought and action in addressing measurement systems in a useful and meaningful manner. The fact that a typical R&R study is not possible in a given case should not prevent deg studies that can provide such data at least in a static or calibration mode. This section contains descriptions of some procedures for performing R&R studies in unique cases. (Cases that clearly can be addressed with the standard procedures will be discussed only briefly.) The following procedures illustrate approaches applicable to many measurement devices, situations, and industries.

37

09.H1149.ch09(037-048).qxd:09.H1149.ch09(037-048).qxd

38

5/31/11

10:21 AM

Page 38

Concepts for R&R Studies

X-RAY GAGE Measurement: Thickness Procedure Thickness is measured by an X-ray unit as the material moves under the head. There is no operator influence on the X-ray reading, nor can the reading be repeated on exactly the same spot. The requirement becomes one of estimating the repeatability of the X-ray. It is not feasible to do this using production material. Instead, the test blocks used for calibration must be utilized and repeatability estimated in an off-line condition. Randomize the order of test for the blocks, and report the results to the maximum available number of decimal places. Calculate repeatability by using the standard procedure. Note that there is no operator (treat such a situation as if there were a single operator) and that the number of test blocks determines the number of trials per block. Complete the calculations as required, using the appropriate factors (in this case, from appendix B). The statistics for repeatability can be calculated, but there is no estimate for reproducibility. Since the reproducibility is zero, repeatability is R&R. Note 1: Determine the procedures used or recommended by the gage manufacturer. If they are compatible with or superior to the procedures proposed here, apply them. Note 2: It is important that the repeatability is consistent across the range of thickness covered by the test blocks. If doubt arises over this, you may need to conduct a special study to see if it is necessary to divide the operating range into homogeneous categories. That is, you wish to assure that the average range for each test block is not significantly different from the others. Note 3: Do not zero the system between test blocks unless that is also the procedure used in running production material. Note 4: If nearly all the resulting ranges are zeroes, there is inadequate measurement discrimination.

ELECTRONIC WIDTH GAGE Measurement: Width Procedure There is no operator effect on this measurement, nor can measurements be repeated on production material. Only an estimate of repeatability is appropriate in this case. The test pieces used to calibrate the device must be used to estimate repeatability. Measure each of the test pieces at least the number of times indicated in appendix B, in random order. That is, run the test pieces in random order and repeat the necessary number

09.H1149.ch09(037-048).qxd:09.H1149.ch09(037-048).qxd

5/31/11

10:21 AM

Page 39

Chapter Nine: Examples of Problems and Procedures

39

of times. If this is not practical, it may be necessary to run all the tests on a test piece before going to the next test piece. Report the results to as many decimal places as possible. Calculate repeatability using the standard procedure. Since there is no operator, treat this case as if there were a single operator. The number of test pieces will determine the minimum number of trials. Complete the analysis, using the appropriate factors from appendix B. Only the repeatability can be estimated. Note 1: Determine the procedures used or recommended by the gage manufacturer. If they are compatible or superior to the procedures proposed here, apply them. Note 2: Examine manually generated width measurements with a standard R&R study of the device used. Note 3: Do not zero the system between test blocks unless that is also the procedure used in running production material.

ELECTRONIC TEMPERATURE EQUIPMENT Measurement: Temperature Procedure Temperatures are measured by electronic equipment that provides results on material in high-speed operations. There is no opportunity for an operator to affect the measurements. It is also not possible to run meaningful repetitions of measurements with the equipment during actual operation. Since this equipment combines optical and electrical systems, use the calibration system to establish the repeatability of the equipment. There is no reproducibility in this case. As part of the routine calibration of the system, repeatedly measure the test box (a calibrated source of electrical energy used to simulate the emissions of a hot product). This test unit should be measured at least 20 successive times. Then calculate the standard deviation of these 20 measurements. Plot the data as a moving range control chart. This permits a check on statistical control and allows estimation of the standard deviation by:

σ= Then, % Repeatability =

R 1.13

5.15 × σ × 100 and % R & R = % Repeatability. tolerance

Note 1: If there is more than one test box or test setting, you must repeat the procedure. Note 2: Solicit the manufacturer’s advice to assure that the best and most reasonable procedure is used.

09.H1149.ch09(037-048).qxd:09.H1149.ch09(037-048).qxd

40

5/31/11

10:21 AM

Page 40

Concepts for R&R Studies

Note 3: The above calculation of σ can also be done directly on most pocket calculators. However, the use of this approach does not determine whether the process is in statistical control.

CHEMICALS Measurement: Chemical Analyses Procedure Chemical analyses will fit the standard procedure for an R&R study. Take care to report results to as many decimal places as is possible, even if that is not the procedure of routine analyses. Use the standard procedure, with samples from a homogeneous range. Select the number of samples and operators according to the rules of appendix B. Use routine samples unless there is evidence that an analysis is affected by variability within a sample. In that case, it will be necessary to use National Institute of Standards and Technology standards or any other source of extremely consistent material to permit a true R&R measurement. Note 1: If you must carry out special studies or use special samples to avoid variability of material within the samples, this indicates the need for a separate study of this material variability. Such variation, a part of process capability, may be more important than the R&R study. Refer to appendix E. Note 2: Pay particular attention to the previous discussion of calibration procedures. It is essential that the calibration procedure of a chemical R&R be carefully defined and consistent for all operators. Similarly, the results should be consistently reported in the same fashion that production information is produced. In particular, if routine results provide the average of multiple analyses, then use that procedure here. Note 3: If possible, perform the R&R on the same turn and not over a longer period.

SHEET FLATNESS Measurement: Flatness Procedure Flatness of steel blanks at an outside processor may be a key variable. Performance of an R&R study first requires that you adequately define the procedure for measuring flatness. Current procedures require placing a blank on a “flat” surface and measuring the distance from this surface to the bottom edge of the blank with a ruler. Such a measurement is probably made at the point of maximum apparent deviation; that is, an effort is

09.H1149.ch09(037-048).qxd:09.H1149.ch09(037-048).qxd

5/31/11

10:21 AM

Page 41

Chapter Nine: Examples of Problems and Procedures

41

made to estimate by eye the point most likely to exceed a specification. This procedure in an R&R would require applying the procedures of “attributes” (go/no-go) testing, requiring voluminous repetitions of measurements over a range of deviations from flatness that include the specification. While this can be done, it is much easier to make a quantitative measurement. To approach this parameter as a “variables” measurement requires using an appropriate micrometer to measure the gap between the reference surface and the bottom of the blank. This micrometer should be capable of reading to at least three decimal places. The requirements then are: 1. Use a gage capable of measuring to at least three decimal places. 2. Assure that the reference surface is “flat.” 3. Define where and how to make the measurement on a blank. That is, is the reported value to be the maximum of n tests on a blank? The average of n tests? The measurement of a single randomly selected position? The eyeball estimate of the point of maximum deviation from planarity? How this measurement is defined is less important than the fact that it is indeed defined precisely and applied consistently. 4. Apply the standard procedure, with the following exceptions: a. Have each operator measure the same blank once while it is on the reference surface. Then have operators repeat the measurement without repositioning the blank. (The variability possibly induced by repositioning is equivalent to that of variability within a sample.) b. Repeat this procedure for each of the randomly selected blanks. c. Select the number of blanks sufficient for (# of blanks) × (# of operators) to be 16 or more. Use the standard procedure for analysis.

PHYSICAL TESTS Measurement: Physical Properties Procedure Physical properties (yield strength, tensile, modular R, elongation, hardness, LDH, and so on) are unique only in that most of these are destructive tests or cannot be repeated in exactly the same position. In order to perform an R&R evaluation, you must obtain test pieces from adjacent areas of the sample to ensure homogeneity. Otherwise, the analysis is routine. Obtain samples that cover the typical range of measured properties for a given category of product, that is, an appropriate grouping of grades or properties. From each sample, at least two test pieces per operator must be cut as close together as possible. In the

09.H1149.ch09(037-048).qxd:09.H1149.ch09(037-048).qxd

42

5/31/11

10:21 AM

Page 42

Concepts for R&R Studies

case of tensile tests, for example, the test pieces should be adjacent pieces from the center of the coil sample. In the case of LDH tests, the test pieces should be adjacent pieces long enough to cut the five subpieces from adjacent areas. Once the sample material has been selected and the test pieces prepared per the usual sample preparation procedures, randomly assign the test pieces to an operator. From this point, the analyses can utilize the standard form. Note 1: The intent in preparing test pieces is to have the test pieces from a sample as identical as possible with respect to the properties being measured. This will require selecting the test pieces in a different manner from normal sample generation, where the intent is to represent the coil. Note 2: Production samples should be used initially in these studies. If prior knowledge or an unsatisfactory repeatability estimate indicates the possibility of significant variability within the sample, standards must be used to separate the R&R variation from the material variation. Refer to appendix E. Note 3: Any dimensional measurements must be made to four decimal places. Note 4: If all test pieces are not prepared by one operator at one time, be ready to explore sample preparation as a major source of variation. If sample prep is not directly involved in an R&R study, direct a separate study at that potential source of variation.

PROFILOMETERS Measurement: Surface Texture Procedure Surface texture is measured with a profilometer. These measurements are unique in that they are very susceptible to the variability within the test piece. Selection of samples, number of operators, and number of trials is subject to the usual logic for a standard study. However, you should apply the following additional procedures: 1. To minimize the effect of variability within the sample, mark the point of test. Each operator should run his tests at that same place. Refer to appendix E. 2. A reported result should be the average of more than one test. Once the data are available, follow the standard analysis. Note 1: Define the averaging technique by the proper standard. Use the current standard until data are available to determine the number of tests to average. Note 2: It may be necessary to move the location of test enough after each test to avoid a “trench” effect. If so, material variability may become a factor.

09.H1149.ch09(037-048).qxd:09.H1149.ch09(037-048).qxd

5/31/11

10:21 AM

Page 43

Chapter Nine: Examples of Problems and Procedures

43

MICROMETERS Measurement: Dimensions Procedure Many measurements are made with various types of micrometers. This is the classic application of the standard procedure. A random sample of operators and test material is used to carry out a typical R&R study on each device. Consistently report results to as many decimal places as possible. It is also important that the operators measure at the same place on each sample. Otherwise, the variability within the sample will intrude on the R&R study. Analysis uses the standard procedures. Shortcut Procedure Due to the large population of micrometers to be analyzed, a shortcut procedure was developed that may reduce the amount of testing. The key change in approach is to have each operator use a different micrometer for the R&R study. By doing so, the effect of reproducibility (the operator effect) is mixed (the statistical term is “confounded”) with that of the various gages (calibration effects). If the normal R&R estimates are acceptable, then several micrometers have been approved in one study; if the results are not acceptable, then an R&R study must be performed on each micrometer in order to isolate the R&R effects. The procedure is: 1. Select the number of operators and samples to meet the requirement of a minimum of 16 operator–sample combinations. Maximize the number of operators in the study. It is recommended that you use four operators and 10 samples. This will allow four micrometers to be evaluated in one study. 2. Assign a different micrometer to each operator; each operator is to use this same micrometer for the entire study. Note that you will not be able to tell the difference between the effects of operator and micrometer. 3. Have each operator carry out at least two trials. 4. Analyze using the standard procedure. The only unique feature in the analysis is that reproducibility is actually a combination of the differences in operators, micrometers, and calibration. If the results are acceptable, there is no need to separate the mixed components in the reproducibility estimate; all the micrometers involved are acceptable. If the results are not acceptable, perform a separate R&R on each micrometer. Note: This procedure is applicable to other device types where the number of devices is large.

09.H1149.ch09(037-048).qxd:09.H1149.ch09(037-048).qxd

44

5/31/11

10:21 AM

Page 44

Concepts for R&R Studies

SCALES Measurement: Weight Procedure A scale R&R is performed by making repeated measurements of the same item, whether that item is a test weight or a product. The R&R may be run over a short or long period of time, depending on the time available at the scale for making the measurements. There may not be an operator effect in most cases, although this should be considered carefully. (For instance, differences between operators in placement of the product on the scale may create a difference in measurements. If this is possible, consider an operator effect.)

Case A: Short-Term Studies 1. Calibrate the scale, or assure that it is within the required calibration period. 2. Weigh the test weight or the product. If the scale is routinely zeroed before each measurement, do this. If not, don’t. 3. Remove the weight. 4. Repeat steps 2 and 3 until you have made 20 measurements. 5. Calculate the standard deviation of these measurements as the σ of repeatability. Method 1 Plot a moving range control chart from the time-ordered data. statistical control and calculate the standard deviation as

σ=

R d2

Method 2 Calculate the standard deviation (σ ) using the formula on your calculator (make sure it is the formula that divides by n – 1), recognizing that you need to assess statistical control.

Case B: Long-Term Studies 1. Calibrate the scale. 2. Weigh the test weight or the product. If the scale is routinely zeroed before each measurement, do so here. If not, don’t. 3. Remove the weight. 4. Repeat steps 2 and 3 twice more.

09.H1149.ch09(037-048).qxd:09.H1149.ch09(037-048).qxd

5/31/11

10:21 AM

Page 45

Chapter Nine: Examples of Problems and Procedures

45

5. At the next regular scale calibration, repeat steps 1 through 4. 6. Repeat this procedure (steps 1 through 5) until at least eight sets of three measurements are accumulated. (This is based on appendix B.) 7. Analyze, except treat each set of three measurements as a sample. That is, the eight time periods act as samples, while the three measurements at each time period are the trials. Note 1: The procedures described above assume that there is no operator effect. If there is concern or question about this point, then repeat each of the above procedures for each operator used in the study. The standard procedure is directly applicable. Note 2: If, as part of calibration control, you maintain a control chart for ranges on repeated measurements of a test weight, you can use that control chart in place of these procedures. Note that such a control chart is nothing more than a range chart – of the data in procedure 2. The R can be used to estimate repeatability by using: σ EV =

R and EV = 5.15 × σ EV d2

Note 3: If the calibration procedure is not currently control-charted, it would be very use– ful to establish an X and R-chart on the data generated in Case B. The R-chart, as – note 2 indicated, covers the R&R. The X-chart would be useful in assessing the effectiveness of the calibration procedure.

BORE GAGE Measurement: Tubing Inside Diameter Procedure A bore gage measures the inside diameter of steel tubing. Insert the gage in the tube, resting on the three positioning prongs. The gage must be held parallel to the tube and held steady. As you rotate the gage in the tube, the “measurement” prong follows the contour of the inside diameter of the tube, with the dial indicating the diameter. Studies may analyze the diameter, the ovality (maximum diameter minus minimum diameter), or the maximum and minimum diameters separately. To perform an R&R study: 1. Calibrate or the bore gage 2. Select operators and samples for the study to meet the criteria of appendix B 3. Instruct operators to measure at the same distance from the end of the tube 4. Each operator measures the samples (trial 1)

09.H1149.ch09(037-048).qxd:09.H1149.ch09(037-048).qxd

46

5/31/11

10:21 AM

Page 46

Concepts for R&R Studies

5. Repeat the trials the necessary number of times 6. Use the standard procedure Note 1: Consistent lack of planarity by an operator in holding the gage will enlarge the reported reproducibility; inconsistency in planarity will enlarge the reported repeatability. Note 2: Variation in the depth within the tube at which the measurements are made may introduce variation from within the sample. Consistent differences between operators in location may enlarge the reported reproducibility; inconsistent differences may enlarge reported repeatability.

NUCLEAR MOISTURE GAGE Measurement: Moisture Procedure A nuclear moisture gage is an electronic device mounted at a weigh hopper or material transfer point that measures the hydrogen atoms in the material in the immediate vicinity of the device head. This reading is translated to an estimate of moisture via a calibration curve that is validated periodically. As soon as the gage has completed the measurement, the estimated moisture is printed and the material is transferred. Due to the brief time available and the necessity to avoid delaying a time-sensitive operation, the problem is how to carry out a meaningful R&R study. In this situation, there is no operator effect. Since we are dealing only with repeatability, recall that repeatability, as the name implies, stems from repeated measurements of the same material by a measurement device. In this situation, the device cannot return at a later time to repeat this measurement as would be the procedure in a standard R&R study; the only way to repeat the measurement is to do it immediately. Since such operations are time-sensitive already, it would be difficult for operations to wait for more than two successive readings; in fact, such repeated measurements may need to be spread over several days to avoid affecting the operating rate. This may be desirable anyway to carry out the study over a range of moisture conditions. Therefore, use the following procedure: 1. Since there is no operator effect, treat this case as if there were a single operator. To satisfy appendix B, there must be at least 16 pairs of measurements. 2. Make the usual measurement on a weigh hopper, then immediately repeat the measurement. You must do this when the material is not moving. This allows the device to sense the moisture in exactly the same material as the first reading.

09.H1149.ch09(037-048).qxd:09.H1149.ch09(037-048).qxd

5/31/11

10:21 AM

Page 47

Chapter Nine: Examples of Problems and Procedures

47

3. Repeat this procedure over several days to obtain 16 pairs. These could be done, for example, at the rate of two pairs per day for eight days or one pair per day for 16 days. 4. Analyze as usual, using the standard procedure. Note: If the moisture gage requires recalibration during this program, you must repeat the study after the recalibration.

10.H1149.ch10(049-050).qxd:10.H1149.ch10(049-050).qxd

5/31/11

10:22 AM

Page 49

10 Conclusion

T

he continuous improvement philosophy is essential in the realm of measurement. It is a realm where no procedure holds universally. Each device or situation must be thoughtfully considered and appropriate procedures tailored to fit the conditions. R&R studies provide an opportunity to improve your process and reduce variation. Establishment of an index or criterion does not mean that any correctable measurement problem should be tolerated. Even if not a priority, the continuous improvement philosophy requires that corrective action be defined and a timetable assigned to accomplish it. The intent with R&R studies is that they are to be ongoing, not just performed once and forgotten. This manual has covered the basic concepts for classic R&R—repeatability and reproducibility. In conclusion, refer again to Figure 1.1 and that other components of measurement variation were assumed to be negligible. By ignoring any significant contributions from other components of measurement, we have included them with the estimate of process variation. Once an R&R program is under way, it is important to evaluate calibration, stability, and linearity. A comprehensive R&R program should tie into a comprehensive calibration program. Finally, go do some R&R studies. Find some software or use spreadsheets for the number crunching. After trying a couple of studies, it will be obvious that your program is neither as difficult nor as massive as most people assume. Good luck!

49

11.H1149.Appen(051-072).qxd:11.H1149.Appen(051-072).qxd

6/1/11

8:35 AM

Page 51

Index to Appendixes

I

t is not essential to understand the origin of the statistical methodology used for R&R calculations, but it helps. The following appendices are provided to explain the origin of the standard procedure, the tables, and the various procedures developed in this book. To help anyone who wishes to dig deeper, a set of recommended references is provided. Appendix A—The Standard Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

52

Appendix B—Factors for Varying Sample Sizes . . . . . . . . . . . . . . . . . . . . . . . . . .

53

Appendix C—Derivation of Repeatability and K1 and K2 Factors . . . . . . . . . . . . .

54

Appendix D—Derivation of the Calculation of Reproducibility . . . . . . . . . . . . . .

57

Appendix E—How to Address Variation within a Sample . . . . . . . . . . . . . . . . . .

60

Appendix F—Confidence Levels in Estimating Standard Deviations . . . . . . . . . .

64

Appendix G—Use of Control Charts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

66

Appendix H—Relationship of Process Capability and Measurement Capability Indices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

68

Appendix I—The Effect of R&R on Process Capability . . . . . . . . . . . . . . . . . . . .

70

51

11.H1149.Appen(051-072).qxd:11.H1149.Appen(051-072).qxd

52

6/1/11

8:35 AM

Page 52

Concepts for R&R Studies

APPENDIX A—THE STANDARD PROCEDURE Repeatability and Reproducibility Report Product Characteristic

Date

Upper Specification Limit

Performed By

Lower Specification Limit

Gage

Process Sigma

Gage Number

Number of Operators Oper.

A

B

Sample # 1st Trial 2nd Trial 3rd Trial

Range

C

1st Trial 2nd Trial 3rd Trial

Range

D

1st Trial 2nd Trial 3rd Trial

Range

1st Trial 2nd Trial 3rd Trial

Range

1 2 3 4 5 6 7 8 9 10 Totals

– RA

SumA – XA – RA – RB – RC – RD

# Trials

Sum – R

– RB

SumB – XB

– RC

SumC – XC d2

2

3.27

1.13

D4 =

3

2.58

1.69

*UCLR

4

2.28

2.06

– – *UCLR = D4 × R

Repeatability

– Maximum X = –= Minimum X – X DIFF =

– R=

D4

– RD

SumD – XD

*Limit of individual Rs. Identify the cause and correct for those beyond limit. Either a) repeat those readings using the same operator and gage, or b) discard those values and recompute – R and UCL from the remaining observations. Reproducibility Appraiser variation (AV) = 99% range

Equipment variation (EV) = 99% range

AV =

– EV = R × K1 =

(X

diff

×K2

) − ⎡⎢⎣(EV ) ÷ (n × r )⎤⎥⎦ = 2

– σEV = R /d2 = EV/5.15 = A.V. =

**

n = # of samples r = # of trials

2

σ AV =

AV = 5.15

# Trials

2

3

# of operators

2

3

4

K1

4.56

3.05

K2

3.65

2.7

2.3

**If a negative value is calculated under the square root or if there is only one operator, AV = 0.

Repeatability and Reproducibility

%Repeatability = 100 ×

R&R = 99% range R &R =

(EV ) + (AV )

σ R &R

2

R &R = = 5.15

Measurement Capability Index 1 As a Percentage of Process Variation

2

=

%Reproducibility = 100 ×

%R & R = 100 ×

σ EV σt σ AV σt σ R &R σt

% R&R Acceptance Criteria MCI1 Good Marginal Unacceptable

>0 – <20% >20%; <30% >30%

11.H1149.Appen(051-072).qxd:11.H1149.Appen(051-072).qxd

6/1/11

8:35 AM

Page 53

Appendixes

53

APPENDIX B—FACTORS FOR VARYING SAMPLE SIZES The purpose of this table is to permit selection of varying numbers of samples and yet maintain roughly equivalent confidence in the results. The derivation of the modifications to the procedural parameter K1 is shown in appendix C.

K1 Factors When (# operators) × (# samples) ≤ 15 (# operators) × (# samples) # of trials↓

4

5

6

7

8

9

10

11

12

13

14

15

≥ 16

2 3 4 5 6

4.26 2.94 2.44 2.17 2.00

4.33 2.96 2.45 2.18 2.01

4.36 2.98 2.46 2.19 2.02

4.40 2.98 2.46 2.19 2.02

4.40 2.99 2.48 2.20 2.02

4.44 2.99 2.48 2.20 2.02

4.44 2.99 2.48 2.20 2.02

4.44 3.01 2.48 2.20 2.02

4.48 3.01 2.49 2.20 2.02

4.48 3.01 2.49 2.20 2.02

4.48 3.01 2.49 2.20 2.03

4.48 3.01 2.49 2.20 2.03

4.56 3.05 2.50 2.21 2.04

Notes: 1. If possible, select the number of operators and samples such that (# operators) × (# samples) exceeds 15. 2. If (# operators) × (# samples) < 16, enough trials should be run to avoid the shaded area of the table. (Otherwise, be aware that the estimates will be less precise.)

11.H1149.Appen(051-072).qxd:11.H1149.Appen(051-072).qxd

54

6/1/11

8:35 AM

Page 54

Concepts for R&R Studies

APPENDIX C—DERIVATION OF REPEATABILITY AND K1 AND K2 FACTORS K1 Factors To understand the tables for determining the proper K1 factors, we first need to understand what is going on. In the standard procedure, the range of the trials for each operator–sample – combination is calculated. These are– averaged to get the average range for each operator (Ra, – Rb, and so on). The grand average, R, is the average of all operators. These ranges of measurements on the same sample by the same operator are used to ascertain the standard deviation for repeatability (σEV ). (Since repeatability is also referred to as the equipment variation, the notation EV is used.) From quality control statistics, recall that ranges and standard deviations are related by the formula:

σ EV =

R d2

where d2 depends on the number of trials used to calculate a single range. Repeatability (EV) is defined here as the 99 percent spread or 5. 15 σEV. ⎛R⎞ 5.15 Repeatability = 5.15 σ EV = 5.15⎜ ⎟ = K1 R where K1 = d2 ⎝ d2 ⎠ For example, for a study with three trials, the sample size for each range is 3. The tabular value for d2 for a sample size of 3 is d2 = 1.69. ⎛R⎞ R Repeatability = 5.15 σ EV = 5.15⎜ ⎟ = 5.15 = 3.05 R 1.69 ⎝ d2 ⎠ This is the source of the K1 factors for a given number of trials. However, Duncan (1974) has shown that this estimation procedure should be corrected when there are a small number of ranges. Since there is a range calculated for each operator–sample combination, the product of (# operators) × (# samples) is the number of ranges. If there are at least 16 such ranges, the usual estimation procedure using d2 is appropriate. In such a case, the values of K1 from the standard procedure are correct. If (# operators) × (# samples) is less than 16, then the estimation procedure should be adjusted. This is done by using a parameter, d2*, instead of d2 and K1 =

5.15 d2*

The values of d2* appear in Duncan’s Table D3. The particular value to use depends on the number of ranges calculated from (# operators) × (# samples) and the number of trials.

11.H1149.Appen(051-072).qxd:11.H1149.Appen(051-072).qxd

6/1/11

8:35 AM

Page 55

Appendixes

55

(Duncan refers to them as g and m respectively.) For example, if three operators are used with four samples for three trials, then (# operators) × (# samples) = 12. Since each trial provides a subgroup size of three, the referenced table for g = 12 and m = 3 provides d2* = 1.71. Then ⎛R⎞ 5.15 5.15 EV = 5.15 σ EV = 5.15⎜ * ⎟ and K1 = * = = 3.01 d2 1.71 ⎝ d2 ⎠ This is the value shown in appendix B and is the source of the adjusted K1 factors when (# operators) × (# samples) ≤15. The table of adjusted K1 factors recommends a minimum number of trials in an R&R study. However, the recommended number of trials for a given number of ranges ([# operators] × [# samples]) assures that the estimates of σEV are based on at least 14 degrees of freedom, the minimum for the case where (# operators) × (# samples) is > 15 and there are two trials. If one does the standard study with 10 samples, two trials are sufficient if at least two operators are used. When the combinations of numbers of operators, samples, and trials fall into the “forbidden” area of the table, the level of confidence in the resulting estimates is reduced. Avoid these conditions where possible. This procedure also is valid when there is a single operator (if there is no operator, assume a single operator). In that case, the rule concerning (# operators) × (# samples) > 15 becomes simply (# samples) > 15. If the number of samples is 15 or less, then the number of trials should be increased accordingly. Previous discussions indicated that the parts used in the study should cover the range of measurements expected. However, there is also a requirement that the range of measurements used in the study be defined such that the average range on each part is expected to estimate a common σEV. If this homogeneity is not met, the results become erratic.

K2 Factors The same rationale applied to obtain the K1 factors is also applied to obtain the K2 factors. The K2 factor is used to ascertain the standard deviation of the operators’ averages, leading to the estimation of reproducibility. The range of the operators’ averages is labeled – “ Xdiff ”. Again,

σ operators averages =

R Xdiff = d2 d2

Such a small number of ranges (one!) forces us to use d2* instead of d2. Then,

σ operators averages =

R Xdiff = * d2 d2

11.H1149.Appen(051-072).qxd:11.H1149.Appen(051-072).qxd

56

6/1/11

8:35 AM

Page 56

Concepts for R&R Studies

Use Table D3 in Duncan’s text again to find the proper value of d2* for g = 1, m = # of operators. Reproducibility is defined as a 5.15 range. Using the same basis for the σ of operators’ averages, 5.15 σ operators averages =

5.15 Xdiff d2*

= K2 Xdiff where K2 =

5.15 d2*

For example, for four operators, Duncan’s table provides (g = 1, m = 4) a value of d2* = 2.24. Then K2 =

5.15 5.15 = = 2.30 d2* 2.24

The other values for K2 are estimated similarly. The estimate of K2 provides degrees of freedom = (# operators – 1). Thus, if we use two operators, there is only one degree of freedom for the estimate. This is why it is recommended to use three or four operators if practical. Note that if there is a single operator, it is not possible to calculate the K2 factor, and thus there is no AV value.

11.H1149.Appen(051-072).qxd:11.H1149.Appen(051-072).qxd

6/1/11

8:35 AM

Page 57

Appendixes

57

APPENDIX D—DERIVATION OF THE CALCULATION OF REPRODUCIBILITY The preceding section covered the derivation of the calculation of repeatability, as well as the origins of the K2 factors used in calculating reproducibility. In this section, we examine the origin of the formula for reproducibility (AV). (Since reproducibility is also referred to as “appraiser variation,” the notation AV is used.) The procedure of an R&R study is based on an analysis of variance. If you know what an analysis of variance is, continue this appendix. If not, quit reading this appendix! Go find a basic statistics text (such as Hicks [1982] in the reference list) and study it first. An R&R study is a designed experiment in which the factors are 1) samples, and 2) operators. The trials are nested within operators and samples. The standard procedure assumes that: 1. The samples are random representatives from a very large population of possible samples. 2. The operators are random samples from a very large population of possible operators. (This assumption is clearly debatable if there are only a few operators.) This assumption implies a random component of variation for operators versus a fixed effect if the operator population is limited. There may be occasions where minor differences in conclusions result from this assumption. 3. The range methods assume that the interaction of operators and samples does not exist. The calculation of the upper control limit of the ranges on the R&R form is an attempt to confirm this assumption. Use of operators who are equally skilled also tends to prevent interaction. If analysis is performed with ANOVA methods, this assumption is obviously not required and interaction components can be measured. Interaction could result from too broad a range for the sample, as well as mixing operators with different skill levels in the study. 4. The study has been performed in such a manner, using random order of tests, that the pooled variation of the trials for each operator–sample combination is an estimate of the error mean square in an analysis of variance.

11.H1149.Appen(051-072).qxd:11.H1149.Appen(051-072).qxd

58

6/1/11

8:35 AM

Page 58

Concepts for R&R Studies

Then the ANOVA table, with expected mean squares, is: ANOVA Table Source

Expected Mean Squares

Samples

2 σEV + or σS2

Operators

2 σEV + rnσ 2AV

Error

2 σEV

where n = # samples, o = # operators, and r = # trials.

We have no interest in the variability due to samples; in fact, we select the samples to cover the range of possible materials. (Some software analyses with ANOVA reference the R&R variation to the total variation in the study, using the sample variation. This assumes that the small number of samples can estimate the process variation. This part of their analysis should be ignored.) Based on appendix C, 2 σ EV

⎛R⎞ =⎜ ⎟ ⎝ d2 ⎠

2

The variance in operators’ averages is given by: ⎛ Xdiff ⎞ ⎜ d* ⎟ ⎝ 2 ⎠

2

Since each operator’s average is the average of (r × n) measurements, then using the ANOVA table’s expected mean square, 2

2 σ operators averages

2 2 ⎛X ⎞ + rnσ AV σ EV = = ⎜ diff * ⎟ rn ⎝ d2 ⎠

(Although this all looks formidable, it is a restatement of this rule: If the variance of one observation is σ2, the variance of an average of m observations is σ2 .) m 2

Thus,

2 σ operators average

2 and σ AV

2 ⎛X ⎞ ⎛X ⎞ σ EV 2 = + = ⎜ diff = ⎜ diff σ AV * ⎟ rn ⎝ d2* ⎟⎠ ⎝ d2 ⎠

⎛R⎞ 2 2 ⎜d ⎟ 2 ⎛ Xdiff ⎞ ⎛ Xdiff ⎞ ⎝ 2⎠ σ EV =⎜ * ⎟ − =⎜ * ⎟ − rn ⎝ d2 ⎠ rn ⎝ d2 ⎠

2

2

11.H1149.Appen(051-072).qxd:11.H1149.Appen(051-072).qxd

6/1/11

8:35 AM

Page 59

Appendixes

59

Look familiar? This is what is happening in the calculations of the standard procedure, using range methods. The only difference is that the standard procedure is working with 5.15 times the standard deviations, thus using K1 and K2. That results in:

( )

( EV )2 = (5.15σ EV ) = K1 R 2

( AV ) = (5.15) 2

2

(

σ 2AV

)

2

⎡⎛ X ⎞ 2 σ 2 ⎤ EV ⎥ = K2 Xdiff = (5.15) ⎢⎜ diff * ⎟ − d rn ⎥⎦ ⎢⎣⎝ 2 ⎠

(

2

Then, AV =

(

K2 Xdiff

)

2

−

)

2

−

( EV )2 rn

( EV )2 rn

How about that? This is what the range methods require. For a discussion of the respective roles and advantages of analysis of variance – versus X and R-charts, refer to Duncan (1974).

11.H1149.Appen(051-072).qxd:11.H1149.Appen(051-072).qxd

60

6/1/11

8:35 AM

Page 60

Concepts for R&R Studies

APPENDIX E—HOW TO ADDRESS VARIATION WITHIN A SAMPLE In some measurement situations, variation within the sample cannot be prevented from affecting the R&R study. A case in point is the measurement of “out of roundness” of a bar. There is variability in this measurement, depending upon where on the sample bar the measurement is made. Another example is the surface roughness of a sample, which may vary significantly across the sample. A third example is a destructive measurement (for example, tensile strength) where the same test cannot be repeated. what has been said previously: This variation within the sample is extremely important and can be a major component of process capability. We can understand neither the measurement variation nor the process variation without defining the variation within the sample. Refer to Figure 1.1, page 2. The question is one of order of attack. In order, the priorities must be: 1. There must be a calibration procedure in place 2. Define the measurement capability (the R&R), excluding variation within the sample 3. Define the variation within the sample 4. Move on to process control Two situations that must be considered are illustrated by the following cases.

Case A: Either we don’t know what to expect or it is logical to assume no significant variation within the sample. In such instances, perform an R&R in the routine fashion, making no effort to avoid the effect of variation within the sample. This would mean that you would not tell the operator to test or retest at the same location. 1. If the R&R calculations for repeatability are satisfactory, any variation within the sample is either insignificant or not a pressing concern. Move on. 2. If the result of the calculations of repeatability is poor or marginal, you must address and define the existence of significant variation within the sample. The R&R study will have provided an estimate of σEV. If there is significant variation within the sample, this estimate is actually the estimate of:

σ EV = σ e2 + σ v2 where σe estimates the actual sigma of repeatability and σv estimates the sigma of the variation within the sample. We need to separate and estimate each component. It cannot be done as the study was performed. We must perform a second study to estimate σe, allowing us to come back and obtain σv from the equation above.

11.H1149.Appen(051-072).qxd:11.H1149.Appen(051-072).qxd

6/1/11

8:35 AM

Page 61

Appendixes

61

How do we estimate σe without the effect of σv? We do this by repeating an R&R study, either with special samples that we know do not contain variation within a sample or by specifying exactly where on the production sample the repeated measurements are to be made. Examples would be using standard test blocks in testing a hardness tester or specifying the exact point on a bar to test for out-of-roundness. This time, σe = σEV of the second study. Use this value in the prior equation to solve for σv. For example, assume in an R&R study of a profilometer that σEV = 2.33, an undesirable result. The design of the study permitted the variation within the sample to be mixed with that of repeatability. Then, σ EV = 2.33 = σ e2 + σ v2 Assume the study is rerun, only this time the point of measurement is controlled. The new estimate is σe = 1.21 = σEV of the second study. Now, 2.33 = σ e2 + σ v2 = (1.21)2 + σ v2 5.43 = 1.46 + σ v2

σ v2 = 3.97 σ v = 1.99 Thus, we have found an estimate of σEV that does not include σv. We have also obtained an estimate of σv. In this example, we could continue to measure the R&R results based on the “clean” estimate of σEV. This will indicate our ability to measure. Note that the actual, routine measurements will be more variable than indicated by the R&R if σv (the variation within the sample) is a significant contributor to the variation. What is meant by these results? In this example,

σ EV = 2.33 = σ e2 + σ v2 where

the estimate of the combined effect of device repeatability and within-sample variation

σe = 1.21

the estimate of repeatibility without the effect of variation within the sample

σe = 1.99

the estimate of the variation within the sample, excluding repeatability

If measurements are made at a randomly selected location on the samples, that measurement variation contains the effect of both within-sample variation and repeatability of the measurement devices. Figure A.1 illustrates this situation (in interpreting Figure A.1, that the squared sigmas are added, not the sigmas.)

11.H1149.Appen(051-072).qxd:11.H1149.Appen(051-072).qxd

62

6/1/11

8:35 AM

Page 62

Concepts for R&R Studies

5.15σe

Actual repeatability

+

5.15σv

Within-sample variation

=

5.15σEV

Observed repeatability

Figure A.1 Repeatability and within-sample variation.

If the measurements now are made at the same location on each sample, then the measurement variation does not contain the effect of variation within the sample. In that case σEV = σe and σv is not measured at all. Note that in this example, the estimation of repeatability is seriously influenced by whether variation within the sample is present. The results of the variation within the sample are extremely important, providing vital information on the process control characteristics to be addressed. What do we do about these results? The fact that a “clean” estimate of R&R is now available does not mean that the variation within the sample can be ignored. That information must be brought forward to stimulate process improvement. There are two ways to indicate the significance of the variation within the sample (either of the two measurement capability indices can be used): 1. Calculate percent of variation within sample =

σv × 100 σt

2. Calculate % R&R plus variation within sample =

2 σ e2 + σ v2 + σ AV × 100 σt

This allows the relative importance of the variations within the sample to be described and compared with % EV, % AV, and % R&R.

11.H1149.Appen(051-072).qxd:11.H1149.Appen(051-072).qxd

6/1/11

8:35 AM

Page 63

Appendixes

63

Case B: We know there is significant variation within the sample. The only difference between this situation and the previous case is the order of the successive R&R studies. 1. If you already know there is significant variation within the sample, then first perform the R&R study using whatever procedure will eliminate or avoid the variation within the sample. This may mean using standards or special samples, and/or carefully identifying and controlling the point of measurement. The resulting R&R study will estimate σEV, an estimate that is free of the within-sample variation; that is, σEV = σe. This study will provide the official R&R results. 2. Next, rerun an R&R on production samples or in a fashion that will produce the combined estimate of:

σ EV = σ e2 + σ v2 Use the two sets of results to solve as before for σv. This second study will provide the estimate of variation within the sample. Three final comments: 1. R&R studies are limited, by definition, to the effects of operators and the measurement device. 2. Any indication or expectation of variation within the sample must be followed up with an appropriate study. Otherwise, our knowledge of the process is incomplete and may mislead our efforts to achieve improvement. To understand the measurement system, we must measure and understand this component of variation. 3. This appendix has touched briefly on the mechanics, striving to convey the logic and philosophy necessary for dealing with variation within a sample. In confusing or difficult cases, don’t hesitate to seek help from those more knowledgeable in statistics.

11.H1149.Appen(051-072).qxd:11.H1149.Appen(051-072).qxd

64

6/1/11

8:35 AM

Page 64

Concepts for R&R Studies

APPENDIX F—CONFIDENCE LEVELS IN ESTIMATING STANDARD DEVIATIONS The R&R process is one of estimating standard deviations. Appendix B allows assurance that there will be at least 14 degrees of freedom in the estimation of repeatability. Typical statistical analysis defines degrees of freedom as d.f = k(n – 1), where n is the number of observations of each of k samples used to make the estimate. In R&R studies using the range estimates, the degrees of freedom will be slightly less than this, because of the small number of samples and/or the small sample sizes. D. C. McCune prepared a chart that related sample size to precision of estimates of standard deviations. The chart is not exact for R&R estimates, but it is close enough to indicate the relative precision of our results if we approximate degrees of freedom by d.f = .9k(n – 1), where k is the number of ranges and n is the number of trials. Figure A.2 uses the chi-square relationship:

χ a2,df =

(d. f .)s 2 σ2

to estimate the curves. It is important to note that: 1. Reproducibility typically has 1, 2, or 3 degrees of freedom (one less than the number of operators). Such small degrees of freedom imply that extreme variability is possible in that estimate. 2. Repeatability should have a minimum of 14 degrees of freedom. Note that there is still a potential error of as much as 30 percent at the 90 percent confidence level and 37 percent at the 95 percent confidence level. 3. Should the precision of estimate be important, it is necessary to use a software package that will not only compute the ANOVA procedure but also will compute the confidence intervals for the estimated components. NCSS is excellent in this regard. An R&R study should not be run once and forgotten. Such studies must be performed regularly and records kept on corrective actions to monitor the performance over time.

11.H1149.Appen(051-072).qxd:11.H1149.Appen(051-072).qxd

6/1/11

8:35 AM

Page 65

Appendixes

65

p = confidence level

Maximum Percent Error (±)

75 65 55 p = 95%

45 35 p = 99%

25 p = 90%

15 10 5

10

15

20

25

30

35

40

45

50

55

60

65

70

75

80

Degrees of Freedom

Figure A.2 Precision in estimating the standard deviation as a function of degrees of freedom.

11.H1149.Appen(051-072).qxd:11.H1149.Appen(051-072).qxd

66

6/1/11

8:35 AM

Page 66

Concepts for R&R Studies

APPENDIX G—USE OF CONTROL CHARTS Sometimes a graphical approach is best for shedding light on a problem. Figure A.3 is an R&R on chemical analysis. The results are poor! Before we start replacing chemists with random number tables, review the range chart of the ranges calculated to estimate repeatability. The control chart is in Figure A.4. Does it appear that there is a significant difference between the operators? A careful – – observer would have noted that the two average ranges Ra = 8.1 and Rb = 30.2) appear to differ seriously. A quick test (either using an F-test or calculating the control limits for the R’s) would confirm that the operators differ significantly in repeatability. Operator A is definitely more repeatable than operator B. Look at the detailed data for the “poor” operator. See anything nonrandom? Compare his first trial with his second trial. Is it likely that his second measurement of a sample would be higher than the first in 10 of 10 cases? The probability of such a random outcome is slight, about one chance in a thousand. What happened between operator B’s first trial and his second trial? Did he recalibrate between trials? At this point, there is evidence that the R&R has a serious flaw (just what the flaw is has yet to be determined) that has drastically increased the size of the R&R estimates. This problem must be resolved and the study repeated. This is actually an opportunity for immediate improvement! You may be interested in knowing that Figure 2.1, page 7, provides the corrected data for this study. Note that the % EV or repeatability changed from 34 percent to 17 percent! When all else fails, look at the data. In addition to facilitating such insights, the control charts are a necessary validation of the statistical control of the measurement process. The above problem would not have appeared as an interaction in an ANOVA. Regardless of the preferred method of analysis, simple graphs are a powerful supplement!

80 70 UCL = 62.8

Ranges

60 50 40 30

_ R = 19.2

20 10 2

Figure A.4 Range chart.

4 6 8 Operator A

10

2

4 6 8 Operator B

10

11.H1149.Appen(051-072).qxd:11.H1149.Appen(051-072).qxd

6/1/11

8:35 AM

Page 67

Appendixes

Repeatability and Reproducibility Report Aluminum analysis

Product Characteristic

Date

Upper Specification Limit

650

Performed By

Lower Specification Limit

350

Gage

50

Process Sigma

Gage Number

2

Number of Operators Oper.

A

B

Sample # 1st Trial 2nd Trial 3rd Trial 1

Range

471 484 742 765 328 326 446 433 456 454 443 450 557 552 477 479 509 508 371 384 4831 4804

2 3 4 5 6 7 8 9 10 Totals

SumA – XA – RA – RB – RC – RD

13 485 778 23 2 328 13 455 2 470 7 460 551 5 2 499 513 1 13 390 81 4929 8.1

510 837 344 480 491 485 577 522 570 415 5231

SumB – XB

10160 508.0

– RA

9635 481.8 8.1 30.2

38.3 19.2

Sum – R

C

1st Trial 2nd Trial 3rd Trial

Range

Range

1st Trial 2nd Trial 3rd Trial

Range

25 59 16 25 21 25 26 23 57 25 302 30.2 – RB

– RC

SumC – XC

# Trials

D4

d2

– R=

2

3.27

1.13

D4 =

3

2.58

1.69

*UCLR

4

2.28

2.06

–– *UCLR = D4 × R

Repeatability

D

1st Trial 2nd Trial 3rd Trial

– RD

SumD – XD

– Maximum X = 508.0 – Minimum X = 481.8 – X DIFF = 26.2

19.2 3.27 62.8

*Limit of individual Rs. Identify the cause and correct for those beyond limit. Either a) repeat those readings using the same operator and gage, or b) discard those values and recompute – R and UCL from the remaining observations. Reproducibility Appraiser variation (AV) = 99% range

Equipment variation (EV) = 99% range – EV = R × K1 = 19.2

AV =

× 4.56 = 87.55

(X

diff

×K2

) − ⎡⎣⎢(EV ) / (n × r )⎤⎦⎥ = 2

n = # of samples r = # of trials

2

(26.2 × 3.65)2 − ⎡⎣⎢(87.55 )2 / (10 × 2)⎤⎦⎥

– σEV = R /d2 = EV/5.15 = 17.0

A.V. = 93.6 * *

σ AV =

AV = 18.2 5.15

# Trials

2

3

# of operators

2

3

4

K1

4.56

3.05

K2

3.65

2.7

2.3

**If a negative value is calculated under the square root or if there is only one operator, AV = 0.

Repeatability and Reproducibility

R &R =

(EV ) + (AV )

(87.55 )2 + (96.3)2 σ R &R

σ EV σt = 100 × 17.0/50 = 34.0% σ % Reproducibility = 100 × AV σt = 100 × 18.2/50 = 36.4% σ % R & R = 100 × R &R σt = 100 × 24.9/50 = 49.8% % Repeatability = 100 ×

R&R = 99% range 2

Measurement Capability Index 1 As a Percentage of Process Variation ( σ t )

2

=

= 128.2

R &R = = 24.9 5.15

Figure A.3 Repeatability and reproducibility report.

% R&R Acceptance Criteria MCI-1 Good Marginal Unacceptable

>0 – <20% >20%; <30% >30%

67

11.H1149.Appen(051-072).qxd:11.H1149.Appen(051-072).qxd

68

6/1/11

8:35 AM

Page 68

Concepts for R&R Studies

APPENDIX H—RELATIONSHIP OF PROCESS CAPABILITY AND MEASUREMENT CAPABILITY INDICES The two measurement capability indices and process capability (as indicated by ) are related such that any two of these indices will specify the third. ( is used since it specifies the process spread.) MCI1 =

σ R& R 5.15σ R& R USL − LSL ; MCI 2 = ; = σt 6σ t USL − LSL

From the first two equations,

σt =

σ R& R 5.15σ R& R and USL − LSL = MCI1 MCI2

Then substituting these last two expressions into the formula: 5.15σ R& R USL − LSL 5.15 MCI1 MCI2 = = = 6σ t 6 MCI2 ⎛σ ⎞ 6⎜ R& R ⎟ ⎝ MCI1 ⎠ = 0.858

MCI1 MCI2

Figure A.5 plots s for various values of the two measurement capability indices. Review of these plots show that a good is possible with a poor MCI only if there is a generous specification and/or the R&R variation (σR&R) is a dominant part of the observed variation (σt). The primary value of these relationships is that it validates the advantage of using with a measurement index to better understand the total process and measurement situation.

11.H1149.Appen(051-072).qxd:11.H1149.Appen(051-072).qxd

6/1/11

8:35 AM

Page 69

Appendixes

0.70 = 0.5

= 0.75 = 1.0

MCI2 = 5.15 Sigma R&R/Tolerance

0.60 0.50

= 1.25

0.40 = 1.5

0.30 0.20 0.10 0.00 0.1

0.2

0.3

0.4

0.5

0.6

MCI1 = Sigma of R&R/Sigma Process

Figure A.5 contours for MCI1 and MCI2.

0.7

69

11.H1149.Appen(051-072).qxd:11.H1149.Appen(051-072).qxd

70

6/1/11

8:35 AM

Page 70

Concepts for R&R Studies

APPENDIX I—THE EFFECT OF R&R ON PROCESS CAPABILITY The improvement possible in process capability resulting from improvement in R&R can be calculated using the formula

σ t2 = σ 2p + σ R2 & R where σt represents the observed process variation as determined from a control chart; σp represents the actual process variation; σR&R represents the standard deviation of R&R from a study. Given the values of σt and σR&R, σp can be calculated. As reduced values of σR&R are estimated, the reduced value of σt can be estimated. Then the improved measurement capability index can be calculated by either of the two indices. This can be also expedited by the following derivations.

MCI1 (in Proportions) MCI1 =

σ R& R σt

Based on the equation at the top of this page, σ 2p = σ t2 − σ R2 & R − observed = Then − actual =

− actual =

USL − LSL USL − LSL and − actual = 6σ t 6σ p USL − LSL 6

σ t2

− σ R2 & R

=

USL − LSL 6σ t 1 −

σ R2 & R σ t2

=

− observed

σ R2 & R 1− 2 σt

− observed 1 − ( MCI1 )

2

This is the equation used to develop Figure 8.1, page 34, to show the effect of MCI1 on .

11.H1149.Appen(051-072).qxd:11.H1149.Appen(051-072).qxd

6/1/11

8:35 AM

Page 71

Appendixes

71

MCI2 (in Proportions) MCI2 = Then σ t =

USL − LSL 5.15σ R& R and − observed = USL − LSL 6σ t USL − LSL 6 − actual 2

σ 2actual

⎛ USL − LSL ⎞ 2 =⎜ ⎟ − σ R& R C 6 ⎝ p − observed ⎠

− actual =

=

And finally − actual =

USL − LSL = 6σ actual

USL − LSL 2

⎛ USL − LSL ⎞ 6 ⎜ − σ 2R& R ⎟ ⎝ 6 − observed ⎠ 1 2

⎞ ⎛ σ R& R ⎞ 2 ⎛ 1 6 ⎜ ⎟ − ⎝ 6 − observed ⎠ ⎝ USL − LSL ⎠ 1 2

⎛ ⎞ ⎛ MCI2 ⎞ 2 1 6 ⎜ ⎟ − ⎝ 6 − observed ⎠ ⎝ 5.15 ⎠

This expression was used to generate Figure 8.2 on page 34. Note that some combinations are not possible.

12.H1149.Ref(073-074).qxd:12.H1149.Ref(073-074).qxd

6/1/11

8:35 AM

Page 73

References

Automotive Industry Action Group. Measurement Systems Analysis. 2nd ed. Dearborn, MI: Automotive Industry Action Group, 1995. Duncan, A. J. Quality Control and Industrial Statistics. 4th ed. Homewood, IL: Richard D. Irwin, 1974. Hicks, C. R. Fundamental Concepts in the Design of Experiments. 3rd ed. New York: Holt, Rinehart and Winston, 1982. Wheeler, D. J., and D. S. Chambers. Understanding Statistical Process Control. Knoxville, TN: SPC Press, 1986. Wheeler, D. J., and R. W. Lyday. Evaluating the Measurement Process. 2nd ed. Knoxville, TN: SPC Press, 1989.

73

13.H1149.Index.(075-076).qxd:13.H1149.Index.(075-076).qxd

5/31/11

10:24 AM

Page 75

Index

A

E

actual process variation, 3 analysis of variance (ANOVA), 5, 27–31 ANOVA table, 58 analysis, of results, 19 ANOVA. See analysis of variance appraiser variation, 6, 8 attributes testing, 41 AV. See appraiser variation averaging, 23

electronic temperature equipment, 39–40 electronic width gage, 38–39 equipment variation, 6, 8 errors, caused by R&R, 14 EV. See equipment variation

B

G

bore gage, 45–46

gage, definition, 1 gage specificity, 21 gage variability, components of, 1–2 General Motors long form. See standard procedure go/no-go testing, 41

F flatness, measuring, 40–41

C calibration, 17 chemical analyses, 40 chemicals, 40 chi-square relationship, 64 component values, 12 control charts, 9–10 appendix G, 66–67 distortion of, 34 and measurement capability indices, 35 See also process capability index k, 15

I interaction assumption, 28 intervals, 5

K K1 factors, 54–55 K2 factors, 55–56

M

D

MCI. See measurement capability indices MCI1, 8–9, 11–13 MCI2, 13–14 measurement capability indices, 8–9, 11 comparison of, 15 and , 35

degrees of freedom, 64 device manufacturer’s specifications, 25 device. See gage dimensions, measuring, 43 discrimination ratio, 12

75

13.H1149.Index.(075-076).qxd:13.H1149.Index.(075-076).qxd

76

5/31/11

10:24 AM

Page 76

Index

distortion of, 34 and process capability, appendix H, 68–69 process specifications, 13–14 process variation, 11–13 R&R effect on, appendix I, 70–71 measurement discrimination, 20 measurement systems, sources of, 1 measurement variation, 1, 3 measurements, individual, 19 micrometers, 43 moisture, measuring, 46–47

repeatability, 8 definition, 2 and K1 and K2 factors, appendix C, 54–56 repeatability and reproducibility, 8 repeatability interval, 6 reproducibility, 8 definition, 2 derivation of calculation of, appendix D, 57–59 results, analyzing, 19

N

sample size variation, factors for, appendix B, 53 sample variation, 18–19 appendix E, 60–63 samples, 18 scales, 44–45 sheet flatness, 40–41 sigma, for R&R, 3 special causes, 10 specifications, device manufacturer’s, 25 standard deviations, 8 estimating, appendix F, 64–65 standard procedure, 5 format, 6 report, appendix A, 52 surface texture, measuring, 42

nuclear moisture gage, 46–47

O observed process variation, 3 operators, 17–18

P performance statistics, 8 physical properties, measuring, 41–42 physical tests, 41–42 process capability, 33 components of, 2 effect of R&R on, 33–34 and measurement capability indices, appendix H, 68–69 process capability index, 14 process control, goal of, 1 process specifications, 5–6, 13–14 process variation, 1, 3, 5–6 measured, 12 and measurement capability index, 8–9 sources of, 2 profilometers, 42

R R&R, 8 effect on measurement capability indices, appendix I, 70–71 effect on process capability, 33–34 errors caused by, 14 R&R distribution, 3 R&R study definition, 1 typical procedure for, 6–8 range chart, 9 ranges, 5 R-chart, and ANOVA, 27

S

T temperature, measuring, 39–40 thickness, measuring, 38 tolerance range, 13–14 tolerances, one-sided, 20 trials, 5, 18 tubing, inside diameter, measuring, 45–46

U unacceptable rating, 13 unacceptable results, 23–24

V variation, within sample, 2–3, 18–19 appendix E, 60–63

W weight, measuring, 44–45 width, measuring, 38–39

X x-ray gage, 38