Are More Out-of-Control Rules Always Better? — With All Eight Rules Enabled, False Alarm Intervals Shrink from 370 to 85 Points
Control chart software defaults to enabling all eight out-of-control rules, which may seem very "rigorous." However, each additional rule increases the likelihood of false alarms. When alarms go off so frequently that no one bothers to look up, SPC (Statistical Process Control) is reduced to merely a charting function.
The following data comes from a Monte Carlo simulation: n=5 subgroups, 3σ control limits, two-sided test, 1500 segments, each segment with 3000 points. The data shows the average number of subgroups before a false alarm (ARL0, or the average run length when the process is in control). You can directly compare these numbers with the actual alarm frequency at your site.
1. First, Look at Alarm Density: Each Additional Rule Doubles the False Alarm Rate
| Rules Enabled | ARL0 (Average Run Length in Control, Points) | False Alarms per Day (96 Points/Day) |
|---|---|---|
| Only Rule 1 (Single Point Beyond 3σ) | Approximately 375 | 0.26 |
| Rule 1 + 2 (9 Consecutive Points on the Same Side) | Approximately 225 | 0.43 |
| Rule 1 + 2 + 6 (4 out of 5 Points Beyond 1σ) | Approximately 136 | 0.71 |
| Rule 1 + 2 + 3 + 6 | Approximately 111 | 0.86 |
| Rules 1 to 6 (Including Alternating and 2 out of 3 Beyond 2σ) | Approximately 100 | 0.96 |
| All Eight Rules | Approximately 85 | 1.13 |
Interpretation: Using only Rule 1, a false alarm occurs on average every 375 points; with all eight rules enabled, a false alarm occurs every 85 points. The sensitivity increases several times, but the false alarm rate quadruples. Rule 6 (4 out of 5 Points Beyond 1σ) and Rule 2 (9 Consecutive Points on the Same Side) are the main contributors to reducing ARL0, as they are sensitive to small shifts of 1 to 1.5σ, but they are also the most likely to be triggered by normal minor fluctuations.
An often overlooked fact is that when Rules 2, 4, 5, and 6 are triggered, several consecutive points are often in an "alarm state." On a chart, this might appear as three or four red points, but they actually represent the same event. Therefore, the number of alarm points and the number of alarm events should be counted separately. Counting only the red points overestimates the number of issues, while counting only the emails underestimates it.
2. The Real Cost of False Alarms: Over 20 Days of Unnecessary Work Per Year
Consider a specific scenario: a process samples 4 subgroups per hour, operates in two shifts for 24 hours, and has 96 points per day, running 250 days a year.
- All Eight Rules Enabled: 96 ÷ 85 ≈ 1.13 false alarms per day, approximately 282 per year.
- Only Rules 1 + 2 Enabled: 96 ÷ 225 ≈ 0.43 false alarms per day, approximately 107 per year.
The difference is 175 "investigations that find nothing" each year. Assuming each false alarm involves two people (one from process and one from quality) investigating for 40 minutes, 175 × 40 = 7000 minutes, or about 117 labor hours, equivalent to one person working for 15 full days. If each false alarm also involves a 30-minute production stop for confirmation, the loss doubles.
More expensive than labor hours is the cost of trust. Once the false alarm rate exceeds "once every two to three days," the team's response will degrade from "stopping production to investigate" to "clicking confirm and writing a note." When a real anomaly occurs—such as a tool breaking or a mixed batch of incoming materials—the red point will look identical to the previous 30. The true cost of false alarms is not the investigation time but the fact that they drown out real alarms.
3. Select Rules Based on Characteristic Importance: Three Levels of Activation
Do not use the same set of rules for the entire factory. Divide them into three levels based on characteristic importance and response cost:
| Level | Applicable Scope | Recommended Rules | ARL0 | Expected False Alarms |
|---|---|---|---|---|
| Conservative Level | General dimensions, appearance characteristics, high point count, high manual response cost | Rules 1 + 2 | Approximately 225 | Once every 2 to 3 days |
| Standard Level | Key characteristics (CC), customer-specified SPC items | Rules 1 + 2 + 6 | Approximately 136 | Less than once per day |
| Sensitive Level | Safety characteristics, large batches, processes where downstream cannot intercept | Rules 1 + 2 + 3 + 5 + 6 | Approximately 100 | About once per day, must be paired with rapid response |
Three judgment principles:
- The more points, the fewer rules. For processes that automatically collect data every 5 minutes, resulting in 288 points per day, enabling all eight rules will lead to more than 3 false alarms per day, inevitably causing "alarms that are ignored."
- The slower the response, the fewer rules. If an alarm requires waiting for a process engineer to come from another workshop, the sensitive level should not be used—sensitivity that no one can handle is equivalent to not being enabled.
- Safety characteristics are not protected by the number of rules but by the response time. The premise of the sensitive level is that there is a mechanism to reach the site within 15 minutes. Otherwise, using the standard level with a denser sampling interval is more practical.
Do not expect to catch small shifts (slow drifts within 1σ) by piling on more rules. The cost of adding rules is an exponential increase in false alarms, while charts like EWMA and CUSUM are specifically designed to detect small shifts. Using the right chart is more cost-effective than enabling all rules.
4. Tiered Alarm Response: Which Ones Require a Production Stop, Which Ones Only Need Recording
Different rules indicate different issues and should not be treated with the same response. It is recommended to explicitly define this table in the control chart work instruction:
| Level | Trigger Conditions | Response Time | Actions | Record Requirements |
|---|---|---|---|---|
| L3 Production Stop Level | Rule 1 (Single Point Beyond 3σ), Rule 5 (2 out of 3 Points Beyond 2σ) | Immediate | Stop the workstation, isolate products since the last known good point, and follow the response plan | Abnormal handling form + cause + disposition conclusion |
| L2 Verification Level | Rule 2 (9 Consecutive Points on the Same Side), Rule 6 (4 out of 5 Points Beyond 1σ) | Within 30 minutes | Verify records of tool changes, batch changes, personnel changes, and material changes to confirm if there is a real shift; if confirmed, escalate to L3 | Verification record (including investigation items) |
| L1 Observation Level | Rule 3 (6-point trend), Rule 4 (14-point alternating), Rule 7 (15 points within 1σ), Rule 8 (8 points outside 1σ) | Within the current shift | Do not interrupt production, record in the trend board, and escalate to L2 if the same trend continues for two shifts | Trend notes |
The key is that the tiered response must be documented. Simply writing "determine abnormalities based on out-of-control rules" will lead to the most convenient execution on the floor: either ignoring all alarms or stopping production for all. Neither is sustainable.
5. Three-Step Back-Calculation Before Launch: Verify with Three Months of Normal Data
When setting up new rules or changing the control chart type, do not start immediately. Instead, use historical data for back-calculation, which takes about half a day and can save six months of debate.
Step 1: Collect Data. Gather raw subgroup data from the past 3 months, excluding known abnormal periods (such as model changeover, major equipment maintenance, and special incoming material batches), ensuring the data is from a "controlled period." The sample size should be at least 300 subgroups; otherwise, the back-calculation results will lack credibility.
Step 2: Back-Calculate with Proposed Rules. Apply the selected combination of rules, count the number of alarms and alarm events (count consecutive triggers as one event), and calculate ARL0 = total points ÷ number of alarm events.
Step 3: Compare and Determine. If the back-calculated ARL0 deviates from the theoretical value by ±30%, it indicates that the subgroup division and data quality are acceptable, and the rules can be implemented. If the deviation exceeds 2 times (e.g., theoretical 136, back-calculated 50), investigate three things: whether different shifts/batches are mixed within subgroups, whether control limits are calculated using mixed data, and whether the measurement system has insufficient resolution. Address these issues before adjusting the rules.
Back-calculation also provides a side benefit: it can determine the "actual handling cost of each false alarm" at your site. Combining this number with ARL0 makes the selection of rule levels a clear management decision, not just a preference of the quality department.
The selection of out-of-control rules is not about "how strict" but about "how many false alarms you are willing to accept for increased sensitivity" — first, do the math, then decide how many rules to enable.
Knowledge code: 6.3.1
Version: v20261022
Author: QTank QTank is dedicated to providing systematic professional knowledge, methodologies, and practical tools for quality management practitioners, helping companies continuously improve their quality capabilities.