"Nondestructive Evaluation System
Reliability Assessment"

mh1823 Home Page

  1. Background
  2. QNDE Theory
  3. mh1823 POD a vs a Menu
  4. mh1823 POD Hit/Miss Menu
  5. How to Download R
  6. Install mh1823 POD software
  7. Special problems with Field-Finds
  8. POD "floor" and POD "ceiling"
  9. POD Short course/Workshop


POD Short Course/Workshop

This two-day short course is based on the new (2009) MIL-HDBK-1823A "Nondestructive Evaluation System Reliability Assessment" and uses the mh1823 POD software.  The course provides the latest methods for measuring your NDE system's effectiveness and the workshop will use these state-of-the-art techniques to analyze your data.

Course layout is reverse-chronological ? we discuss the analysis before we discuss how to design the experiment to produce the results we are analyzing. We will work through examples using real data, and time will be allocated for analyzing your enterprise-specific data.


Course Content Details ? Day One:

30 Years of Quantitative POD History (Understanding how we got here.)

    1970s Have Cracks ? Will Travel

    Early 1980s ? Flight propulsion manufacturers? individual efforts to improve POD analysis

    Late 1980s ? USAF, UDRI, GEAE, P&W, and Allied-Signal (now Honeywell) working group produced MIL-HDBK-1823, ?Nondestructive Evaluation System Reliability Assessment? draft.  I was the editor and lead author.

    1993 ? NATO AGARD sponsored 2-day POD Short Course based on MIL-HDBK-1823 that I presented in Ankara, Turkey, Lisbon, Portugal, Patras, Greece, and Ottawa, Canada.

    Late 1990s ? USAF officially publishes MIL-HDBK-1823, 30 April, 1999

    Early 2000s ? Model-Assisted POD gains a following

    February, 2007 ? Draft of revised and updated MIL-HDBK-1823 released for comment, with all-new software incorporating the latest statistical best practices for NDE data.

    7 April, 2009 ? The 2007 update was released by the USAF as MIL-HDBK-1823A.


Probability and Confidence

    What is Probability?  (Two incompatible definitions; both are correct)

    What is Probability of Detection?

    What is Confidence and how is that distinct from Probability?

    What is likelihood?  How is it related to, but distinct from, probability?

    What does ?90/95? really mean?

        Are all methods for assessing ?sub>90/95 equally effective? (Answer: No.)

    2 kinds of NDE data. (There are more, but this is a two-day course)


How to install the mh1823 POD software

This  short-course comes with a self-contained CD with R installed along with the necessary ancillary R routines (RColorBrewer, rcom and RODBC), the installed mh1823 POD software, and the example datasets ? everything.  You open the CD, drag the mh1823 POD icon to the desktop and you?re up and running. You only need to put the icon on the desktop once.  Next time, just click the icon and begin.  (We will, for completeness, spend some class time to demonstrate how to install R from the internet, and then how to install the mh1823 POD package.)

How to analyze ?vs a data


        The ?ideal? POD(a) a curve

        Why ?vs a data is different from Hit/Miss data

        When ?/i> is less informative than simple Hit/Miss

    ?vs a Data Analysis

        Read ?vs a data

         Preliminary Data Assessment: Plot the data and choose the best ?vs a model.

        Build the ?vs a linear model

         Four ?vs a Requirements (Warning: If any of these assumptions is false, or, if the model is a line and the data describe a curve, then the subsequent POD analysis will be wrong even though the computational steps are correct.)

        How to go from ?vs a to POD vs a ? The Delta Method

         Compute the transition matrix from ?vs a to POD vs a

         The POD(a) Curve

        Wald method to compute ?vs a confidence bounds

         Plot POD(a); compute POD confidence bounds

    Classwork ?

        Analyze a simple ?vs a example.

        Effects of analysis decisions on a90/95


How to Analyze ?vs a data with Repeated Measures (Multiple inspections of the same Target Set)

    Why repeated measures are not simply ?more data?

        Red apples and green apples


    Special Situations

        How to recognize pathological ?vs a data (which is unfortunately common)

        Special difficulties with Field-Finds ? When mh1823 methods are not enough


How to Analyze Noise

    Understanding Noise

    Definition of Noise

    Choosing a probability density to describe the noise

    False Positive Analysis (with ?vs a data)

        Noise analysis and the Combined ?vs a Plot

        The POD(a) Curve

        Miscellaneous mh1823 POD algorithms

    Analyze the noise; compute the false-positive rate


Analysis of enterprise-specific ?vs a Data

    Hands-on individual POD problem-solving


Day Two:

How to analyze Binary (Hit/Miss) Data

    Understanding binary data ? why ordinary regression methods fail

    Read Hit/Miss data

    Build the GLM (Generalized Linear Model)

        Understanding Generalized Linear Models

        Choosing Link Functions

    Hit/Miss Confidence Bounds

        Not all statistical confidence methods are equally accurate

        How the LogLikelihood Ratio Criterion Works

        How to compute likelihood ratio confidence bounds

        Constructing Hit/Miss Confidence Bounds

    Classwork ?

        Analyze a simple Hit/Miss example.

        Effects of Hit/Miss analysis decisions on a90/95

    Special Situations

        Choosing an Asymmetric Link Function

        How to analyze Repeated Measures

        How to analyze Disparate Data correctly

        How to analyze Hit/Miss Noise

        How to recognize Hit/Miss pathological data


Analysis of enterprise-specific Hit/Miss Data

Statistical Design Of eXperiments (DOX)

    What is Statistical Experimental Design?

    Variable types

    Nuisance variables

    Objective of Experimental Design

    Factorial experiments

    Categorical variables

    Noise ? Probability of False Positive (PFP)

    How to Design an NDE Experiment

        Philosophy of NDE demonstrations

        How many specimens are enough?

        Specimen Design, Fabrication, Documentation, and Maintenance

         Examples of NDE Specimens


Other Important Topics:

    False Positives, Sensitivity and Specificity

    Receiver Operating Characteristic (ROC) Curve

    Model-Assisted POD (MAPOD)

    Data that do not meet MIL-HDBK-1823 requirements

        min(POD) > 0 or max(POD) < 1

        Floor, Ceiling POD(a) models


Training Review & Course Wrap-up


POD Short Course/Workshop

Please send me more information on setting up a POD Short Course and Workshop at my workplace.

My e-mail address is ...

My name is ...

Our POD interest is focused on ...


SElogo.gif (5955 bytes)

[ HOME ] [ mh1823 homepage ]  [ Feedback ]

Mail to

Copyright ? 1998-2009 Charles Annis, P.E.

Last modified: June 08, 2014