Evaluation of a Sample Design

It is almost always desired that a sample design be evaluated for its perfection and a perfect sample design is excepted to meet certain criteria which include among others, the criteria of accuracy, reliability, validity and efficiency. We provide below a brief account of these concepts.


Accuracy: The accuracy of a sample estimate refers to its closeness to true population value. The closer the sample estimate to the population value, the greater is its accuracy. In our foregoing example with four population values 10, 17, 21 and 24, the population mean is 18, while the sample mean base on the observations 10, 24 and 17. The difference between these two means in an indication of inaccuracy in the estimate. If the draw results in the selection of the observations 10 and 21, the sample mean is 15.5, which is further away from the true mean and hence the estimate is more inaccurate. The accuracy of an estimate is generally assessed on the basis of its mean square error (MSE). The smaller the MSE of an estimator, the greater is its accuracy.

Reliability: if we assume that there is no measurement error in the survey, then the reliability or precision of an estimate can be stated in terms of its sampling variance or equivalently, of its standard error. The standard error measures the precision with which the estimate from a particular sample approximates the hypothetical average result from all possible samples. The smaller the standard error of an estimate, the greater is its reliability. Samples with high precision are regarded as efficient samples.

Validity: if we assume that there is no measurement error in the survey, then the validity of an estimator can be evaluated by examining the bias of the estimator. The smaller the bias, the greater is the validity. The validity of an estimated population characteristic thus refers to how the mean of the estimator over repetitions of the process, yielding the estimate differs from the true value of the parameter being estimated.

Efficiency: The criteria of efficiency are related to the cost of sampling. A sampling design is considered to be more efficient than another, if the former results in lower costs than the later design, with the same degree of reliability.

The discussion above helps us to set criteria to identify a good sample design. We speak of these criteria with reference to only probability sampling methods, because probability-sampling methods are the only sampling plans that allow us to assess the reliability of the estimates to be derived from the sample data. Keeping this in view, we summarize below what a sample design requires to quality as a good sample design.

(a) A good sample design should be oriented to the research objectives in terms of its selection and estimation of the population values. Furthermore, it must have the compliance with the survey design and suit to the survey environment.

(b) A good sample design should allow statistical inference to draw regarding the population values. This is possible only, when the sample is probability sample. A design must allow us to measure valid estimates of its sampling variability; which is ordinarily expressed with SE or MSE.

(c) A sample design must judge in terms of its practicability. This means that a good design is one, which permits execution with simplicity, clarity, practicability and completeness.

(d) Economy is another aspect of a sample design. A good design must therefore involve lowest cost for the fulfillment of the survey objectives, which are commonly stated in terms of the variance of the estimates.