Title: Click here to view
Course: PEBL 101: RESEARCH METHODOLOGY & DESIGN FOR BUSINESS > Section: UNIT 4b: SAMPLING (Ctd…)

Unit 4b Learning Outcomes

Indicative Content

Non response, Determining Sample size, Sampling and non-sampling errors

Intended Learning Outcomes for Unit 4b:

By the end of Unit 4b, you will be able to;

1. explain the factors that determine sample size
2. determine an appropriate sample size
3. identify sampling and non-sampling errors.

Unit 4b E-tivities

Outcome(s) addressed: Unit 4b addresses Module Learning Outcomes 2 & 3.

Purpose: As already learnt in Unit 4a, sampling is necessary for different reasons. However, one of the challenges is to select a suitable number of elements to include in the sample. This section will enable you learn how to determine how many elements to include in a sample and the errors that arise in sampling.

Over to you: In this Unit, you will describe the different ways of how to determine sample size and the sampling errors.  Although it is a relatively short unit, the concepts it introduces are vital for completing the concept paper in Unit 2 and the activities in Unit 8. There you go!

Preparatory research: In order to be able to select suitable sample sizes;

1. Read the minilecture on determining sample size here.
2. Read minilecture on sampling and non-sampling errors here.
3. Read minilecture on non-response here.
4. Read about sample size on pages 292 - 295 of Research Methods for Business by Sekarani (2003) available here.

E-tivity 4.4: Search the internet and identify appropriate considerations needed to be taken into account when determine sample size. Post to the personal portfolio by 13^th March 2019.

E-tivity 4.5: Using a search engine like www.google.com and any other resources accessible to you, read and prepare a summary of 200 words on how to reduce non-response. Post to the group discussion forum by 15^th March 2019.

E-tivity 4.6: In the group discussion forum, create another new discussion topic in your group forum titled ‘E-tivity 4.6’. Under it, discuss and determine the sample size that is suitable for the concept paper that you are developing. Group members should discuss and agree on the most suitable sample size. Select a weaver for this section who should summarise the final solution for E-tivity 4.6 and post it under the discussion topic titled ‘Concept Paper’ that you created under E-tivity 2.1 by 17^th March 2019.

Link to other Units: Unit4a links to Units 1, 2 and 8.

Anticipated time required for Unit 4b activities: 4 hours

Deadlines for Unit 4b activities:

E-tivity 4.4 - Considerations for determining sample size: 13^th March 2019.

E-tivity 4.5 – How to reduce non-response: 15^th March 2019.

E-tivity 4.6 – Including sample size in Concept Paper: 17^th March 2019.

DETERMINING SAMPLE SIZE

Determining the sample size, n, is an important aspect of the sampling design issue.A reliable and valid sample should enable us to generalise the findings from the sample to the population under investigation.In other words, the sample statistics should be good estimates and reflect the population parameters as closely as possible within a narrow margin of error.

No sample statistic is going to be exactly the same as the population parameter, no matter how sophisticated the probability sampling design may be. Remember that the very reason for a probability design is to increase the probability that the sample statistics will be as close as possible to the population parameters. However, it is noteworthy that the larger the sample size, the higher the precision and confidence, ie, the more representative it is of the population.

Factors That Determine Sample Size

Sample size, n, is a function of

a)statistical considerations

b)non-statistical considerations.

a) Statistical Considerations

i) Nature of the population: Nature of the population means the amount of heterogeneity present in it. The greater the heterogeneity in the population, the bigger should be the sample. If all the units in the population have more or less similar characteristics, ie, if there is homogeneity, a small sample can represent all the elements. Variability in the population can be known by estimating its standard deviation. Estimates of the standard deviation are sometimes available from previous studies. If not, it can be calculated from a sample taken for this purpose or estimated at one-sixth of the population range if the population is normally distributed.

ii) Degree of accuracy or precision desired from the estimate: Precision is the amount of error that one is willing to tolerate in the sample estimates. For example, a researcher investigating mean income might want the sample estimate to be within ± Shs.50,000 of the true population value. In general, the larger the sample, the smaller the error or the higher the degree of accuracy although accuracy does not increase in direct proportion to size. Rather the accuracy of a sample increases in proportion to increase in the square root of the number of items in the sample. Therefore, to double the accuracy, we must quadruple the size of the sample. The reliability of a sample increases at a lower rate than the increase in the size of a sample and after a certain point, the benefit may not be worth the cost. Researchers should always assess both the worth and the cost of the additional precision they will obtain from a large sample before they commit resources to take it.

iii) Degree of confidence desired: This means how much confidence we want so that the error is not any greater than what was estimated. Do we with to be almost certain or be satisfied with say, 95% or 90% degree of confidence that the specified limits will contain true parameter? To what extent do the findings from the sample hold true for the population? A higher confidence level will require a higher sample size.

iv) Population size

b) Non-statistical considerations

i) Resources available: Gains in precision associated with large samples need to be weighed against the costs such large samples might entail. Gains must be commensurate with the amount of time and money involved in doing research on big samples. There should be a trade-off between added information and added costs.

ii) Nature of study: For intensive and longitudinal studies, small samples are enough but not so for extensive and one-time studies. Pilot studies require small sample sizes.

iii) Method of sampling followed: Smaller but properly selected samples are superior to large but badly selected samples. Probability sampling is better than non probability sampling, the former needs smaller samples.

iv) Nature of respondents and other field conditions: Where the non-response rate is expected to be high, the size of the sample should be big and vice versa.

v) Urgency with which results are needed

Sample Size Ctd....

Methods of determining sample size

These are several methods and some of them include;

1. Saunders, Lewis and Thornhill (1997) proposed the following formula for determining sample size;

Sample Size= P x Q x (Z /E)^2

Where P = Number of target population that conforms to the characteristics of the sample required

Q = Number of target population that does not conform to the characteristics of the sample required.

Z = Confidence Level required (eg 95%)

E = Margin of error (eg 5%)

2. Using previous studies in the same area: This method of determining sample size was used and recommended by Saunders, Lewis and Thornhill (1997) and Hill (1998). The average of previous studies when computed gives a sample size that can be used for the current study.

4. Sample size Calculators online

These are computer generated and some of them can be accessed online at

a. http://www.surveysystem.com/sscalc.htm

b. http://www.dssresearch.com/toolkit/sscalc/size_a1.asp

5. Krejcie and Morgan (1970) proposed a table for determining the sample size based on confidence level needed from a given population.

From the table it is noted that as the population increases the sample size increases at a diminishing rate and remains relatively constant at slightly more than 380 cases. (See Table below)

Table for Determining Sample Size from a Given Population

  Source: Krejcie and Morgan (1970:608) in Hill (1998).

Where N= Population size, and n= sample size required.

LARGE OR SMALL SAMPLE SIZES?

Isaac and Michael (1995) provide the following conditions where research with large samples is essential and also where small samples are justifiable:

Large Samples are essential:

(a) When a large number of uncontrolled variables are interacting unpredictably and it is desirable to minimise their separate effects; to mix the effects randomly and hence cancel out imbalances.

(b) When the total sample is to be sub-divided into several sub-samples to be compared with one another.

(c) When the parent population consists of a wide range of variables and characteristics, and there is a risk therefore of missing or misrepresenting those differences - a potential with e-survey, considering that the parent population is now global.

(d) When differences in the results are expected to be small.

Small sample sizes are justifiable:

(a) In cases of small sample economy. That is when it is not economically feasible to collect a large sample.

(b) When computer monitoring. This may take two forms: (i) Where the input of huge amounts of data may itself introduce a source of error - namely, key punch mistakes. (ii) Where, as an additional check on the reliability of the computer program, a small sample is selected from the main data and analysed by hand. The purpose of this is to compare the small sample data, and the large sample data for similar results.

(c) In cases of exploratory research and pilot studies. Sample sizes of 10 to 30 are sufficient in these cases. They are large enough to test the null hypothesis and small enough to overlook weak treatment effects. Statistical significance is unlikely to be obtained on this size sample however.

(d) When the research involves in-depth case study. That is, when the study requires methodology such as interview and where enormous amounts of qualitative data are forthcoming from each, individual respondent.

Presumably, to these may be added the converse of those listed under 1, above. Namely: when control is extremely tight and interacting variables are neither large in number nor unpredictable; when the population is homogeneous; when differences in the results are expected to be very large.

Gay & Diehl (1992) state that in one way the typically smaller sample sizes used in applied or practical research have a redeeming feature. Their argument states that large sample sizes enhance the likelihood of yielding statistically significant results. Thus with very large sample sizes, a very small difference between means may yield a significant result, and yet be of little practical use. It results in research that Bannister (1981) described as specifically and precisely irrelevant. On the other hand, if you obtain a statistically significant result from a small sample size, then the impact of the difference is probably more obvious and useful - but this is admittedly tenuous argument and Gay & Diehl advise care in interpretation of results.

NON-RESPONSE

This refers to a situation where elements of the population selected into the sample do not respond to the investigation. Non response implies that some variation that may be relevant to the study is being left out there fore there is need for concerted effort on the part of the researcher to keep the proportion of non-response at minimum.

Non-response rate =  No. of respondents/elements that do not answer the questions (respond) X 100

 The total No. of respondents/elements who were approached

Increasing the response rate.

Response rates for face-to-face interviews are typically in the 75% range, and for mail surveys 10% is considered good in some marketing surveys. However, "good" depends on purpose and resources of the polling organization. It is noteworthy that a high response rate is essential to reliable statistical inference. Response rate can be increased by;

1. Having legitimate sponsorship for the survey, geared to sponsors who are highly-regarded in the community being surveyed

2. Having a good, short explanation justifying the survey

3. Notifying individuals in advance that a survey is coming

4. Keeping the survey instrument short, and letting prospective subjects know this.

5. Assure confidentiality and anonymity.

6. Offer to reschedule interviews to a time of the subject's convenience.

7. Make call-backs (four is typical) where needed. In mail surveys, provide a new copy of the instrument each time. Postcard and e-mail reminders also help. In mail surveys, plan to wait about six weeks for all responses to drift in. Reminders to respondents in form of phone calls.

8. Start the instrument with non-threatening questions which arouse interest.

9. Offer to provide final results later on.

10. In mail surveys, use certified and express mail. In e-mail surveys, use priority e-mail.

11. Offer token remuneration (eg., $1 enclosed in mail surveys; $5 in face-to-face interviews; or use pens, keychains, calendars, or other tokens to make respondents feel like they are getting something in return). Linking the return of the survey to entrance into a lottery for a substantial prize (eg., $1,000) can also boost response rates.

12. Getting information about non-responding individuals from third parties.

13. Addressed and Stamped return envelope

SAMPLING AND NON-SAMPLING ERRORS

Sampling Error

This is the difference between the sample statistic and the population parameter. When a certain statistic is calculated from a sample and is used as an estimate of the population parameter, the researcher subjects him/herself to this type of error. This error arises due to the fact that only part of the population has been used to estimate the population parameter. As such, this error would not be present in a census.

Non-sampling Errors

These errors arise due to factors other than the inductive process of inferring about the population from a sample. Some important factors which give rise to these errors are as under;

a) Defective frame and faulty sampling.

b) Inappropriate methods of data collection, coding and tabulation.

c) Untrained investigators

d) Inadequate inspection and supervision

e) Incomplete coverage

Whereas all sampling errors are random and compensatory, all non-sampling errors are systematic and cumulative. Further, whereas sampling errors occur only in a sample study, non-sampling errors occur both in sample as well as census studies. Finally, whereas sampling errors can be reduced by increasing sample size, non-sampling errors can be reduced by suitably improving the collection and compiling techniques and streamlining the training and supervision of field staff.

In the end, it should be appreciated that it is the reduction of the total error rather than of any single type of error which should be aimed at in designing a research investigation. Hence, it is not always enough to increase the sample size with the hope of reducing sampling errors, which are just one type of errors. By increasing the sample size, there is no doubt that sampling errors are reduced but this may increase non-sampling errors because the larger sample requires more interviews and creates additional burdens in selection, training and control. Hence the gain achieved in one direction may be offset by the loss suffered in the other direction.

REVISION QUESTIONS

1. Suppose you wanted to study the vocabulary of employee evaluations in large American corporations. How would you draw a sample? What sources of bias might affect your sample?

2. What is sampling bias?

3. With short explanations and/ or reasons, state whether the following statements are true or false;

1. There is no significant difference between convenience sampling and purposive sampling.
2. Both quantitative and qualitative researchers who use samples must provide detailed information about the purposive research participants and how they were chosen.
3. The size of the sample influences both the representativeness of the sample itself and the statistical analysis of study data.
4. Sampling error reflects sloppy research.
5. A good researcher can avoid sampling bias.
6. The important difference between convenience sampling and purposive sampling is that, in the latter, one needs clear criteria/ guide in selection of the sample
7. A “good” sample is one that is representative of the population from which it was selected.
8. A simple stratified random sample guarantees that each subgroup is represented in the same proportion that it exists in the population.
9. In a systematic sample, the researcher selects K items.
10. A table of random numbers selects the sample through a purely random, or chance, basis.
11. Purposive sampling does not require the researcher to describe in detail the methods used to select a sample.
12. It is possible to defend purposive samples because the researcher uses clear criteria (e.g., experience and prior knowledge) to identify criteria for selecting the sample.
13. Qualitative research uses sampling strategies that produce samples which are predominantly small and nonrandom.
14. A good sample has a composition precisely identical to that of the population.
15. Cluster sampling oftentimes is the only feasible method of selecting a sample because the population is very large or spread out over a wide geographic area.

Fill in the blank spaces

1. A______ population is a group with different characteristics among its members.
2. _______ is the process of sampling that is completed in stages, involving the selection of samples within samples.
3. _____ is the mental process by which findings from a smaller group are generalized to a larger group.
4. _____ are the characteristics of the sample.
5. A______ population is a group with similar characteristics among its members.
6. A _____ is the group to which research findings are generalizable.
7. _____is the extent to which the results of one study can be applied to other populations or situations.
8. _____ sampling is where identified subgroups in the population are represented in the same proportion that they exist in the population.
9. _____ sampling involves selecting a few individuals who can identify other individuals who can identify still other individuals who might be good participants for a study.
10. ______ sampling selects intact groups, not individuals having similar characteristics, and interview 100% of the elements in the groups.
11. ________ sampling means that all individuals in the defined population have an equal and independent chance of being selected for the sample.
12. ________ sampling is a sampling process in which individuals are selected from a list by taking every Kth name.

Essay type

1. A generalisable study should consider a large sample because the larger the sample the more representative it is of the population. Explain the factors you would consider when determining sample size.

2. Discuss the view that simple random sampling is the only probabilistic sampling method applicable in all situations.

3. Compare Stratified sampling and snowball sampling.