2 Research questions

In this chapter, you will learn to:

identify and write quantitative research questions.
identify the variables implied by a quantitative research question.
identify observational and experimental studies.
describe and identify the units of analysis and units of observations in a study.
write operational and conceptual definitions.

2.1 Introduction

The RQ directs all other components of the research, so writing clear and answerable research questions (RQs) is important. Quantitative research summarises and analyses data using numerical methods (like averages or percentages), so the RQ must be written carefully so they can be answered effectively. In this book, four different types of RQs are studied:

descriptive RQs (Sect. 2.2);
relational RQs (Sect. 2.3);
repeated-measures RQs (Sect. 2.4);
correlational RQs (Sect. 2.6).

2.2 Descriptive RQs

All RQs identify a large group of interest to be studied (a population; Sect. 2.2.1), and study something about that population (the outcome; Sect. 2.2.2).

2.2.1 The population

The population is any broad group of interest; for example:

all German males between $18$ and $35$ years of age.
all bamboo flooring materials manufactured in China.
all elderly females with glaucoma in Canada.
all Pinguicula grandiflora growing in Europe.

Definition 2.1 (Population) A population is a group of individuals from which the total set of observations of interest could be made, and to which the results will (hopefully) generalise.

Populations comprise many individuals (sometimes called cases). If the individuals are people, individuals are also called subjects.

The words population, individuals and cases do not just refer to people, though they may be commonly used that way in general conversation.

The data are rarely taken from all the individuals in the population: all individuals are rarely accessible in practice. For example, testing a new drug cannot possibly study all people who might use the drug (some may not even be born yet). In contrast, a sample is a subset of the population from which data are obtained (Chap. 6). Countless samples are possible from any given population.

Definition 2.2 (Sample) A sample is a subset of individuals from the population from which data are collected.

The population in a RQ is not just those studied; it is the whole group to which results could generalise.

Example 2.1 (Samples) A study of American college women (Woolf et al. 2009) compared iron status in highly-active and sedentary women.

The sample comprised $28$ active and $28$ sedentary American college women, from which data are collected. The population is all active and sedentary American college women. The group of $56$ subjects is the sample.

Completely and precisely defining the population sometimes requires refining or clarifying the population, using exclusion and/or inclusion criteria.

Exclusion and inclusion criteria clarify which individuals are explicitly included or excluded from the population for the purposes of the study. Exclusion and inclusion criteria should be explained when their purpose is not obvious. Exclusion and inclusion criteria are not necessary; none, one or both may be used.

Definition 2.3 (Inclusion and exclusion criteria) Inclusion criteria are characteristics that individuals must meet explicitly to be included in the study.

Exclusion criteria are characteristics that explicitly disqualify potential individuals from being included in the study.

Example 2.2 (Inclusion and exclusion criteria) In a strength study study where the population is 'concrete test cylinders', cylinders with severe cracks may be excluded from strength tests.

In a study of exercise regimes where the population is people over $60$, severe asthmatics may be excluded from the study for health reasons.

In a study on the influenza vaccine, Kheok et al. (2008) listed the Population as 'health-care workers' (Kheok et al. 2008, 466), and the sample comprised healthcare workers at two specific hospitals. The population was refined using exclusion criteria: those (p. 466)

...declining to give consent, a history of egg protein allergy, and neurological or immunological conditions that are contraindications to the influenza vaccine.

Example 2.3 (Inclusion and exclusion criteria) Guirao et al. (2017) studied the walking abilities of amputees. Inclusion criteria included (p. 27):

... length of the femur of the amputated limb of at least $15$ cm measured from the greater trochanter; use of the prosthesis for at least $12$ months prior to enrollment and more than $6$ h/day...

Exclusion criteria included (p. 27) people with:

... cognitive impairment hindering the ability to follow instructions and/or perform the tests; body weight over $100$ kg...

2.2.2 The outcome

Descriptive RQs study something about the identified population, called the outcome. Because the RQ concerns a large group (the population), the outcome describes a group of individuals (not single individuals). Hence, the outcome is a numerical quantity (such as an average or percentage) summarising a group of individuals.

Definition 2.4 (Outcome) The outcome in a RQ is the result, output, consequence or effect of interest in a study, numerically summarised for a group.

The outcome of interest in a population may be (for example) the

average amount of wear after $1000$ h of use.
proportion of people whose pupils dilate.
average weight loss after three weeks on a diet.
percentage of seedlings that die.

The outcome in a RQ summarises a population; it does not describe the individuals in the population.

We can now introduce descriptive RQs.

Definition 2.5 (Descriptive RQ) Descriptive RQs have only a population and an outcome.

Descriptive RQs have one of these forms, depending on what information is sought:

Estimation RQs: Among {the population}, what is {the outcome}?
Decision-making RQs: Among {the population}, is {the outcome} equal to {a given value}?

These are not 'recipes', but guidelines.

Answering estimation descriptive RQs is studied in Chaps. 23 and 24. Answering decision-making descriptive RQs is studied in Chaps. 30 and 31.

Example 2.4 (Descriptive RQs) Mackowiak, Wasserman, and Levine (1992) studied men and women aged $18$ to $40$; this is the population. The outcome of interest in this population is the average body temperature. The sample comprised $148$ 'healthy men and women' aged $18$ to $40$.

One descriptive RQ was:

What is the average body temperature?

This is an estimation RQ. They also studied a decision-making descriptive RQ (where $98.6^\circ$F ($37.0^\circ$C) is a commonly-accepted value):

Is the average body temperature really $98.6^\circ$F ($37.0^\circ$C)?

2.3 Relational RQs

Studying relationships usually is more interesting than simply describing a population. Relational RQs compare the outcome for groups of different individuals in the population. These comparisons are called between-individuals comparisons, as they compare the outcome between (or among) groups of different individuals.

Examples include:

Comparing the average amount of wear in floor boards between two groups: standard wooden floor boards, and bamboo floor boards.
Comparing the average heart rates across three groups of people: those who received no dose of a drug, those who received a daily dose of the drug, and those who received a weekly dose of the drug.

Definition 2.6 (Comparison (between individuals)) The between-individuals comparison in a RQ identifies the small number of groups of different individuals for which the outcome is compared.

Example 2.5 (Between-individuals comparison) Bird et al. (2008) gave one group of participants a diet using refined flour, and gave another group of participants a diet using a new flour variety. Average faecal weight (the outcome) was compared for the two groups. 'Type of diet' is the between-individuals comparison.

Definition 2.7 (Relational RQ) Relational RQs have a population, outcome, and a between-individuals comparison.

Relational RQs have one of these forms, depending on what information is sought:

Estimation RQ: Among {the population}, what is the difference in {the outcome} for {the groups being compared}?
Decision-making RQ: Among {the population}, is {the outcome} the same for {the groups being compared}?

Example 2.6 (Relational RQs) Consider this RQ (based on Estévez-Báez et al. (2019)):

Among Cubans between $13$ and $20$ years of age, is the average heart rate the same for females and males?

The population is 'Cubans $13$ and $20$ years of age', the outcome is 'average heart rate', and the between-individuals comparison is between two separate groups: 'between females and males'. This is a relational RQ.

This is a decision-making RQ, since it asks if the average heart rate is the same for females and males. An estimation-type relational RQ would ask about the size of difference in the average heart rate between females and males.

2.4 Repeated-measures RQs

Rather than comparing the outcome for groups of different individuals, repeated-measures RQs compare the outcome multiple times within the same individuals.

These comparisons are called within-individuals comparisons, as they compare the outcome within the same individuals, not across groups of different individuals. The multiple measurements may be different points in time (e.g., the height of the same trees at one, two and five years after planting), but do not have to be time points.

Examples include:

Comparing the average strength of hind legs of horses to the fore legs of the same horses.
Comparing the average thickness of the cornea in left eyes and right eyes.
Comparing the average amount of wear in many individual floor boards after one, five and ten years of use.

Definition 2.8 (Within-individuals comparison) The within-individuals comparison in the RQ identifies the small number of different, distinct situations for which the outcome is compared for each individual.

Example 2.7 (Between- and within-individual comparisons) Consider comparing left- and right-leg strengths of professional football players.

A between-individuals comparison would compare the left- and right-leg strengths between different groups of footballers: one group would have their left-leg strength measured, and the other would have their right-leg strength measured. This is a between-individuals comparison.

In contrast, the left- and right-leg strength could be recorded on the same individuals. This study examines within-individuals changes: the differences between the left- and right-leg strength within individuals. In this study, no between-individuals comparison exists: different groups are not being compared.

Studies may use both within- and between-individuals comparisons (see Sect. 34.6). For instance, a study may examine the change in each individuals blood pressure (the within-individuals comparison), for two drugs given to different groups (the between-groups comparison).

Definition 2.9 (Repeated-measures RQ) Repeated-measures RQs have a population, outcome and a within-individuals comparison.

Repeated-measures RQs have one of these forms, depending on what information is sought:

Estimation RQ: Among {the population}, what is the change in {the outcome} for {the alternatives being compared}?
Decision-making RQ: Among {the population}, is {there a change in the outcome} for {the alternatives being compared}?

Example 2.8 (Within-individuals relational RQ) To understand tree-dwelling marsupials, a study compared the temperature in the same tree hollows in summer and winter (Rowland, Briscoe, and Handasyde 2017):

For tree hollows in the Strathbogie Ranges, Australia, what is the mean temperature difference between summer and winter?

The comparison is within individuals, as the temperature is measured for the same tree hollows at the two times. This is a repeated-measures, estimation-type RQ.

Repeated-measures RQs with only two comparisons are often called paired.

Example 2.9 (Paired repeated-measures study) D. A. Levitsky, Halbmaier, and Mrdjenovic (2004) compared the weights of the same university students at the beginning university, and then after $12$ weeks. The comparison is within individuals, and the study is a repeated measures study. Since each student has a pair of weight measurements, this is a paired study.

2.5 Variables: from populations to individuals

RQ are about populations. However, the data to answer a RQ come from individuals in that population. Each piece of information obtained from or about each individual is called a variable, because the values can vary from individual to individual.

Definition 2.10 (Variable) A variable is a single aspect or characteristic associated with the individuals, whose values can vary from individual to individual.

Examples of variables include: the duration of cold symptoms, sex, rate of tree growth, or hair colour.

A variable is a single aspect that can vary from individual to individual. While your city of birth does not change, 'city of birth' is a variable because it varies from individual to individual.

Example 2.10 (Variables) 'Duration of cold symptoms' is a variable: its value can vary from individual to individual. The 'average duration of cold symptoms' is the outcome, a numerical summary of many individuals' cold durations.

While many variables can be recorded from individuals in the population, two essential variables are (Table 2.1):

The response variable, which records information to determine the outcome.
The explanatory variable, which records information to determine the comparison.

The value of the response variable may change in response to the value of the explanatory variable. The value of the explanatory variable may explain changes in the value of the response variable.

Definition 2.11 (Explanatory variable) An explanatory variable may (partially) explain or be associated with changes in another variable of interest (the response variable).

Definition 2.12 (Response variable) A response variable records the result, output, consequence or effect of interest from changes in another variable (the explanatory variable).

The response variable is sometimes called the dependent variable, and the explanatory variable is sometimes called the independent variable. We avoid these terms, since the words 'dependent' and 'independent' have many different meanings in research.

TABLE 2.1: The relationship between the population and the individuals

Population		Individuals
Outcome:	$\rightarrow$	Response variable
Comparison:	$\rightarrow$	Explanatory variable

The RQ cannot be answered without values for the response and explanatory variables. The outcome is a summary of the values of the response variable (Table 2.2) recorded from many individuals. Similarly, the values of the explanatory variable measured on the individuals distinguish between the values of the comparison (Table 2.3 and 2.4) being made.

TABLE 2.2: Examples of the outcome and the corresponding response variable
	$\rightarrow$
Outcome describing the population	$\rightarrow$	Response variable in individuals
Average increase in diastolic blood pressure, from before to after exercise	$\rightarrow$	Increase in diastolic blood pressure of individuals, from before to after exercise
Percentage of seedlings that sprout	$\rightarrow$	Whether or not an individual seedling sprouts
Proportion owning iPad	$\rightarrow$	Whether or not an individual owns an iPad
Average cold duration	$\rightarrow$	Cold duration for individuals
Percentage of concrete cylinders having fissures	$\rightarrow$	Whether or not an individual cylinder has fissures

TABLE 2.3: Examples of the between-individuals comparison and the corresponding explanatory variable
Comparison being made		Explanatory variable in individuals
Between jarrah, beech, bamboo boards	$\rightarrow$	Type of floorboard in homes
Between $3$ kg/ha, $4$ kg/ha fertilizer rates	$\rightarrow$	Application rate in paddocks
Between people in $20$s, $30$s and $40$s	$\rightarrow$	Age group for each person

TABLE 2.4: Examples of the within-individuals comparison and the corresponding explanatory variable
Comparison being made		Explanatory variable in individuals
Before and after	$\rightarrow$	Time
Between left and right arms	$\rightarrow$	Which arm is used
Between fore legs and hind legs	$\rightarrow$	Which legs are measured

Free Online Poll Maker

The Population is 'carrots grown in Buderim' 8 weeks after planting. From these carrots, we need to collect Whether or not Thrive was applied and the weight of the carrots $8$ weeks after planting.

The response variable is 'the weight of each individual carrot $8$ weeks after planting', and the explanatory variable is 'whether or not Thrive was used on each carrot'.

('The number of carrots planted' is not even a variable: it is not information recorded about the individuals, but a summary of information.)

Example 2.11 (Variables) Consider a study of the ground surface temperature of public playgrounds in Boston in summer.

The population comprises all public playgrounds in Boston; each public playground is an individual. The outcome is the average ground surface temperature in summer over many playgrounds; the response variable is the ground surface temperature for individual ground surfaces in summer.

The between-individuals comparison is between the four types of ground surfaces (rubber, soil, sand, mulch). The explanatory variable is the type of surface.

2.6 Correlational RQs

A different type of RQ is a correlational RQ. Correlational RQs are not concerned with summarising outcomes in comparison groups. Instead, correlational RQs explore relationships between two variables that are measured or observed on the individuals.

Definition 2.13 (Correlational RQ) Correlational RQs explore the relationship between two variables.

Correlational RQs have one of these forms, depending on what information is sought:

Estimation RQ: Among {the population}, how strong is the relationship between {the outcome} and {something else}?
Decision making RQ: Among {the population}, is {the outcome} related to {something else}?

Examples include:

Studying the relationship between the height of plants and the number of hours of sunlight per day.
Studying the relationship between heart rate and the number of grams of caffeine consumed that day.

In some (but not all) situations, one variable can be considered as perhaps influencing the value of the other variable. This variable is called the explanatory variable (which may explain changes in the other variable). The other is the response variable (whose values respond to changes in the explanatory variable). To be able to influence the response variable, the explanatory variable must occur before (or at the same time) as the response variable.

Example 2.12 (Correlational RQs) Consider studying marathon runners. A RQ exploring the relationship between the individuals' water intake on the day before the race and the individuals' race times would be a correlational RQ. The water intake on the day before the race may influence the race time.

The water intake on the day before the race is the explanatory variable, and the race time is the response variable.

Example 2.13 (Within-individuals relational study) González-Acosta et al. (2024) studied the size of $39$ demersal fish species, recorded the length and weight of $14\ 040$ fish. There are two variables (fish length; fish weight), but it makes no sense to identify a response variable and explanatory variable.

Example 2.14 (Correlational RQs) The Wollemi pine was discovered by science in 1994. Offord and Zimmer (2023) studied the growth of these rare plants.

One correlational RQ they studied was the relationship between the diameter of trees at breast height (response variable), and the pH of the soil (explanatory variable). The two variables are the DBH and pH, both recorded for many trees.

They also studied the relationship between the DBH for each tree at at various times after the planting date. Each tree has the DBH measured over time, for many time points. Time is the within-individuals comparison.

2.7 Interventions

Sometimes, the explanatory variable naturally occurs (e.g., the height of people, or the age of trees) without manipulation by the researchers. Sometimes, however, the explanatory variable is manipulated by researchers (e.g., the dose of fertilizer applied); this is called an intervention.

Definition 2.14 (Intervention) An intervention is present when researchers can manipulate (or impose) the values of the explanatory variable on the individuals to determine the impact on the response variable.

Explanatory variables not manipulated by the researchers are called conditions. Explanatory variables that are manipulated by the researchers, or imposed on the individuals by the researchers, are called treatments. The analysis is the same whether an intervention is used or not, but the interpretation of the results depend on whether or not an intervention is used (Sect. 4.6).

An intervention is present when the researchers:

explicitly give a dose of a new drug to patients.
explicitly apply wear-testing loads to two different flooring materials.
explicitly expose people to different stimuli.
explicitly apply different doses of fertiliser.

Example 2.15 (Intervention) Bird et al. (2008) supplied one group of participants a diet using refined flour, and supplied another group of participants a diet using a new flour variety. The type of diet is the explanatory variable. Since the researchers manipulate which subjects ate which flour, this study has an intervention.

Example 2.16 (No intervention) To compare the average blood pressure in female and male Scots, blood pressure was measured using a blood pressure machine (a sphygmomanometer). The researchers interact with the participants to measure blood pressure, but there is no intervention. Using the sphygmomanometer is just a way to measure blood pressure, to obtain the data.

The comparison is between females and males, which cannot be manipulated or imposed on the individuals by the researchers, so there is no intervention.

Example 2.17 (POCI) Woolf et al. (2009) measured iron status in highly-active and sedentary American college women.

The outcome is the 'average iron status'. The between-individuals comparison is between highly active and sedentary women. To have an intervention, the researchers would need to tell each individual woman to be highly active or sedentary; this seems unlikely, so the study does not have an intervention.

Often, one of the comparison groups is the control group. The control group is a comparison group not receiving the treatment being studied, or not having the condition being studied, but as similar as possible to the other units of analysis in all other ways. The control group is like a benchmark for detecting changes in the outcome (Sect. 7.5). Sometimes the control group receives a placebo: a non-effective treatment that appears to be the real treatment.

Definition 2.15 (Control) A control is an individual without the treatment or condition of interest, but as similar as possible in every other way to other individuals.

Definition 2.16 (Placebo) A placebo is a treatment with no intended effect or active ingredient, but appears to be the real treatment.

Example 2.18 (Control group) To test the effectiveness of a new medication, patients report to a doctor to receive injections of the new drug. Some patients are assigned to the control group, and do not get the drug injection. The controls ideally would also report to a doctor and receive an injection (like those receiving the drug); the injection, however, would be ineffective (a placebo).

Together, the Population, Outcome, Comparison and Intervention form the POCI acronym (sometimes written as PICO) to aid remembering the elements of RQs. The POCI acronym is not helpful for correlational RQs.

2.8 Two purposes of RQs

As noted earlier, RQs can be written with one of two purposes. Estimation RQs ask how precisely a value in the population is estimated by using the sample; these are answered using confidence intervals. Answering estimation RQs is discussed in Chaps. 23 to 28, and Sect. 38.6.

Decisions-making RQs require a decision to be made about the population, and are answered using hypothesis testing. Answering decision-making RQs is discussed in Chaps. 30 to 35, plus Sects. 38.7 and 37.2.

Example 2.19 (Decision-making RQs) Thane, Bates, and Prentice (2004) studied 'British young people aged $4$--$18$' and asked numerous RQs. One relational RQ was:

In British young people aged $4$--$18$, is the average zinc intake the same for boys and girls?

This is an decision-making RQ.

Decision-making RQ have two possible answers. For the example above, the average zinc intake either is the same for boys and girls, or is not the same for boys and girls (Fig. 2.1). However, answers are rarely clear in practice, since only one of the countless possible samples from the population is studied. Instead, researchers decide how strongly the sample evidence support a particular hypothesis about the population.

Evidence may support or contradict a hypothesis; evidence rarely proves a hypothesis (at least, without any other support, such as theoretical support). Ultimately, after collecting data from a sample, a decision must be made about which explanation about the population is more consistent with the data collected.

FIGURE 2.1: Two possible answers to the RQ about zinc intake

Decision-making RQs can be asked in different ways. For the zinc-intake study above (Fig. 2.1), the RQ could ask if:

the average zinc intake is the same for boys and girls?
the average zinc intake is different for boys and girls?
the average zinc intake is lower for boys, compared to girls?
the average zinc intake is higher for boys, compared to girls?

The first two are two-tailed RQs (and are essentially asking the same thing but in different ways): the average zinc intake could be higher for girls or higher for boys. We are just interested in whether any difference is present. The last two are one-tailed RQ, since they ask specifically about a difference in just one direction: girls greater than boys, or boys greater than girls.

Most RQs are two-tailed, unless a good reason exists to ask a one-tailed RQ (e.g., a drug has been developed specifically to reduce blood pressure). One of the last two would only be adopted if there was a specific reason why a one-tailed direction was suspected before the data were collected. RQs should be formed before the data are collected.

In general, RQs should be written as two-tailed RQs, unless a good (and justifiable) reason exists for asking a one-tailed question before the data are collected.

2.9 Units of observation and analysis

Units of observation and units of analysis are different yet similar concepts that must be distinguished to properly identify a population.

Consider this descriptive RQ:

In English $20$-something men, what is the average thickness of head-hair strands?

To answer this question, the thickness of individual hair strands need to measured. The 'thing' from which measurements are taken are called units of observation.

Definition 2.17 (Unit of observation) The unit of observation is the 'who' or 'what' that are observed, from which measurements are taken and data collected.

Suppose we have $100$ strands of hair from which we can record the thickness. These hair strands could be obtained in different ways. For example,

we may have $100$ hair strands, all from the same single man; or
we may have one hair strand, one each from $100$ different men.

While each approach gives $100$ measurements of hair-strand thickness, these two approaches are very different. Only one man is represented in the first scenario, but $100$ different men are represented in the second. The difference is related to the concept of unit of analysis.

The purpose of the study is to make conclusions about 'men'. Since the RQ is asking about 'men', the 'man' is the unit of analysis.

The first scenario above has only one unit of analysis (which provided all $100$ units of observation); the second scenario has $100$ units of analysis (each providing one unit of observation).

Definition 2.18 (Unit of analysis) The unit of analysis is the smallest collection of units of observations (and perhaps the units of observations themselves) about which conclusions are made; the smallest independent elements of the population for which information is analysed.

All the hair strands from the same man have essentially 'lived their life together': they are washed together with the same shampoo, exposed to the same amount of sunlight and exercise, share the same genetics, etc. However, different men potentially use different shampoo, exercise differently, have different genetics, and so on. The hair on different men tends to be independent of the hairs on other men. Each man is a collection of units of observations (hair strands).

Example 2.20 (Units of analysis, observation) The Spectrum website (accessed 18 Nov 2022) reported a study where researchers examined '$10$ neurons from each of the $16$ mice' in the study (November 2022). The researchers treated each neuron as an independent observation, giving a sample size of $n = 16\times 10 = 160$.

However, neurons in the brain of the same animal are not independent observations. The unit of analysis is the mouse; the unit of observation is the neuron. The actual sample size was $n = 16$; each unit of analysis has $10$ units of observation.

A total of $160$ neurons from $16$ mice is very different to a study of $160$ neurons from $160$ genetically-different mice.

The units of observation and units of analysis may be the same, and often are the same. However, they are sometimes different, and identifying these situations is crucial. Importantly, studies compare units of analysis, not units of observation.

The sample size is the number of units of analysis.

Sometimes the units of analysis and units of observation are the same.

Example 2.21 (Units of analysis) A study compared two physical activity (PA) programs. Each of $44$ children in the study was allocated to one of two PA programs (with parental agreement). The children's fitness was measured for every student at the end of the six-month study.

The units of observation are the individual students, as the fitness measurements are taken from each student. The units of analysis are also the individual students, as the students using the different programs are being compared. In addition, the PA program was allocated to each student individually, and each student has their own sport, family routines and activities, etc. Each unit of analysis (student) has one unit of observation.

There are $44$ units of analysis, each with one unit of observation.

Example 2.22 (Units of analysis) Consider comparing the percentage of females and males wearing hats at a specific beach.

People in a group at the beach will probably not be operating independently: people in groups tend to behave similarly. For example, a couple will often both be wearing or both not wearing hats.

The researchers may decide not to use data from groups, and only gather data from individuals (when the 'individual' is the unit of analysis and unit of observation). Alternatively, the researchers may decide to use people groups as the unit of analysis (some will be groups of one), and record data from just one person in any group (ideally specifying before-hand from which group member to take data; e.g., the person closest to the researchers when the group is noticed).

Free Online Poll Maker

Units of observation: the individual students, as the fitness measurements are taken from the students individually.

Units of analysis: the schools, as the PA program was allocated to each school. All students at School A are exposed Program 1, but all students at School A are also likely to be exposed to similar weather, fitness opportunities, physical conditions, teachers and school-based philosophies, and so on.

The improvement in the children's fitness levels and the program are both variables.

The following short video may help explain some of these concepts:

2.10 Definitions

Research studies usually include terms that must be carefully and precisely defined, so that others know exactly what words and terms mean, without ambiguity. Two types of definitions can be given when necessary.

Definition 2.19 (Conceptual definition) A conceptual definition articulates precisely what words or phrases mean in a study.

Definition 2.20 (Operational definition) An operational definition articulates exactly how something will be identified, measured, observed or assessed.

In many cases, a clear operational definition is needed to describe how data will be collected to ensure repeatability and consistent data collection, by removing any ambiguity about how data are obtained.

Example 2.23 (Operational and conceptual definitions) Consider a study examining stress in students. A conceptual definition would describe what is meant by 'stress' (in contrast to, say, 'anxiety'). An operational definition would describe how 'stress' is measured, since stress cannot be measured directly (like height, for example).

'Stress' could be measured using a questionnaire or measuring physical characteristics, for instance. Other ways of measuring stress are also possible, and all have advantages and disadvantages.

Sometimes the definitions themselves are not important, provided a clear definition is given. However, to avoid confusion, commonly-accepted definitions should be used unless good reasons exist for using a different definition. When a commonly-accepted definition does not exist, the definition being used should be very clearly articulated, and the reason given if necessary.

Example 2.24 (Operational and conceptual definitions) A research article (Gillet et al. 2018) entitled 'Shoulder range of motion and strength in young competitive tennis players with and without history of shoulder problems' provided these necessary conceptual definitions (among others):

Young: $8$--$15$ years;
Competitive tennis players: Some of the best players in their age category in France, and members of a French tennis centre of excellence.

An operational definition was provided for 'Shoulder strength': as measured using a hand-held dynamometer.

Players, administrators and fans are wary of concussions and head injuries in sport. A conference on concussion in sport developed this conceptual definition (McCrory et al. 2013):

... a complex pathophysiological process affecting the brain, induced by biomechanical forces...

However, an operational definition is needed to explain how to identify a player with concussion during a game. Rugby decided on this operational definition (Raftery et al. 2016):

... a concussion applies with any of the following:

The presence, pitch side, of any Criteria Set 1 signs or symptoms (table 1)... [this table includes symptoms such as 'convulsion', 'clearly dazed', etc.];

An abnormal post game, same day assessment...;

An abnormal $36$--$48$ h assessment...;

The presence of clinical suspicion by the treating doctor at any time...

Example 2.25 (Operational and conceptual definitions) Consider a study requiring water temperature to be measured.

An operational definition would explain how the temperature is measured: the thermometer type, how the thermometer was positioned, how long was it left in the water, and so on.

A conceptual definition would describe the scientific definition of temperature (and would not be needed, as 'temperature' is a well-understood term).

A study of snacking in Australia (Fayet-Moore et al. 2017) used this conceptual definition of an 'eating occasion':

...one or more food or beverage items consumed at the same time of day...

and a 'snacking occasion' as

...one or more food or beverage items consumed at the same time of day within a snacking time period...

Finally then, 'snacking' was defined as:

Eating occasions that occurred during breakfast, midday and evening meals were meals and all eating occasions that occurred between these meals were classified as snacking.

These are all conceptual definitions, explaining what the terms mean.

An operational definition would explain how the data were obtained from the participants (e.g., using a food diary).

Meline (2006) discusses five studies about stuttering, each using a different operational definition:

Study 1: As diagnosed by speech-language pathologist.
Study 2: Within-word disfluences greater than $5$ per $150$ words.
Study 3: Unnatural hesitation, interjections, restarted or incomplete phrases, etc.
Study 4: More than $3$ stuttered words per minute.
Study 5: State guidelines for fluency disorders.

People may be classified as stutterers by some definitions but not others, so it is important to know which definition is used.

A study examined the possible relationship between the 'pace of life' and the incidence of heart disease (Levine 1990) in $36$ US cities.

The researchers used four different operational definitions for 'pace of life' (remember the article was published in 1990!):

The walking speed of randomly chosen pedestrians.
The speed with which bank clerks gave 'change for two $20 bills or [gave] two $20 bills for change'.
The talking speed of postal clerks.
The proportion of men and women wearing a wristwatch.

None of these perfectly measure 'pace of life', of course. Nonetheless, the researchers found that, compared to people on the West Coast,

... people in the Northeast walk faster, make change faster, talk faster and are more likely to wear a watch...

--- Levine (1990) (p. 455)

2.11 Writing RQs: an example

Suppose you notice some people taking echinacea (a herb) when they get a common cold. You may wonder: does taking echinacea help with a cold? This may lead to an initial RQ:

Is it better to take echinacea when you have a cold?

This RQ is clearly poor, but is a starting point. This RQ can be refined by clarifying the POCI elements. For example, what population could we study? Many options exist: all residents of your country, or just adults in a specific part of your country. Some of these may not be practical (i.e., when a sample cannot easily be obtained from the population).

What outcome could be used to determine echinacea's effectiveness? Options include the average cold duration, or the percentage of people who take days off work.

The initial RQ is also vague: better than what? The outcome could be compared between groups (between those taking echinacea and those who do not). A within-individuals comparison seems unsuitable for this RQ.

The study could also have intervention or not, which has implications for how the study is conducted and how the results are interpreted. If the study did not have an intervention, the subjects would decide for themselves how to treat their cold. If the study did have an intervention, the use of echinacea would be imposed by the reserchers.

Many terms need defining, too: What is meant by 'echinacea' (fresh? tablet form? as a tea?); 'cold' (self-diagnosed? diagnosed by a doctor?), and so on.

Based on the above, this RQ could be considered (based on Barrett et al. (2010)):

Among Australian teenagers with a common cold, is the average duration of cold symptoms shorter for teens given a daily dose of echinacea, compared to teens taking no echinacea?

The following short video may help explain some of these concepts:

2.12 Preparing software

Most statistical software packages use the same approach for organising the data (though exceptions exist for some types of analyses):

Each row represents one unit of analysis: the number of rows equals the number of units of analysis (i.e., the sample size).
Each column represents one variable: the number of columns equals the number of variables. (An additional column of identifying information may also appear, such as the person's name, or concrete batch number.)

In statistical software, the variable names are not placed in a row (say, in Row 1, above the data itself), which might happen when using a spreadsheet. The names of the variables are the names of the columns.

Example 2.26 (Preparing statistical software) In Sect. 2.5, a RQ was asked about whether using echinacea or not reduced the duration of the common cold.

For this RQ, the variables are 'Duration of cold symptoms' (response), and 'Type of treatment' (explanatory). The number of rows in the data worksheet will equal the number of people in the study, since the person is the unit of analysis. The data worksheet needs at least two columns (Fig. 2.2, left panel):

one for duration of each individual's cold symptoms (say, Duration);
one for whether the individual received a dose of echinacea or received no medication (say, Treatment).

There may be an additional column recording the name or ID of each individual, and more columns recording other variables (such as age and height).

Example 2.27 (Preparing statistical software) Example 2.9 discussed a study (D. A. Levitsky, Halbmaier, and Mrdjenovic 2004) where the weights of university students were recorded both at the beginning university, and then after $12$ weeks. The number of rows in the data worksheet will equal the number of students in the study, since the student is the unit of analysis. The data worksheet needs at least two columns (Fig. 2.2, right panel):

one for the student's weight at the start of university (say, Week1);
one for the student's weight after $12$ weeks at university (say, Week12).

jamovi prepared for the data, with some data entered, and the variable names as the column headers. Left: a between-individuals comparison. Right: a within-individuals comparison.

FIGURE 2.2: jamovi prepared for the data, with some data entered, and the variable names as the column headers. Left: a between-individuals comparison. Right: a within-individuals comparison.

2.13 Chapter summary

In this chapter, you have learnt to write research questions for quantitative analysis. All research questions (RQs) study some population (P). Descriptive RQs study some outcome (O) in the population. Relational RQs compare the outcome between different groups of individuals (a between-individuals comparison). Repeated-measures RQs compare the same outcome when measured on the same individuals multiple times (a within-individuals comparison). Some RQs also have an intervention (I): when the values of the comparison can be manipulated by the researchers. Correlational RQs ask about the relationship between variables in the population.

RQs may take one of two forms: Decision-making RQs (which may be one- or two-tailed) or Estimation RQs.

Data comes from individuals in the population by measuring, observing or assessing the response (or dependent) variable. The outcome is a numerical summary of the values of the response variable from many individuals. Similarly, the data concerning the comparison comes from measuring or observing the values of the explanatory (or independent) variables from individuals.

The who or what that observations are made from are called the units of observation. The smallest independent collections of units of observations (that is, independent examples of the population) are called the units of analysis.

The following short video may help explain some of these concepts:

2.15 Exercises

Answers to odd-numbered exercises are available in App. E.

Exercise 2.1 For the following response variables, what are the corresponding outcomes?

Whether a vehicle crashes or not.
The height people can jump.
The number of tomatoes per plant.

Exercise 2.2 For the following response variables, what are the corresponding outcomes?

Whether or not a person owns a car.
The time it takes for seedlings to sprout.
The amount of caffeine in cola drinks.

Exercise 2.3 For the following comparisons, what are the corresponding explanatory variables?

Between vegans and vegetarians.
Between caffeinated and decaffeinated coffee.
Between taking zero, one or two $7$ mg iron tablet per day.

Exercise 2.4 For the following comparisons, what are the corresponding explanatory variables?

Between frozen vegetables and fresh vegetables.
Between $91$-octane, $95$-octane, and ethanol-blended car fuel.
Between large cities and small cities.

Exercise 2.5 For the following studies, determine which have a between-individuals comparison and which have a within-individuals comparison. In each case, identify the outcome.

A study to determine if a higher percentage of people at a particular city park wear hats in winter compared to summer.
A study to determine if the average yield of a specific variety of tomato plants is the same when three different fertilisers are applied.

Exercise 2.6 For the following studies, determine which have a between-individuals comparison and which have a within-individuals comparison. In each case, identify the outcome.

A study to determine if a person's average balance-time on their right leg is the same as on their left leg.
A study to determine if average cholesterol levels are the same when measured on the same people before and after a diet change.

Exercise 2.7 A study of Phu Quoc Ridgeback dogs (Canis familiaris) explored the relationship between body length and body height (Quan, Tran, and Chung 2017).

What type of RQ would be asked about the dogs?
What are the response and explanatory variables?

Exercise 2.8 Pinet et al. (2022) recorded typing speed and age for $1301$ students.

What type of RQ could be asked in this study?
What are the response and explanatory variables?

Exercise 2.9 Consider this RQ:

Among Danish university students, is the average resting diastolic blood pressure the same for students who regularly drive to university and those who regularly ride their bicycles to university?

For this RQ, identify the population, outcome, and comparison (if any).
For this RQ, is there an intervention? Explain.
What type of question is used: descriptive; relational; repeated measures; correlational?
What is the purpose of the RQ: estimation or decision-making?
What operational and conceptual definitions would be needed?
What information must be collected from each individual to answer the RQ (i.e., the variables)?
Identify the units of analysis and the units of observation.

Exercise 2.10 Consider this article extract (Checkley et al. (2002), p. 210):

We conducted a $4$-year (1995--1998) field study in a Peruvian peri-urban community... to examine the relation between diarrhea and nutritional status in $230$ children $< 3$ years of age

For this study:

Identify POCI.
Infer the primary research question.
What type of question is used: descriptive; relational; repeated measures; correlational?
What is the purpose of the RQ: estimation or decision-making?
What operational definitions would be needed?
What are the response and explanatory variables?

Exercise 2.11 Consider this RQ: 'Is the average walking speed the same when texting and talking on a mobile phone?'

What type of question is used (descriptive; relational; repeated measures; correlational)?
Is this RQ one- or two-tailed?
Is there an intervention?
What is the explanatory variable?
What is the response variable?
What is the outcome?

Exercise 2.12 Animals in an experiment are divided into pens (three animals per pen), and feed is allocated to each pen (Sterndale et al. 2017). Animals in different pens receive different feed; animals in the same pen receive the same feed. The weight gain of each animal is recorded.

What is the unit of observation? Why?
What is the unit of analysis? Why?
Identify the between-individuals comparison.

Exercise 2.13 Consider this actual student RQ from the university where I work.

Among $10$ Australian adults, does the time taken to read a passage of text change when different fonts are used?

Critique the RQ, and write a better RQ (if necessary).

Exercise 2.14 Consider this actual student RQ from the university where I work.

Of students that study at (a University), do males have a larger lung capacity than females?

Critique the RQ, and write a better RQ (if necessary).

Exercise 2.15 Consider this RQ, with an intervention:

For Japanese adults with a common cold, do people who take Vitamin C tablets daily have, on average, a shorter cold duration than people who do not take any Vitamin C tablets?

What is the population?
What is the comparison?
What is the outcome?
What is the response variable?
What is the explanatory variable?
What type of RQ is this: estimation or decision-making?
Is the RQ one-tailed or two-tailed?

Exercise 2.16 A research study was comparing the average size of Blue Gum eucalypt leaves in two areas of Queensland. A student takes $40$ leaves from each of ten trees in Area A, and $40$ leaves from each of ten trees in Area B.

Are the following statements true or false?

The unit of analysis is the individual leaf.
The unit of observation is the individual leaf.
The unit of analysis is the tree.

What is the size of the sample in the study?

8020800

Exercise 2.17 Prinz and Murray (2023) examined the strength needed to pull out nose-hairs. Fifty nose-hairs were pulled from one author's nose, and $50$ nose hairs pulled from the other author's nose, and the average pull-out strengths for each man compared.

What are the units of analysis and units of observation?
What is the sample size in this study?

Exercise 2.18 Huang et al. (2020) placed different people into one of three different virtual-reality (VR) environments: trees, grass or concrete. Stress levels were measured using 'skin conductance level' (SCL) for each individual, before and after exposure to the VR environment.

Identify the between-individuals comparisons.
Identify the within-individuals comparisons.
Is the definition given for SCL (p. 2) conceptual or operational?

SCLs are an unbiased measure of sympathetic activity via the electric impulses on the skin’s surface and sweat glands, which are innervated only by the sympathetic nervous system...

Exercise 2.19 Consider this two-tailed RQ (based on Tudor-Locke, Barreira, and Schuna Jr (2015)):

For American adults, is the average number of recorded steps per day the same when recorded using both a waist accelerometer, and a wrist accelerometer?

Identify the population and the individuals.
Identify the outcome.
Identify the response and explanatory variables.
Determine if the comparison is between- or within-individuals.

Exercise 2.20 Studies can incorporate many types of RQs. For example, Thane, Bates, and Prentice (2004) studied 'British young people aged $4$--$18$' and answered numerous RQs.

What is the average zinc intake of the children?
Does the average zinc intake meet recommended dietary guidelines?
What is the strength of the association between plasma zinc and retinol concentrations?
Is the average zinc intake the same for boys and girls?

For each RQ, classify these RQs as descriptive, relational, repeated-measures, or correlational RQs. Then, classify them as estimation or decision-making RQs. Does the study have an invention?

Exercise 2.21 Stern et al. (2021) studied the relationship between daily sodium excretion and whether people had been diagnosed with diabetes or not, in Israeli adults. The study also explored the relationship between the daily sodium excretion and the systolic blood pressure.

Classify the two RQs as descriptive, relational, repeated-measures, or correlational RQs. Then, classify them as estimation or decision-making RQs. Does the study have an invention?

Exercise 2.22 Ghasemi and Pirzadeh (2019) studied the incidence of musculoskeletal disorders in Iranian bus drivers. They introduced a program that aimed to provide relief for the drivers. Each bus driver was evaluated both before and after the intervention.

Classify the RQ as descriptive, relational, repeated-measures, or correlational RQs. Then, classify the RQ as estimation or decision-making RQs. Does the study have an invention?

Exercise 2.23 A study compares the wear on two brands of car tyres. Four tyres of Brand A are allocated to each of Cars 1--5, and four tyres of Brand B are allocated to each of Cars 6--10. After $12$ months, the amount of wear is recorded on each tyre, and the two brands compared.

What are the units of analysis, and the units of observation?

Exercise 2.24 Parsons, Teare, and Sitch (2018) discuss a scenario where six subjects with colorectal cancer underwent therapy. Another six, similar subjects did not receive the therapy. The size of the subjects' lymph nodes (removed through surgery) were then measured. Each subjects' specimen (p. 6)

... was divided into two sub-samples after collection [..] processed and analysed at two occasions, by different members of the laboratory team [...] Three slices per sub-sample were collected for each subject.

How many units of analysis and the units of observation are present?

Exercise 2.25 Bamboo is a fast-growing, strong grass often used for green building practices. A small research study explored the hardness of bamboo when used as flooring material.

The Janka hardness¹ of bamboo flooring provided by Bamboo Flooring Australia Pty Ltd was measured by the Queensland Department of Primary Industries (Gerber 2004). Five floorboards were taken, and two hardness measurements were taken on each board (units not given, but probably kilonewtons; Table 2.5).

What is the unit of analysis: The test, the board, each measurement, kilonewtons, or something else? Explain your answer.
How many units of analysis are there?
How many units of observation are there?
Suppose the measurements were all taken from $10$ different places on the same board (rather than from five different boards). How many units of analysis are there now? Explain your answer.
Comment on the amount of variation between the boards compared to the amount of variation within boards.

TABLE 2.5: Two Janka hardness measurements from five different bamboo boards
Board 1	Board 2	Board 3	Board 4	Board 5
10.5	8	11.5	10.3	10.2
7.5	8	11.2	9.9	9.3

Exercise 2.26 Critique the following research questions, outlining how and why they can be improved (if at all).

Among domestic watertanks used in southeast Queensland, are lead concentrations in water in concrete tanks higher than in poly tanks?
Are lower-limb amputees more likely to die?
Is the amount of salt the same for homebrand as for non-homebrand beans?
Among zoo animals, is the weight of adult elephants greater than that of juvenile kangaroos (joeys)?
Is the average reaction time related to gender?

What terms might need defining for each RQ?

1 Research: an introduction

3 Overview of research design

Scientific Research and Methodology: An introduction to quantitative research and statistics

2 Research questions

2.1 Introduction

2.2 Descriptive RQs

2.2.1 The population

2.2.2 The outcome

2.3 Relational RQs

2.4 Repeated-measures RQs

2.5 Variables: from populations to individuals

2.6 Correlational RQs

2.7 Interventions

2.8 Two purposes of RQs

2.9 Units of observation and analysis

2.10 Definitions

2.11 Writing RQs: an example

2.12 Preparing software

2.13 Chapter summary

2.14 Quick review questions

2.15 Exercises

	\(\rightarrow\)
Outcome describing the population	\(\rightarrow\)	Response variable in individuals
Average increase in diastolic blood pressure, from before to after exercise	\(\rightarrow\)	Increase in diastolic blood pressure of individuals, from before to after exercise
Percentage of seedlings that sprout	\(\rightarrow\)	Whether or not an individual seedling sprouts
Proportion owning iPad	\(\rightarrow\)	Whether or not an individual owns an iPad
Average cold duration	\(\rightarrow\)	Cold duration for individuals
Percentage of concrete cylinders having fissures	\(\rightarrow\)	Whether or not an individual cylinder has fissures