Data Analysis

Data analysis gives you the language and tools for all the statistics in Year 12. It was worth 4 marks in the sample HSC paper, mostly multiple choice and short calculator questions.

• Statistical investigation: pose a question, design a fair survey, and consider privacy, bias, ethics and cultural sensitivity.
• Sampling: systematic, self-selected, random and stratified samples, and whether a sample is representative.
• Classifying data: numerical (discrete or continuous) or categorical (nominal or ordinal).
• Displaying data: frequency tables, histograms and polygons, column graphs, dot plots, stem-and-leaf plots, sector graphs, divided bar charts, line graphs, and misleading graphs.
• Shape: symmetric, positively or negatively skewed; unimodal, bimodal or multimodal.
• Centre and spread: mean, median, mode, range and population standard deviation, with a calculator.
• Quartiles and IQR, five-number summaries and box plots, including parallel box plots.
• Clusters, gaps and outliers, and the effect of outliers.

A statistical investigation starts with a clear question for a target population (the whole group you want to know about). Usually you collect data from a sample, a smaller group, because surveying everyone costs too much.

Good survey questions use clear, simple language, ask one thing at a time, avoid leading words ("Don't you agree that..."), and give a sensible number of answer choices that cover all possibilities. Surveys must also respect privacy, avoid bias, follow ethical practice (consent, no harm) and be respectful of cultural differences.

Self-selected samples are usually biased (not representative), because people with strong opinions are more likely to respond. Other faults to look for: a sample too small, questions that lead the respondent, or a sample taken from only one group (a survey about public transport given only at a train station).

• Categorical data are labels. Nominal has no order (favourite sport, eye colour); ordinal has an order (shirt size S, M, L; satisfaction ratings).
• Numerical data are numbers. Discrete data are counted (number of siblings); continuous data are measured and can take any value in a range (height, time).

Some data could be either. Age in whole years is often treated as discrete, although age itself is continuous; a postcode is a number but categorical, since doing arithmetic with it makes no sense.

Choosing a graph: categorical data suit column graphs, sector (pie) graphs and divided bar charts. Numerical data suit dot plots, stem-and-leaf plots, histograms and box plots. Data over time suit a line graph. Large or continuous numerical data are grouped into classes so the pattern can be seen.

Misleading graphs often have a vertical axis that doesn't start at zero, uneven scales, 3D effects that distort size, or missing labels.

Shape of a distribution

• Symmetric: both sides roughly mirror each other.
• Positively skewed: the tail stretches to the right (higher values), like incomes. The mean is usually greater than the median.
• Negatively skewed: the tail stretches to the left (lower values), like scores on an easy test.
• Unimodal has one peak, bimodal two, multimodal more; uniform is roughly flat.

• Mean $\bar{x}$: the sum divided by the number of values. Uses every value, so it is pulled by outliers.
• Median: the middle value of the ordered data. For $n$ values, it is the $\frac{n+1}{2}$th value. Not affected much by outliers, so it is better for skewed data such as house prices.
• Mode: the most common value. The only measure for categorical data.
• Range: highest value minus lowest value.
• Population standard deviation $\sigma$: a measure of how spread out the values are around the mean. A larger $\sigma$ means more spread. Calculate it with your calculator's statistics mode, choosing $\sigma x$ (not $sx$).

Worked example: summary statistics with a calculator

Ten students recorded their travel time to school, in minutes:

14, 17, 17, 20, 22, 23, 25, 26, 30, 46

Find the mean, median, mode, range and population standard deviation.

Enter the data in statistics mode, then read off the values.

With 10 values, the median is the 5.5th value, halfway between the 5th and 6th. If you get 9.09 for the standard deviation, you used $sx$ (the sample standard deviation) instead of $\sigma x$.

Grouped data

For grouped data, use the class centre (midpoint of each class) as the value. For classes 0–10, 10–20, 20–30 and 30–40 with frequencies 4, 9, 5 and 2, the centres are 5, 15, 25 and 35:

This is an estimate, because we don't know the actual values within each class.

The quartiles split ordered data into four equal parts. $Q_2$ is the median. $Q_1$ is the median of the lower half and $Q_3$ is the median of the upper half; if there is an odd number of values, leave the median out of both halves. The interquartile range is

It measures the spread of the middle 50% of the data, and unlike the range it is not affected by extreme values. From a dot plot or a histogram of ungrouped values, list the values in order using the dots or the frequencies (a column of height 3 at 20 means 20, 20, 20), then find the quartiles the same way.

The five-number summary is the minimum, $Q_1$, median, $Q_3$ and maximum. A box plot draws it: a box from $Q_1$ to $Q_3$ with a line at the median, and whiskers out to the minimum and maximum. Parallel box plots on the same scale let you compare two groups by centre (medians), spread (IQRs and ranges) and shape.

An outlier is a value well away from the rest. Formally, a value is an outlier if it is below the lower fence or above the upper fence:

Clusters are groups of values close together and gaps are intervals with no values. When you identify any of these, suggest a reason in context: an outlier may be a recording error or a genuine unusual value.

Worked example: is 46 an outlier?

For the travel times above, find the IQR and decide whether 46 minutes is an outlier. Then describe its effect on the mean and median.

The lower half is 14, 17, 17, 20, 22 and the upper half is 23, 25, 26, 30, 46. Calculators may use a different quartile method; show the halves.

Since $46>39.5$, the value 46 is an outlier. Removing it, the mean falls from 24 to 21.6 minutes (1 d.p.), but the median only moves from 22.5 to 22. The standard deviation drops from 8.63 to 4.79. The outlier has a big effect on the mean and standard deviation and very little on the median or IQR, which is why the median and IQR are preferred for skewed data or data with outliers.

• Forgetting to order the data before finding the median or quartiles.
• Using $sx$ instead of $\sigma x$ for the standard deviation.
• Calling skewness by the side with the bulk of the data. Skew is named after the tail.
• Calling a numerical code, like a postcode, numerical data.
• Saying an outlier "should be removed" without a reason. Only remove it if it is an error.
• Comparing box plots by one feature only. Compare centre, spread and shape.

• Compare two data sets means a sentence for centre (with values) and a sentence for spread (with values), for example "Group A has a higher median (24 vs 19) and a larger IQR (12 vs 7)".
• Justify an outlier with the fence calculation, not "it looks far away".
• In sampling questions, name the method and give a reason it is or isn't representative, linked to the context.
• Clear the calculator's statistics memory before entering a new data set, and check the number of values $n$ it shows.

• Bivariate data analysis (Year 12): two numerical variables, scatterplots and lines of best fit.
• The normal distribution (Year 12): z-scores use the mean and standard deviation from this topic.
• Relative frequency and probability (Year 12): relative frequencies come from frequency tables.
• Earning money: median incomes are a classic use of the median over the mean.