To Notes

IT 223 -- Jan 5, 2026

IT 223 Course Website

• See these links on the left side of the Announcements Page:
  
  • Announcements, Prof Info, Lecture Notes, Projects, Documents, Datasets, Exercises, Exam Info, COL Website (submit projects, view grades), Course Syllabus.

Early History of Statistics

• Some early pioneers in statistics:
  

  1654	Blaise Pascal is the first to develop the theory of probability. He is known for Pascal's Triangle, and was the first known person to apply the laws of probability to gambling.
  1662	John Graunt writes Natural and Political Observations upon the Bills of Mortality. He was the first demographer to use statistics to estimate the population of London.
  1718	Abraham de Moivre writes Doctrine of Chances. He is the first to write down the formula for the normal histogram and the first to state the Central Limit Theorem, which states that if a random variable is a sum of many independent influences, then that random variable is approximately normally distributed.
  1722	Roger Cotes published Opera Miscellanea. It is a study of the theory of errors, particularly in astronomy.
  1809	Carl Friedrich Gauss discovers the Method of Least Squares, the standard method for fitting a regression line to data.
  1835	Adolphe Quetelet publishes a study of human measurements that describes "the average man" that nature is trying to produce. He shows that human measurements tend to follow normal histograms. He is the first to introduce the body mass index that is still used today.
  1888	Sir Francis Galton first introduced the concept of correlation, which describes the degree of linear association between two measurements, such as height and weight.
  1890	R. A. Fisher is born. Many call him the father of modern statistics. He popularized the use of the standard deviation to estimate the spread of a dataset, as well as many other statistical techniques.
  1977	John Tukey published the book Exploratory Data Analysis. It popularized many of the techniques we will study in this class. He also coined the terms "bit" and "software." He died in 2000.

A Medical Application of Statistics

• Look at the document Experimental Trials for the Salk Polio Vaccine. What are the meanings of these terms?
       Lurking Variable   Treatment Group   Control Group
       Randomized Study   Double-blind Study  Controlled Experiment
       Observational Study
  
  Answers:
  Lurking Variable: a variable not included in the dataset that may nevertheless affect the results of the study.
  Treatment Group: the group of subjects that receives the new medication being tested.
  Control Group: the group of subjects that receives the placebo or the existing medication not being tested.
  Randomized Study: a study that assigns subjects randomly to the treatment and control groups.
  Double-blind Study: a study where neither the subject or the doctor knows which group, treatment or control, where the subjects are assigned.
  Controlled Experiment: an experiment where the patients are randomly assigned to the treatment or control groups. Although it is best for a controlled experiment to be double blind, this is not always possible.

Exploratory Data Analysis

• The term Exploratory Data Analysis or EDA was coined by John Tukey.
• Although EDA is usually associated with techniques like stemplots and boxplots, it is more accurately defined as a philosophy: understand the dataset before deciding which statistical tests to perform.
• Some terminology:
  
  • A univariate dataset is a dataset containing only one variable. For example these hypothetical exam scores form a univariate dataset:
    
    98  93  85  39  91  75  5  93  79  90 
  • A bivariate dataset is a dataset containing two variables. Some examples of bivariate datasets are weight vs. height, price vs. date, and score on exam vs. hours studied for it.
  • A multivariate dataset is a dataset containing two or more variables. We will only consider univariate and bivariate datasets.
  • A categorical variable (also called a nominal variable) is a variable that contains non-numeric values, such as gender (female, male, non-binary), is-smoker (yes, no), football player position (offense, defense).
  • A continuous variable (also called a scale variable or quantitative variable) is a variable that can contain any value in a continuous range. Examples are: age, height, weight, gpa, price.
  • An ordinal variable is a variable that contains discrete numeric values. Examples are: college-year (1, 2, 3, 4), survey-answer (1, 2, 3, 4, 5). Coin condition for collectable coins is also an ordinal variable because the conditions are rank ordered highest to lowest: Poor, Fine, Very-fine, Extremely-fine, Uncirculated, Brilliant-uncirculated. Coin collectors also use a numeric grade for rating coins from 1 to 70.
  • To summarize the various value of a univariate dataset, tally marks are traditionally used. Here are the sorted hypothetical exam scores:
    
    5  39  75  79  85  90  91  93  93  98
  • Here is the tally diagram:
    
  • A more informative technique is the stemplot, also called the stem-and-leaf display.
    

    95 | 8
    90 | 0133
    85 | 5
    80 | 
    75 | 59
    70 |  
    65 |
    60 |
    55 |
    50 |
    45 |
    40 |
    35 | 9
    30 |
    25 |
    20 |
    15 |
    10 |
     5 | 5
     0 |
    

    The last digit of each observation is recorded to the right of the vertical line down the middle, which is the "stem."

The R Programming Environment

• R is a free, open-source programming environment for statistical analysis and graphics.
  Use the google search string "Why use R?" to see explanations and videos that answer this question.
• Some answers:
  
  1. R is free to use, unlike other statistical analysis systems like SPSS, Statlab, or SAS.
  2. R uses command line scripts similar to Python scripts. Here are two R one line "Hello, World" R scripts:
     > print("Hello, World")
     [1] "Hello, World."
     > cat("Hello, World")
     Hello, World>
  3. R is a vector-oriented language, which means that the inputs and outputs are often vectors. (A vector of length n is an ordered set of n numbers. For example, the unsorted hypothetical exam scores form a vector of length 10:
     

     scores <- c(98, 93, 85, 39, 91, 75, 5, 93, 79, 90)
     print(scores)
     [1] 98 93 85 39 91 75  5 93 79
     

     The initial [1] means that this line starts with the element of index 1. Note that R indices are 1-based, not 0-based like Python, Java, or C#.
  4. An advantage of using R scripts is that the calculations are reproducible; they can be run by others to validate the results on possibly different datasets.
  5. Another advantage of using R scripts is that data analysts can write packages that can be used by the R community. As of November 2025, there are more than over 23,000 packages available in the Comprehensive R Archive Network (CRAN).
• Use the search string "download R for Windows" or "download R for Mac" to find the website for downloading R.
• Use R to compute the mean and median of the hypothetical exam scores:
  

  98  93  85  39  91  75   5  93  79  90
  

  Answer:
  

  > scores <- c(98, 93, 85, 39, 91, 75, 5, 93, 79, 90)
  > mean(scores)
  [1] 74.8
  > median(scores)
  [1] 87.5
  

• Use R to complete Project 1.

Quartiles

• The Quartiles divide the sorted dataset into fourths.
• Q0 is the minimum point in the dataset. 0/4 (none) of the data are less than than Q0.
• Q4 is the maximum point in the dataset. 4/4 (all) of the data are less than than Q4.
• Q2 is the median (middle point) of the dataset. 2/4 = 1/2 of the data are less than than Q2. If the dataset has an even number of observations, the Q2 is the average of the middle two observations.
• Q1 is the median of the bottom half of the data. (If the dataset has an odd number of observations, the bottom half of the data contains the middle observation). 1/4 of the data are below Q1.
• Q3 is the median of the top half of the data. (If the dataset has an odd number of observations, the top half of the data contains the middle observation). 3/4 of the data are below Q3.
• Q0, Q1, Q2, Q3, and Q4 and also called the 0th, 25th, 50th, 75th, and 100th percentiles, respectively.
• Exercise: Find the quartiles of the hypothetical exam scores:
  
  • 5  39  75  79  85  90  91  93  93  98
  Answer: use the R function summary with the hypothetical exam scores as input.
• The interquartile range (IQR), is defined as IQR = Q3 - Q1.
• Here are some alternative ways to compute percentiles.
• There are at least different eight methods for computing precentiles.
• Boxplots

Histograms

Project 2

• Look at Project 2a.
• Start thinking about what data you want to collect for Project 2.