To Notes

IT 403 -- Sept 7, 2016

Go over IT 223 course web site

• See links in the left frame:
  
  • Announcements, Prof Info, Lecture Notes, Projects, Documents, Datasets, Exercises, Exam Info, COL Website (submit projects, view grades), Course Syllabus.

Early History of Statistics

• Some early pioneers in statistics:
  

  1654	Blaise Pascal is the first to develop the theory of probability. He is known for Pascal's Triangle, and was the first known person to apply the laws of probability to gambling.
  1662	John Graunt writes Natural and Political Observations upon the Bills of Mortality. He was the first demographer to use statistics to estimate the population of London.
  1718	Abraham de Moivre writes Doctrine of Chances. He is the first to write down the formula for the normal histogram and the first to state the Central Limit Theorem, which states that if a random variable is a sum of many independent influences, then that random variable is approximately normally distributed.
  1722	Roger Cotes published Opera Miscellanea. It is a study of the theory of errors, particularly in astronomy.
  1809	Carl Friedrich Gauss discovers the Method of Least Squares, the standard method for fitting a regression line to data.
  1835	Adolphe Quetelet publishes a study of human measurements that describes "the average man" that nature is trying to produce. He shows that human measurements tend to follow normal histograms. He is the first to introduce the body mass index that is still used today.
  1888	Sir Francis Galton first introduced the concept of correlation, which describes the degree of linear association between two measurements, such as height and weight.
  1890	R. A. Fisher is born. Many call him the father of modern statistics. He popularized the use of the standard deviation to estimate the spread of a dataset, as well as many other statistical techniques.
  1977	John Tukey published the book Exploratory Data Analysis. It popularized many of the techniques we will study in this class. He also coined the terms "bit" and "software." He died in 2000.

Applications of Statistics in the 20th Century

• Reference: Freedman, Pisani, and Purves, Statistics, Fourth Edition, W. W. Norton and Company, 2007.
  
  • Experimental Trials for the Salk Polio Vaccine
  • Forecasting the 1936 Presidential Election
  • Statistical Experiments vs. Observational Studies

  What do Statisticians Do?

  • Reference: The Statistics in Practice articles on the CD that comes with your textbook:
    
    • Jennifer Karas, Marketing Research: A Voice for Consumers
    • Cynthia Perez, Tracking Disease in Puerto Rico
    • Tommy Wright, Sampling the Nation
    • Jaime Sam Mart'in, Research in Probability Theory
    • Jing Shyr, Statistical Software for Solving Business Problems
    • Marcey Abate, Testing Nuclear Weapons without Explosions
    • Iogn' Aido' Murcheartaigh, From p-values to President
    • Aarti Ssiram, Management Consulting
    • Rebecca Doerge, Finding Genes in Plants and People
    • Gregory Campbell, Making Sure Medical Devices are Safe and Effective
    • Dexter Wilson, Improving the Quality of a Bank's Services
    • Christopher Presley, Statistics for Science, and More
    • Steve Sun, Developing New Drugs

  Exploratory Data Analysis

  • The term Exploratory Data Analysis or EDA was coined by John Tukey.
  • Although EDA is usually associated with techniques like stemplots and boxplots, it is more accurately defined as a philosophy: understand the dataset before deciding which statistical tests to perform.
  • Some terminology:
    
    • A univariate dataset is a dataset containing only one variable. For example these hypothetical exam scores form a univariate dataset:
      
      98  93  85  39  91  75  5  93  79  90 
    • A bivariate dataset is a dataset containing two variables. Some examples of bivariate datasets are weight vs. height, price vs. date, and score on exam vs. hours studied for it.
    • A multivariate dataset is a dataset containing two or more variables.
    • A nominal variable (also called a categorical variable) is a variable that contains non-numeric values, such as gender (female, male), is-smoker (yes, no), position (offense, defense), coin-condition (Poor, Fair/Good, F, VF, EF, UNC, BU).
    • A scale variable (also called a quantitative variable or continuous variable) is a variable that can contain any value in a continuous range. Examples are: age, height, weight, gpa, price.
    • An ordinal variable is a variable that contains discrete numeric values. Examples are: college-year (1, 2, 3, 4), survey-answer (1, 2, 3, 4, 5).
    • To summarize the various value of a univariate dataset, tally marks are traditionally used. Here are the sorted hypothetical exam scores:
      
      5  39  75  79  85  90  91  93  93  98
    • Here is the tally diagram:
      
    • A more informative technique is the stemplot, also called the stem-and-leaf display.
      

      95 | 8
      90 | 0133
      85 | 5
      80 | 
      75 | 59
      70 |  
      65 |
      60 |
      55 |
      50 |
      45 |
      40 |
      35 | 9
      30 |
      25 |
      20 |
      15 |
      10 |
       5 | 5
       0 |
      

      The last digit of each observation is recorded to the right of the vertical line down the middle, which is the "stem."
    • Histograms

  Quartiles

  • The Quartiles divide the sorted dataset into fourths.
  • Q0 is the minimum point in the dataset. Approximately 0/4 (none) of the data are less than than Q0.
  • Q4 is the maximum point in the dataset. Approximately 4/4 (all) of the data are less than than Q4.
  • Q2 is the median (middle point) of the dataset. Approximately 2/4 = 1/2 of the data are less than than Q2. If the dataset has an even number of observations, the Q2 is the average of the middle two observations.
  • Q1 is the median of the bottom half of the data. (If the dataset has an odd number of observations, the bottom half of the data contains the middle observation). Approximately 1/4 of the data are below Q1.
  • Q3 is the median of the top half of the data. (If the dataset has an odd number of observations, the top half of the data contains the middle observation). Approximately 3/4 of the data are below Q3.
  • Q1, Q2, and Q3 and also called the 25th, 50th, and 75th percentiles, respectively.
  • Exercise: Find the quartiles of the hypothetical exam scores:
    
    • 5  39  75  79  85  90  91  93  93  98
  • The interquartile range (IQR), is defined as IQR = Q3 - Q1.
  • Here are some alternative ways to compute percentiles.
  • There are at least different eight methods for computing precentiles.
  • Boxplots

  Practice Problems

  1. Match the descriptions with the histograms below:
     
     1. The gender of all persons in a college class (male = 0, female = 1).
     2. The handedness of all persons in a college class (left handed = 0, right handed = 1).
     3. The heights of all married persons counted separately.
     4. The heights of all persons in families where both parents are 28 years old or less.
     5. The heights of all automobiles.
     6. The incomes of all persons in the U.S.
     
      
     Ans: 1 iv, 2 iii, 3 i, 4 ii, 5 v, 6 vi.

  Using SPSS

  • Use SPSS to complete Project 1 in the computer lab after the break.
  • Use the reference documents found on the Documents Page.
  • How do you make a histogram when the variable is categorical? Solution: create a bar graph.

  Project 2

  • Look at Project 2a.
  • Start thinking about what data you want to collect for Project 2.