Search results
Effect size. Significance tests inform us about the likelihood of a meaningful difference between groups, but they don’t always tell us the magnitude of that difference. Because any difference will become “significant” with an arbitrarily large sample, it’s important to quantify the effect size that you observe.
Review: statistics • The language of statistics –Describes a universe where we sample datasets from a population • Interesting properties are proved for sampling distributions of parameter estimates • Statistical hypothesis testing –Helps us decide if a sample belongs to a population • A priori calculation of important statistical
- 1MB
- 56
ustments:Tukey or Scheffe are gener. ly used. For testing treatments against a control use Dunnett, if group sample sizes vary use Hochberg’s GT2 and if there is a difference between the group variances (Levene’s test gives a p-value < 0.05) use Gam. SPSS: Analyse General Linear Model Univariate.
- 1MB
- 53
Chapter 8. Statistical Inference 8.3: Introduction to Hypothesis Testing (From \Probability & Statistics with Applications to Computing" by Alex Tsun) Hypothesis testing allows us to \statistically prove" claims. For example, if a drug company wants to claim that their new drug reduces the risk of cancer, they might perform a hypothesis test.
- Three Modes of Statistical Inference
- Statistics for Social Scientists
- | {z } | {z } | {z }
- x) for all x
- f (X)
- k pk(1 p)n
- Simple Model-Based Inference
- Estimated Probability of Obama Victory in 2008
- Key Points
- The Key Assumptions
- Yi(0) j Ti = 1)
- Design Considerations
- Tradeoff between internal and external validity
- Identification vs. Estimation
- Joint distribution of yz = (Yi(0) = y; Z = z) is identified:
- Key Points
- Type II error
- Power Analysis
- Statistical Control of False Discovery
- Key Points
Descriptive Inference: summarizing and exploring data Inferring “ideal points” from rollcall votes Inferring “topics” from texts and speeches Inferring “social networks” from surveys Predictive Inference: forecasting out-of-sample data points Inferring future state failures from past failures Inferring population average turnout from a sample of vo...
Quantitative social science research: Find a substantive question Construct theory and hypothesis Design an empirical study and collect data Use statistics to analyze data and test hypothesis Report the results No study in the social sciences is perfect Use best available methods and data, but be aware of limitations Many wrong answers but no singl...
subjective objective subjective Statistical methods are no substitute for good research design
x) being continuous at every x If Xn ! d X, then for any continuous function f ( ), d f (Xn) !
Implication: Justifies asymptotic (normal) approximation
Sir Francis Galton’s Quincunx, Boston Museum of Science, or just check out YouTube
Setup: njk respondents of poll j from state k Model for # of Obama supporters in poll j and state k: indep:
Estimate pk for each state Simulate M elections using pk ^ and its standard error: for state k, sample Obama’s voteshare from N(^ \ pk; V(^ pk)) collect all electoral votes from winning states Plot M draws of total electoral votes Distribution of Obama's Predicted Electoral Votes Electoral Votes
Random sampling enables statistical inference Design-based vs. Model-based inference Design-based: random sampling as basis for inference Model-based: probability model as basis for inference Sampling weights: inverse probability weighting Challenges of survey research: cluster sampling, multi-stage sampling =) loss of efficiency stratified samplin...
The notation implies three assumptions: No simultaneity (different from endogeneity) No interference between units: Yi(T1; T2; : : : ; Tn) Same version of the treatment = Yi(Ti) Stable Unit Treatment Value Assumption (SUTVA) Potential violations: feedback effects spill-over effects, carry-over effects different treatment administration Potential ou...
Treatment effect heterogeneity: Zero ATE doesn’t mean zero effect for everyone! =) Conditional ATE Other quantities: Quantile treatment effects etc.
Randomized experiments Laboratory experiments Survey experiments Field experiments Observational studies
Endogeneity: selection bias Generalizability: sample selection, Hawthorne effects, realism “Designing” observational studies Natural experiments (haphazard treatment assignment) Examples: birthdays, weather, close elections, arbitrary administrative rules Generalizing experimental results: possible extrapolation Bottom line: No study is perfect, st...
Observational studies =) No randomization of treatment Difference in means between two populations can still be estimated without bias Valid inference for ATE requires additional assumptions Law of Decreasing Credibility (Manski): The credibility of inference decreases with the strength of the assumptions maintained Identification: How much can you...
i assumptions are valid, yz should be positive for all y and z Suppose that a negative value of ^yz is observed. Did this happen by chance? Statistical hypothesis test (next topic)
Causal inference is all about predicting counter-factuals Association (comparison between treated and control groups) is not causation (comparison between factuals and counterfactuals) Randomization of treatment eliminates both observed and unobserved confounders Design-based vs. model-based inference Observational studies =) identification problem...
Hypothesis tests control the probability of Type I error They do not control the probability of Type II error Tradeoff between the two types of error Size (level) of test: probability that the null is rejected when it is true Power of test: probability that a test rejects the null Typically, we want a most powerful test with the proper size
Null hypotheses are often uninteresting But, hypothesis testing may indicate the strength of evidence for or against your theory Power analysis: What sample size do I need in order to detect a certain departure from the null? Power = 1 Pr(Type II error) Four steps: Specify the null hypothesis to be tested and the significance level Choose a true va...
Pre-registration system: reduces dishonesty but cannot eliminate multiple testing problem Family-wise error rate (FWER): Pr(making at least one Type I error) Bonferroni procedure: reject the jth null hypothesis Hj if pj < m where m is the total number of tests Very conservative: some improvements by Holm and Hochberg
Stochastic proof by contradiction Assume what you want to disprove (null hypothesis) Derive the reference distribution of test statistic Compare the observed value with the reference distribution Interpretation of hypothesis test Statistical significance 6= Pay attention to effect size scientific significance Power analysis Failure to reject null 6...
Statistical Tests - Signal & Background. The probability to reject a background hypothesis for background events is called the background efficiency: 1. b = g(t; b)dt = ↵. tcut. The probability to accept a signal event as signal is the signal efficiency: s = Z g(t; s)dt = 1. tcut.
People also ask
What is a test statistic?
What is a statistical inference hypothesis test?
What is a study of sampling distribution of statistic for large sample?
The theory of test of significance consists of various test statistic. The theory had been developed under two broad heading. Test of significance for large sample Large sample test or Asymptotic test or Z test (n≥30) Test of significance for small samples(n<30) Small sample test or Exact test-t, F and χ2.
