This is an excerpt from our webinar "Alternatives to the P-value and Power - The effect on Sample Size Determination." where our host Ronan Fitzpatrick, Head of Statistics at Statsols demonstrates an alternative to traditional sample size calculations by examining Confidence Interval (CI) and the role of precision sample size.
When the ASA released their statement on p-values and statistical significance, it created a new wave of discussion and debate on p-values and null hypothesis significance testing (NHST).
With numerous alternative approaches to statistical inference being championed and some calls to abandon statistical significance completely, it has never been a better time to catch up on these alternatives and see their potential effects on the areas of study planning and sample size determination.
Below is an extract from the webinar where our host Ronan Fitzpatrick, Head of Statistics at Statsols demonstrates an alternative to traditional sample size calculations by examining Confidence Interval (CI) and the role of precision sample size.
Can't watch a video right now?
The slides and auto generated transcript for the webinar are below.
A confidence interval (CI) is a type of interval estimate, computed from the statistics of the observed data, that might contain the true value of an unknown population parameter.
The confidence interval in the frequentist school is by far the most widely used statistical interval and the Layman's definition would be the probability that you will have the true value for a parameter such as the mean or the mean difference or the odds ratio under repeated sampling.
Sample size determination is targeting the interval width, that's quite closely tied to the idea of the standard error of the estimate which is tied to your law of large number theory.
But of course, we're not focusing on power today. It's just a small note you may be interested in.
Below are some simple confidence interval sample sizes because the assumption here is that most of you are familiar with doing this. The major thing that we're doing here is that we're changing the framework in the sense that you we have confidence level instead of significance level standard deviation would be the same thing if we were doing a one-sample t-test but now instead of power we're targeting the half width of the confidence interval here. This is a real paper just looking at Rosiglitazone for insulin sensitivity from diabetic medicine and this is just a simple example that we'll explore. You can find these examples in detail in the Webinar video above.
In many cases you'll probably as find sample size for confidence intervals is fine, but there are other frequentist intervals that are used or perhaps are underused that can target other types of parameter.
Two of the other frequentist intervals you may be familiar with but maybe haven't used in a while is the prediction interval and the tolerance interval.
So the prediction interval is an interval to contain a future sample subject or parameter with a given probability.
A tolerance interval is an interval within which, with some confidence level, a specified proportion of a sampled population falls.
