700
edits
Changes
no edit summary
==Concept behind null hypothesis testing==
==Concept behind null hypothesis testing==
Hypothesis tests provide a systematic, objective method of data analysis, but do not actually answer the main question of interest (which is commonly along the lines of 'is there a difference in disease experience between individuals with exposure x and individuals without exposure x?'). Rather, hypothesis tests answer the question ''''if there is no difference in disease experience''' between individuals with or without exposure x, what is the probability of obtaining the current data (or data more 'extreme' than this)?' As such, hypothesis tests do not inform us whether or not there is a difference, but instead they offer us varying degrees of '''evidence''' in support of or against a situation where there is no difference in the population under investigation. This situation of 'no difference' is known as the 'null hypothesis', and should be stated whenever a null hypothesis test is performed. If there is little evidence against the
Hypothesis tests provide a systematic, objective method of data analysis, but do not actually answer the main question of interest (which is commonly along the lines of 'is there a difference in disease experience between individuals with exposure x and individuals without exposure x?'). Rather, hypothesis tests answer the question ''''if there is no difference in disease experience''' between individuals with or without exposure x, what is the probability of obtaining the current data (or data more 'extreme' than this)?' As such, hypothesis tests do not inform us whether or not there is a difference, but instead they offer us varying degrees of '''evidence''' in support of or against a situation where there is no difference in the population under investigation. This situation of 'no difference' is known as the 'null hypothesis', and should be stated whenever a null hypothesis test is performed.<br>
Null hypothesis testing relies on the fact that the likely values obtained in a sample taken from a population in which the null hypothesis is true (i.e. there is no difference in disease experience between groups) can be predicted. That is, although it is most likely that a sample from this population will also show no difference, it is not impossible that, through chance, a sample will contain more diseased animals in the 'exposed' group than in the 'unexposed' group, for example. Through knowledge of the number of animals sampled and the prevalence of disease in the population, the probability of getting any particular pattern of data from this sample can be ascertained. This probability is known as the '''p-value''', and is the main outcome of interest from a hypothesis test. A p-value of 1.00 would suggest that if there was no difference in disease experience according to exposure in the population and repeated samples were taken from this population and performed hypothesis tests on these, all of these samples would be expected to show this pattern (or 'more extreme' - i.e. show more of a difference). However, a p-value of 0.001 would suggest that there is only a 0.1% chance of seeing these data (or more extreme) if there was no true difference in the population. Note that although 0.001 is a very low p-value, it cannot be used to ''prove'' that the null hypothesis is false, or that the 'alternative hypothesis' (of their being a difference) is true - it can only be stated that it gives 'strong evidence against the null hypothesis'.
===Significance levels===
Although the approach described above (of varying degrees of evidence against the null hypothesis) is the most statistically correct interpretation of p-values, it is often not practical to apply this in epidemiological analysis. For example, an investigator may want to identify a number of exposures which appear to be associated with the outcome in order to investigate these further - the approach described above would not allow this (as it will never lead to an association being '''proven'''. As such, many studies will use '''significance levels''' as a method of interpretation of p-values as 'significant' or 'not significant'. Commonly, a p-value of 0.05 or less is used to denote a 'significant' association. This means that if the there is a 5% chance or less of observing the data in question (or more extreme) if the null hypothesis is true, then the association will be denoted as 'significant'. Of course, there remains a 5% chance that there is no true difference in the population, and this can be a problem when testing large numbers of exposures (as shown in the cartoon [http://xkcd.com/882/ here]). Because of this, great care should be taken whenever using significance levels to interpret hypothesis tests, and the actual p-value should always be presented.
===Limitations of null hypothesis tests===
==Hypothesis testing and study power==
==Hypothesis testing and study power==