The level of significance refers to the probability of the null hypothesis that may be rejected in the study by the observer. As the sample size increases, the p value also increases but the significance remains same. When the sample size increase and becomes large, the null hypothesis can be rejected more easily.
The business decisions are taken defending the significance level of the statistical validation in cases of uncertainty. Suppose the null hypothesis is the sales after the marketing campaigning remains same. The company will launch the marketing campaign when the confidence level is 95% so that after the campaign the sales will increase. It means that the significance level must be high enough for rejecting null hypothesis of sales post of the marketing campaign that remains same.
When the sample size increases, the sensitivity of hypothesis increases, and for a large sample, the hypothesis may be rejected even though it is true.
