If I understand what you are asking, you can use the median as an average since it is not affected by the mean ( median is a measure of center). Since a mean is affected by outliers, it might lower the score if there are very low outliers. But note that if there are outliers way <em>greater </em>it can increase the average. You can use the Inter Quartile Range if shown on a box plot ( measure of variability).
