Per se means in or of itself. Here are some examples:
"It's not the facts per se that are important." The facts by themselves may not be important, but with other information, they become important.
"It wasn't her attitude per se that bothered me." Her attitude may not have, by itself, bothered you, but perhaps other things about her did (e.g. her choice in music, clothing, etc.).