It doesn't reduce damage like Armor. It gives you chance to resist part of the damage(increments of 10%).
So, for example, if you have 55 resistance(what you get with 75 Spell pene against totems/auras)
20% chance to resist 0% damage
47% chance to resist 10% damage
28% chance to resist 20% damage
4% chance to resist 30% damage
for average 10% resisted you need something like 45 Resistance
gaining more Resistance will increase your chances to get higher % resists and eliminate lower % resists, there are certain caps for this, though every point helps. I think it was 100 Resistance to have guaranteed at least 10% resist on each spell that hits you, with average 20% resisted over lots of spells hitting you.
Keep in mind, these numbers might not be all accurate, writing from memory.