You'll just lose your arena team, but you won't lose your arena points.