How come there are people able to LOS the guards inside the Shattered Sun Offensive quest areas? I trying to do quests with a mate and we keep getting ganked just inside the building where the guards can't 'see' us. Also I've been attacked in plain sight by the guards before doing these quests. Is this a bug or is this actually how broken the guards are...

Thanks.

EDIT: The guards are friendly during all of this...