I had this problem too, it's a headache, but it's doable. Just use instant ranged attacks on the riders(I used Ice Lance), not all of them are attackable, just get on the drake, get the target, and try to attack it for like 15-20 seconds(sometimes there's only a short space of time when it's attackable, so you might ride a few circles around that temple to actually kill the rider), if it doesn't happen - change the drake and repeat. And for some reason the quest itself tells to kill 10 riders, but I got it "completed" when I killed only 1, but couldn't return the quest. You HAVE to kill exactly 10 of them in order to complete the quest. Hope this helps :)