Simple there is a drop down spot just look for the two little kid pandaren, drop down to them and talk to the old man again. he will just instant teleport you to the right spot now.