I think it's because the Undertaker rarely faces off against "top-level stars" at Wrestlemania...for whatever reason. He's always faced guys that were just slightly below him...probably the closest to him are Edge and Orton, who weren't quite at their peak. Then he faced Triple H in 2001 (not quite at his peak) and Flair (way past his prime), Diesel on his way out in 1996, HBK consecutive years (during his era of putting guys over). I think you could make the argument that if Undertaker were ever to face Cena at Wrestlemania, it would be his biggest WM match of all time. He just hasn't faced the best of the best in that year...it's always been somebody just a notch below him. I think that's part of the reason you've never seen Taker/Austin or Taker/Rock at Wrestlemania (or even Taker/Bret Hart)
Slightly below him? are you friggin serious. All those guys you mentioned were Top level main eventers and multi time world champs at the very least. how is that "below" him. You're talking like Undertaker is the best there ever was and will be which is far from the case. and no, Cena facing Taker now would be a dissapointment. would it draw? hell yes but Taker is way past it. now if it had been back when Cena was only the face of the company for a few years then yes that woulda been huge, but hardly the biggest mania match of all time. Way to overrate things.
Diesel was on his way out of the company but he was hardly "Over the hill", He'd just come off of being the longest reigning champ since Hogan, Flair may have been over the hill but he could still put on a good match despite being way past it. Triple H was ontop of his game, and if anything carried the match and HBK just did the job? seems that's what you are hinting at, well ofcourse he did, they weren't gonna drop the Streak and HBK was leaving anyway but who carried the match? answer HBK
I'll grant you other then the afformentioned names, Taker only faced upper midcarders at best, but that's all about booking and Taker's matches are more or less a, we have nothing for this guy to do match and Taker needs an opponent.
So to your statement i say those people were well above Taker, the fact he beat them was a way to give him props/respect for him being so dedicated and it's the longest running storyline in WWE history and always sells and the majority want to see him win.
Back to the OQ, IMO simple reasons,
A) It woulda been hard to book Austin to lose since he always had Taker's number
B) They had battled so many times already with Austin winning most of the matches
C) The perfect time for it to happen, ergo both guys were out of the title picture, they were both out injured too.