I don't exactly have his medical records to pull out, but that's been rumor pretty much accepted as fact for a while now. Surprisingly it's fairly common amongst comedians, which would strike you as somewhat counter-intuitive.
This all goes back way before he came out as anti-vaccine.
I'm pretty sure he just said that to get Jenny McCarthy to have sex with him. And really, I can't blame him.