Just as with almost everything else in life, it's not a simple case of black and white. Different people have different ideas about what's funny just as they have different ideas about everything else.
Is cracking jokes about someone that's paralyzed from the waist down, AIDS victims, rape victims, etc. generally seen as tastless and crass? Yes it is and rightly so. Does that necessarily mean that it's "wrong"? No, not really.
The thing about Freedom of Speech is that you have to take the bad along with the good. That's just like life in and of itself. Sometimes, it ain't gonna be so rosey and sweet but you've got to put up with that along with the good times. I don't agree with the bigoted views of the Ku Klux Klan, I don't agree with people that use the Bible to justify their hatred of homosexuals, I don't agree with people that claim to be speaking the will of God, etc. But, they have the right to think what they want and say what we think. Otherwise, Freedom of Speech is nothing more than a smokescreen.