Trolls have been in online forums for as long as there have been online forums, and there are generally two types: the good and the bad. The good ones are more akin to satirists; they present an idea or a statement that is demonstrably false, but plausible, and do so in the spirit of play; people who clue into it are welcome to play along and share in the joke. The bad trolls, on the other hand, get their amusement by stirring up trouble with inflammatory posts, starting flamewars, and just generally seeing other people get worked up over nothing; the angrier people get, the happier the bad troll.
The conventional wisdom on how to deal with the latter type is simple: don't feed the trolls. The theory is sound; trolls are looking to provoke a reaction, so if you ignore them, they'll eventually get bored and go away. Personally, however, this is not my policy. I never feel comfortable with assuming someone is a troll, in part because I know there are real people out there who genuinely believe outrageous things, and so someone I dismiss as a troll could actually be sincere. Moreover, in a public forum, even if someone is trolling, it doesn't necessarily follow that everyone in the audience of lurkers recognizes that; there might be someone who might actually agree with what the troll says, and who therefore is in need of a healthy dialogue on the subject. So my policy has always been to take posts at face value, and not to concern myself with whether or not the person posting it actually believes it.
I first started consciously using this policy about 14 or 15 years ago, when I decided to put my money where my mouth was on the subject of free speech. (I've always argued that the solution to hate speech is not censorship, but vigorously debunking the hate speech. In other words, vaccination rather than quarantine.) I spent a few month arguing in the Usenet newsgroup alt.politics.white-power.
Now, I didn't have any illusions going into this. I didn't expect to convert any racists into champions of tolerance. I assumed that anyone actually arguing with me probably had made up their mind and would be at least as resistant to changing it as I was (and I've invested a fair bit into my anti-racist position, having not only entered into an interracial marriage but produced a healthy hybrid child by it). My purpose there was primarily for the lurkers, and those among them who might not have completely made up their minds.
That's why I adopted for myself some ground rules. I committed myself to always be as polite and respectful as possible, and to consider the ideas presented as fairly and rigorously as possible. I never wanted to leave a lurker wondering why I left some question unanswered, or worse, indirectly insulted them by saying that anyone who could believe such rubbish was an idiot. Calling people idiots (even if they are) doesn't really gain you much credibility or respect, and if anything, makes them more resistant to your arguments.
Of course, I have no idea how successful this approach was with the lurkers, since of course they're lurkers; I wouldn't likely hear from them if I made any difference. But I did enjoy a rather surprising indicator of success: several months after I retired from that particular arena, I got an email from one of the people who had been openly arguing with me in the newsgroup, thanking me for taking the time and being patient and respectful, and telling me that he'd come to see he had been wrong about the race thing. Admittedly, he wasn't one of the most hardcore and dogmatic there, but was immensely gratifying to have had a tangible effect. And I like to think that if my approach was effective enough for that, perhaps it helped a good number of lurkers, as well.
That's why I've chosen to use the same approach in dealing with trolls. Remember that for the most part, the bad trolls are trying to provoke a flamewar, so responding emotionally to inflammatory posts plays right into their hands. If you can't respond rationally, then of course it's best to ignore them, but if you can remain calm and logical, a polite and respectful response is often more effective. And it might even be useful to someone some day, googling for an answer to a question no one else seems to think it worth taking the time to answer.
So that's my policy on trolls. Except for the good ones, with whom I hope I'm clever enough to catch on and play along.