Soliciting Feedback for Improvements to the Media Bias Fact Checker Bot
Soliciting Feedback for Improvements to the Media Bias Fact Checker Bot
Hi all!
As many of you have noticed, many Lemmy.World communities introduced a bot: @MediaBiasFactChecker@lemmy.world. This bot was introduced because modding can be pretty tough work at times and we are all just volunteers with regular lives. It has been helpful and we would like to keep it around in one form or another.
The !news@lemmy.world mods want to give the community a chance to voice their thoughts on some potential changes to the MBFC bot. We have heard concerns that tend to fall into a few buckets. The most common concern we’ve heard is that the bot’s comment is too long. To address this, we’ve implemented a spoiler tag so that users need to click to see more information. We’ve also cut wording about donations that people argued made the bot feel like an ad.
Another common concern people have is with MBFC’s definition of “left” and “right,” which tend to be influenced by the American Overton window. Similarly, some have expressed that they feel MBFC’s process of rating reliability and credibility is opaque and/or subjective. To address this, we have discussed creating our own open source system of scoring news sources. We would essentially start with third-party ratings, including MBFC, and create an aggregate rating. We could also open a path for users to vote, so that any rating would reflect our instance’s opinions of a source. We would love to hear your thoughts on this, as well as suggestions for sources that rate news outlets’ bias, reliability, and/or credibility. Feel free to use this thread to share other constructive criticism about the bot too.
You're viewing a single thread.
One problem I’ve noticed is that the bot doesn’t differentiate between news articles and opinion pieces. One of the most egregious examples is the NYT. Opinion pieces aren’t held to the same journalistic standards as news articles and shouldn’t be judged for bias and accuracy in the same way as news content.
I believe most major news organizations include the word “Opinion” in titles and URLs, so perhaps that could be something keyed off of to have the bot label these appropriately. I don’t expect you to judge the bias and accuracy of each opinion writer, but simply labeling them as “Opinion pieces are not required to meet accepted journalistic standards and bias is expected.” would go a long way.
Thanks for this. As a mod of /c/news, I hadn’t really thought about that. We don’t allow opinion pieces, but this is very relevant if we roll out a new bot for all the communities that currently use the MBFC bot.
No problem. Specifically came to my attention about a week ago on this post where the bot reported on an opinion piece as if it was straight news.
BTW, I actually do appreciate the bot and think it’s doing about as well as it can given the technical limitations of the platform.
Hi. I have a suggestion:
Try to make it more clear that this is not a flawless rating (as that is impossible).
Ways to implement:
- Make sure the bot says something along the lines of “MBFC rates X news as Y” and not “X news is Y”.
- Make a caveat (collapsable) at the bottom, that says something along the lines of “MBFC is not flawless. It has an american-centric bias, is not particularly clear on methodology, to the point where wikipedia deems it unreliable; however, we think it is better to have this bot in place as a rough estimate, to discourage posting from bad sources”
- If possible, add other sources, Like: “MBFC rates the Daily Beast as mostly reliable, Ad Fontes Media rates it as unreliable, and Wikipedia says it is of mixed reliability”
- Remove the left right ratings. We already have a reliability and quality rating, which is much more useful. The left-right rating is frankly poorly done and all over the place, and honestly doesn’t serve much purpose.
This contributes significantly to the noise issue most people complain about
Interesting that people say that opinion pieces should not be held to the same standard. I personally see such pieces contribute to fake news going around. Shouldn't a platform with reach, held accountable for wrong information, they hide behind an opinion piece?
It’s not a question of “should” - an opinion piece is rhetoric, not reporting. You can fact check some of it sometimes but functionally can’t hold it to the same standards as a regular news article. I agree that this can sometimes lead to “alternative facts” and disingenuous arguments, but the only other option is to forbid the publication of them which is obviously an infringement of first amendment rights. It’s messy, and it can lead to people being misinformed, but it’s what we’re stuck with.
Can you explain how a piece with a title like "Helldivers is awesome and fun" can be judged at all for factual accuracy?
The NYT ran an opinion recently where the author pretty clearly was using the NYT along with other outlets as part of a voter demobilization tactic in which the author lied about not voting. The NYT was skewered on twitter, and had to alter the opinion after the fact. It seems like some basic fact checking would have been useful in that situation. Or really, just any amount of critical thought on the part of the NYT in general.
This. Otherwise op-eds get a free pass to launder opinions the paper wants to publish, but can't.