curación y moderación de contenidos

LetraChica and the Oversight Board

The “supreme court” created by Facebook to review the most high-profile cases of content removal from the platform has already handed down its first rulings. Undoubtedly, the most important decision is still pending: let’s remember that the Content Advisory Board—as it is formally known—also holds the final say on the permanent suspension of former President Donald Trump’s account.

By CELE

How much will content moderation improve under Facebook’s new “Supreme Court”? The “Supreme Court” created by Facebook to review the platform’s most high-profile content removal cases has already handed down its first rulings. Undoubtedly, the most important decision is still pending: let’s remember that the Content Advisory Board—as it’s formally known—also holds the final say on the permanent removal of former President Donald Trump’s account. For now, the initial cases suggest that while the board is analyzing decisions with additional context and consideration—which makes it a valuable resource—this process is unlikely to resolve the underlying issues in the platform’s moderation process. Similarly, Facebook’s responses to the Council’s recommendations regarding changes to community standards suggest that this body will not have an immediate impact on this structural issue. Below is a brief overview of the Council’s creation, a discussion of the decision-making process, and an analysis of Facebook’s responses to the recommendations for changes to its moderation policies and processes. A Little Background In November 2018, Mark Zuckerberg announced that Facebook would create an independent body to provide final review of its content moderation decisions. The rationale behind creating this body, Zuckerberg said, was to ensure that Facebook did not make “so many” important decisions regarding free speech and safety on its own. In September 2019, Facebook published its Oversight Board Charter, the document that explains the structure and mandate of this “court,” as well as its relationship with Facebook. The Board’s main function is to review certain moderation decisions and overturn them when it deems necessary. However, it can also make recommendations to Facebook regarding its moderation policies. In May 2020, Facebook announced the first twenty members of the Board, who began reviewing their first cases in October. Of the seven removal decisions reviewed, the Board has overturned five and upheld one. In one case, no decision was made after the content was no longer available on the platform (click this link to see a summary of the cases and decisions). The gaps between the Council and the day-to-day moderators are immense The Advisory Council’s bylaws have established a complex process for reviewing and deciding each case within a maximum of ninety days. During this time, a five-member panel must hear from the user involved, review the case, deliberate, draft a decision, present it to the rest of the members, and reach a final decision. In addition, the panel may seek the opinion of experts, who, for example, can provide information on the local context. Between October 2020, when the Council began accepting cases, and March 2021, the Council resolved only seven cases. Here are two examples illustrating the level of detail in the analysis:

  • A user posted, without any context, a quote incorrectly attributed to Joseph Goebbels, Adolf Hitler’s propaganda minister. The quote was removed because Goebbels is included on Facebook’s list of dangerous individuals. Before the Council, the user explained that their intention was to compare the sentiment of the quote to Donald Trump’s presidency. The Council ruled in the user’s favor, explaining that the user’s friends’ responses suggested their intention was not to express support for Nazi ideology.

  • A user posted a video promoting the use of hydroxychloroquine and azithromycin to treat COVID-19, along with text criticizing the French government for not recommending this treatment. The post was removed by Facebook for containing misinformation that could cause imminent physical harm. The Council overturned Facebook’s decision by Facebook, noting that the post was a critique of government policy and that Facebook failed to demonstrate that the post increased the risk of harm, given that the recommended medications are not available over the counter in France and the content did not encourage people to take them. This level of detail in the contextual analysis stands in stark contrast to the day-to-day moderation system, which consists of a mix of algorithms and human moderators who must make decisions within minutes. In that process, there is rarely an analysis of comments from a user’s friends or an assessment of the relationship between the risk of harm and a product’s commercial availability. Will Facebook heed the Council’s recommendations? If the Council’s decisions only affect the people involved, to what extent can they improve Facebook’s basic moderation system? Is the effort even worth it? One interesting aspect of this experiment is that the Council is gaining access to inside information about moderation processes, which it uses to make recommendations to Facebook, in accordance with its mandate. This could be a way to promote changes with broad impact, going beyond specific cases. The initial recommendations have shown that this panel has taken an interest not only in the substantive moderation policies (what is prohibited and what is permitted), but also in the processes for enforcing these policies. Facebook has committed to responding in good faith to all of the Board’s recommendations (you can view here a summary of the recommendations and Facebook’s responses). Reading Facebook’s initial responses reveals some insights into its moderation process: According to Facebook, algorithms can detect text superimposed on images (such as the words “breast cancer” on an Instagram photo), but they still have difficulty distinguishing between permissible and prohibited nudity; automated detection systems are generally as accurate as human moderators, and most appeals are reviewed by moderators. But will the recommendations shape the course of moderation on the platform? Generally speaking, Facebook provides three types of responses: “committed to action,” “assessing feasibility,” and “no further action.” In the blog post where it announces its “detailed responses,” Facebook says it is committed to _“taking steps to address 11 of the _[17] recent recommendations from the Council.” However, while Facebook does commit to some changes, other responses seem to sidestep the recommendations:

  • The Council recommended that Facebook increase the transparency of its moderation procedures regarding health-related content by publishing a transparency report on how the policies were enforced during the pandemic crisis. According to the Council, this report should include: data on the number of removals and other enforcement actions, a breakdown of the types of content to which the rules were applied, methods of detection, a breakdown by region and language, metrics on the effectiveness of measures less invasive than removal, data on appeals and outcomes, and lessons learned. Facebook’s response was that it is “committed to taking action.” However, in this case, its commitment to action consisted of reiterating that it already regularly publishes information about its efforts to combat COVID-19 misinformation (such as removed posts, applied labels, and the number of visits to its COVID-19 Information Center). Facebook also said it will continue to look for ways to communicate the effectiveness of its efforts. However, it clarified that it does not plan to include an additional category in its transparency report, given that the pandemic is a unique and temporary situation.

  • Some responses oversimplify the Board’s recommendation, making the scope of the commitment unclear. For example, the Council recommended adopting less intrusive enforcement measures regarding health-related misinformation. The recommendation included specific details: clarifying the harms it seeks to prevent, being transparent about how harm is assessed, assessing the current range of tools for addressing these cases and considering the development of less intrusive tools, publishing its various enforcement options within the policies and classifying them according to how intrusive they are, explaining how the least intrusive option is selected, and clarifying in the policies which enforcement option corresponds to each rule. Facebook said it is committed to taking action, and in its response noted that it will soon launch a Transparency Center—which it has been working on for months—where it will provide more details on what is not permitted and on when it chooses to provide more context and use labels instead of removing content.

  • The scope of commitments such as “We will continue to invest in making our machine learning models better at detecting the types of nudity that we do allow” is also unclear. Apparently, this is an effort that was already underway (“we will continue”) and Facebook does not commit to specific dates or metrics to demonstrate improvements.

  • The Board recommended that, in conjunction with relevant stakeholders, Facebook conduct an assessment of the impact of its decisions on human rights as part of the rule-updating process. The commitment to action here was to (i) ask the Council to clarify whether the recommendation relates to all rule changes or only those related to COVID-19 misinformation, and (ii) based on this, assess whether there are opportunities to strengthen the incorporation of human rights principles into the policy development process.

  • Some responses stating “assessing feasibility” appear insufficient. For example, the Council recommended that users always be informed if an automated system was involved in the decision to remove content. Facebook is evaluating whether providing this information might cause further confusion among users, arguing that automated systems are typically just as accurate as human moderators. In another recommendation, the Council suggested that users be informed of the specific rule within the policy that justifies the removal. Facebook is evaluating the feasibility of this suggestion, given that neither human moderators nor artificial intelligence systems are trained in this manner. Facebook argues that it is not technically possible to create separate automated systems for every sentence in the policy.

  • Finally, Facebook did not respond in any way to the recommendation to implement an internal audit process to consistently analyze a statistically representative sample of automated content removal decisions that should be reversed, in order to learn from the errors of the compliance system. This first round of recommendations and responses suggests that the power of this “supreme court” will be limited. However, it’s worth giving the interactions between the Council and Facebook some time to see what their scope will be. For now, the experiment remains interesting enough to warrant keeping an eye on it. Joint project @linterna-@celeup