#platformlaw: How do appeals regarding private content moderation work?

In the public interest: Perhaps the distinction regarding the public interest is most useful for informing prioritization decisions. If the conversation or content is in the public interest, it should be prioritized. Such determinations will likely arise, however, from the intersection of two or more of the criteria we have already mentioned: local/regional context; the nature of the person (public or private); and the relevance of the topic. One of the unanimous conclusions among colleagues is that prioritization should likely be based not on a single criterion but on a variety of factors. How to implement this will be another matter… Regardless of the decision companies make on this issue, the exercise served to highlight how little we know about this topic, the lack of information in reports and in the policies themselves, and the need to study the issue more deeply.

In response to the growing trend of private content moderation, major online content intermediaries have, for several years now, adopted private review systems designed to provide an internal review mechanism for potential errors or inconsistencies in the application of their terms and conditions of service. Some authors view the appeals system as a prerequisite for the right to access to justice (M. Common, 2019). Along these lines is also the proposal by Public Knowledge to require large platforms to guarantee access and due process (2018) for their users. For some, it is an element that contributes to the legitimacy of the private system of self-regulation and content moderation, while for others (Klonick, 2019) it is an essential part of the “economic responsiveness” these companies demonstrate toward their users. In other words, in response to their economic interests, these companies establish mechanisms to listen to and respond to what their community asks of them. The truth is that for several years now, major internet companies have been offering complex content moderation systems and systems for reviewing their own decisions regarding such moderation. Following a query from a colleague and the publication of Facebook’s latest transparency reports in September 2020, we began a process of reviewing and studying publicly available information regarding appeals of content that had been removed or “moderated” (to use the terms employed in the reports in question) by these platforms. A first observation is that this topic has been little studied and analyzed, both in our region and in a comparative context. This seems paradoxical given that the three leading global content platforms (Facebook, Twitter, and YouTube) announced in March the suspension—or significant reduction—of their efforts in this area due to social distancing measures and global lockdowns that forced the platforms to send their moderators home. The available reports and various company statements over the past six months indicate that this has had a direct and significant impact on the platforms’ moderation and appeals processes, particularly during the last quarter studied (April through June 2020). Driven by growing curiosity, we at CELE organized two meetings. The first was with Linterna Verde, with whom we have been monitoring changes to the platforms’ Terms of Service (ToS) since early 2020, in order to assess whether there were any formal changes in this regard; and, given that the appeals process is ultimately nothing more than an extension of the content moderation process, to reflect on our limited knowledge in this area. From the very beginning, we confirmed that there have been no significant changes to the community guidelines or the ToS of Facebook, Twitter, and YouTube. In other words, even if there were changes in the companies’ practices, these have not been formally reflected in the rules. This may be due to a variety of reasons, which we will analyze in detail shortly following the first report from the monitoring effort we are conducting with Linterna. It’s worth noting that few practices lead to changes in the Terms of Service, and even those that might do so are often communicated through other mechanisms that complement the initial self-regulatory documents (blog posts, speeches, shareholder reports, etc.). The second meeting was held with members of AlSur—an alliance of civil society organizations and regional academic institutions of which we are a part—specifically to discuss some of the unanswered questions that arose from our conversation and the very preliminary study we have conducted on the topic so far, to find common ground, and to identify new questions that will inform this initiative. What are private content moderation appeal systems, and how do they differ? How are they functioning, and how were they affected during lockdowns? How are resources prioritized as restrictions on movement and gatherings are lifted in different regions of the world? What have the platforms communicated regarding these processes? How should they operate moving forward, given the fluctuations caused by the pandemic and the uncertainty that characterizes this “new normal”? (which is anything but normal). These are some of the questions that guided this preliminary study and the meeting we convened. A bit of context: Across the globe, laws, bills, and/or massive campaigns aimed at regulating major online content platforms are proliferating. These range from structural reforms to intermediary liability regulations (the U.S., with its three current bills, Europe with its DSA—these are just two of the most significant examples due to their potential impact or ripple effect). All of them propose limits on immunity, which currently seems to be the generally desirable principle regarding intermediaries’ liability for third-party content. The private content moderation carried out by companies appears to be one of the major catalysts for these reforms. Views range from those who argue that companies are biased and partial in their content moderation to those who claim that such mechanisms promote hate, fake news, and a host of other ills currently plaguing the internet. The changetheterms campaign sums up this position well. Along these lines, there are major debates—both at the government level and within civil society—about how companies’ terms and conditions of service or community guidelines should be defined so that the content circulating on their platforms protects and guarantees basic human rights, dignity, democracy, and so on. Different perspectives prioritize different objectives. Over the past four years at least, we have seen a dramatic change in these companies’ terms and conditions of service. These documents, which were once short and vague, have taken on a level of complexity, length, and detail never seen before (Klonick 2019). In the companies’ own words, the ToS change regularly to adapt to the supposed needs of users or the communities to which they apply, and in response to changes in context, current developments, and so on. This shift toward self-regulation has been accompanied by growing demands for transparency from both civil society—see, for example, the Santa Clara Principles—and governments —see, for example, the transparency and implementation requirements of the German NetzDG. What do we know about the review of moderation decisions? Very little information is available regarding appeals in companies’ terms of service. Perhaps the most comprehensive information we have on this topic is on the onlinecensorship.org website, which details the appeal process for each platform. According to this website, almost all major platforms offer an appeal system for users whose content has been blocked for violating internal policies or the Terms of Service (ToS). Not all of them provide a review process for other measures, such as account suspension or termination. Twitter, for example, offers such a review process and sends notifications via email. Facebook also offers a complaint form when an account has been deleted “in error,” but it is unclear how this fits into the reported system of appeals and removals. Its appeal system covers removed content, deleted groups, or deleted photos. YouTube operates on a strike system, and users can appeal strikes but not account termination. In response to increased automation and frustration with the appeals system during COVID, YouTube announced that it was suspending the strike system, anticipating that automation might lead to more errors. In September, YouTube acknowledged that the implementation of AI had resulted in twice as many removals as usual and reinstated a significant portion of its human resources assigned to moderation tasks. On the other hand, each platform has a different system for detecting content that violates its Terms of Service (ToS), with varying levels of automation—or at least varying stated levels of automation. Facebook, for example, reports extremely high levels of automation (over 88%) in virtually all content categories except bullying and harassment (15.6%). Twitter does not report automation in the same terms, but in a 2019 blog post, it reported an increase in the proactive detection of content and accounts violating its Terms of Service, which allegedly rose from 20% to 38%. YouTube, for its part, reports greater automation, though this is not quantified. It recently reported that, with regard to extremism, thanks to automation, nearly 90% of problematic content has been proactively detected before it was viewed by more than 10 users (How YouTube uses technology to detect violating content: Violent Extremism). Facebook is the company that provides the most information about how its moderation system and the appeals process work; it distinguishes between content proactively removed versus content reported by users, and content proactively restored versus content restored following an appeal. It also provides the number of appeals by category under its Terms of Service. Notwithstanding this, it is unclear how the two variables interact. Furthermore, the report states that the numbers indicate the quantity of content removed, but this calculation is based on the number of posts removed—even if the same reposted piece has been removed multiple times. In other words, if a user reports a post that has been duplicated, triplicated, and reposted by multiple users, the total number of removals will be counted, not the total number of original posts. It is also unclear whether the number of appeals represents the number of pieces of content appealed or the number of appeals regarding the same piece of content. Finally, the report states that it does not include data on appeals regarding actions taken on accounts, pages, and groups. In other words, it only includes appeals regarding individual pieces of content. As for prioritization—both for moderation and for appeals—different companies prioritize based on different variables, and the systems used to determine this prioritization also vary. YouTube established a system of “trusted flaggers” that consists of granting higher status to NGOs, government agencies, and users who have demonstrated their effectiveness in reporting. These individuals are even offered training to help them carry out this task, and their reports are prioritized during moderation. Facebook, for its part, is relying less and less on user reports and more on technology developed to automate the detection of content that violates its policies (with all the problems that entails). They prioritize moderation of “content related to safety, including material related to terrorism and suicide” (“We prioritize safety-related reports, including material related to terrorism and suicide”). It is unclear what the prevailing criteria are for prioritizing appeals or whether these criteria changed during the pandemic. In Twitter’s case, there is little specific language on this, but the aforementioned April 2019 blog post mentions that they will continue to strive to improve content detection rates, particularly when the content contains “private information, threats, or other abusive content.” Twitter’s content moderation and appeals system at least appears more rudimentary in certain respects than YouTube’s or Facebook’s, which has resulted in both advantages and disadvantages. Among the advantages, as highlighted by the website onlinecensorship.org, regional representatives are given a say whenever possible. The disadvantage is that, much like “trusted flaggers” programs, the prioritization of these appeals often depends on third parties providing context to the company to justify that prioritization. The language, metrics, and even timeframes vary from one company to another, which hinders a comparative analysis of platforms. Currently, for example, YouTube and Facebook have released transparency reports through August 2020. Twitter has not yet published any reports for 2020, as shown on its transparency page. Some Theoretical/Philosophical Issues Regarding Appeals The idea that platforms should guarantee minimum principles of due process to their users in relation to their own content moderation is not new. Professor Dawn Nunziato already argued in her introduction to Toward a Censorship-Free Internet (CELE 2013) that, in order to harmonize the prohibition on prior censorship with platforms’ actions regarding content removal, minimum standards of due process and transparency must be guaranteed. Brenda Dvoskin recently pointed out that the proposal to promote the incorporation of the human rights framework as a guiding framework for platform self-regulation, despite its appeal, raises problems regarding its implementation. Among other things, she highlighted an aspect that I found particularly interesting and which, for the purposes of these reflections, I will refer to as the “dilemma of objectives.” She argued that all human rights treaties require the State to ensure that limitations on freedom of expression serve the legitimate objectives identified by those treaties. Applying this requirement literally to a company implies significant restrictions on its ability to conduct its business—and, indeed, on its own exercise of freedom of expression. Should the same limits apply to the private sector as to the public sector? Should we consider new legitimate objectives for this group? When it comes to due process, we face similar concerns. Human rights treaties address guarantees of impartiality and independence in the determination of rights, as well as minimum safeguards—particularly applicable to criminal and administrative penalty proceedings. Case law on this subject is extensive and wide-ranging, but complex questions remain regarding the possible transfer of these principles to the private sector. Are the requirements of the judicial system transferable to private systems? Do they require any adaptation? Gillespie recently explained that platforms have turned to automation in content moderation as a response to the problem of scale. He added that this has been the solution they have offered to both civil society and governments to address issues of volume and time—through automation, platforms seek to intervene in problematic content as soon as possible (even before any user sees or reports it) and apply the same criteria to as much content as possible (scale). If we are to accept this logic, we obviously need to adapt our understanding of due process, the concept of a tribunal, reasonable timeframes, independence, and impartiality. If this is not the shared logic, questions also arise: How can we guarantee independence and impartiality at scale? Or equality before the law? Faced with unknown moderators or call centers, how can we exercise our rights of defense? In the face of a pandemic that forces moderators to work from home, how do we guarantee this right during quarantine? The Santa Clara Principles are currently being revised precisely to contribute to this debate. And at the regional level, recommendations from a group of organizations led by Observacom were published in August. Both efforts would benefit from more empirical data capable of informing the development of concrete principles and from an in-depth debate capable of building consensus around the standards that should govern these practices. At present, the relevance of these systems and their impact on companies’ internal ecosystems—let alone on the communications ecosystem as a whole—remain unclear. Some questions and a priori assumptions that could inform future research on these topics: Given the systemic lack of information surrounding this particular issue, one of the first conclusions we reached with colleagues is that it is difficult to make recommendations. It even seems difficult to determine whether a prioritization system should be established for moderation or for appeals, at least under normal circumstances. The question, of course, takes on new nuances in the midst of the COVID-19 outbreak or spread, which forced platforms to send most of their moderators home. Even if they reopen now, the “new normal” is currently characterized by high levels of uncertainty: threats of resurgences and new lockdowns, a lack of long-term measures, border closures, offices still shut down, banned gatherings, and so on. In this context of COVID-19, limited resources, and uncertainty, does prioritization even make sense? What happens to content that isn’t prioritized in this context? How is the impact of the prioritizations made during this period measured? Regardless of the context, on what criteria should this prioritization be based? By geographic criteria: Transparency reports do not provide disaggregated data on the impact of content moderation or appeals against such moderation at either the local or regional level. It is not entirely clear how the exercise of rights—such as the right to appeal—is distributed geographically and culturally. Where are there stronger cultures of institutionalization and the exercise of rights, and where are these practices less common? Nor is it clear what impact a community may have on the total number of cases by category. Hence, one recommendation might be to consider different criteria for prioritization based on the region and the current situation affecting it. By topic: In addition to distinguishing by country or region, it may also be advisable to consider the specific topic at hand. If, for example, a country is in an election cycle or experiencing an armed conflict, might it be advisable to prioritize content related to these types of issues (similar to a hashtag)? By type of person: Within the framework of the right to freedom of expression, there is traditionally a distinction between public figures and private individuals, with this distinction serving as a necessary criterion for evaluating the limits of expression against other rights. For example, public figures must have broader margins of tolerance for criticism than private individuals. They are also entitled to more limited remedies, given their ability to intervene and influence public debate through other means. That said, this distinction could also be relevant to this issue. But how? Radically opposing positions emerged in the debate: on one hand, there were those who advocated giving public figures priority in appeals due to the size of their audience and the potential harm of being silenced. On the other hand, there were also those who proposed that, given that public figures typically have other means to disseminate their message, priority should be given to appeals from private individuals who do not have the same resources. In the public interest: Perhaps the distinction regarding the public interest is more useful for informing prioritization decisions. If the conversation or content is in the public interest, it should be prioritized. Such determinations, however, will likely arise from the intersection of two or more of the criteria already mentioned: local/regional context; the individual’s status (public or private figure); and the relevance of the subject matter. One of the unanimous conclusions among colleagues is that prioritization should likely be based not on a single criterion but on a variety of factors. How to implement this will be another matter… Regardless of the decisions companies make on this issue, the exercise served to highlight how little we know about this topic, the lack of information in reports and in the policies themselves, and the need to study the issue more deeply. Agustina Del Campo PhotoCredit: @morningbrew