Too Much Too Early? Amnesty International Calls for Generative AI Ban

A recent report by Amnesty International has stirred significant debate in the artificial intelligence community, asserting that Generative AI systems are "Unlawful by Design" and should be banned. The human rights watchdog has outlined five key dimensions of harm that underpin its call for prohibition, emphasizing the urgent need for regulatory action from nation-states. Firstly, Amnesty points to widespread privacy violations, noting that training datasets for large language models and other generative AI are often scraped indiscriminately from the internet. This practice frequently includes personal data and images, even those created by minors, which Amnesty argues violates international human rights standards requiring data processing to be lawful, necessary, and proportionate. Secondly, the report highlights the issue of discriminatory outputs. By training on vast, web-sourced data, generative AI models inevitably "launder and disseminate" existing cultural, gender, and racial biases at scale, perpetuating and amplifying societal inequalities. A third concern raised is the threat to freedom of thought. Amnesty contends that AI systems capable of influencing user thoughts and shaping personal beliefs effectively commercialize cognitive manipulation, posing a risk to individual autonomy. Environmental costs form the fourth dimension of harm. The immense computational power required to train and run generative AI models leads to a massive carbon footprint. For instance, Google's emissions have reportedly spiked by 48% since 2019, partly due to data center operations. These operations also consume vast amounts of water for cooling, straining resources in drought-prone regions. Finally, the report addresses supply chain and application issues. The development of these models often relies on low-paid, psychologically taxing content moderation labor. Moreover, the end products can be deployed in questionable applications, such as military uses, raising further ethical alarms. Based on these findings, Amnesty International is urging governments to outright ban generative AI systems and to hold technology companies responsible for any human rights abuses. As an alternative, the organization suggests focusing on Small Language Models (SLMs), which it believes can offer significant benefits with lower human rights risks and costs due to their specialized and curated nature. However, the article also presents a critique of this prohibitionist stance. One major concern is the "geopolitical blind spot" of such a ban. Critics argue that prohibiting generative AI development in democratic states would not halt its progress but merely shift its advancement to less accountable actors in jurisdictions with weaker human rights protections. Furthermore, requiring explicit consent for every piece of training data could ironically lead to more biased models, as datasets might then disproportionately represent privileged demographics. While acknowledging algorithmic bias as a critical threat, especially in education, the piece suggests that prohibition isn't the only solution, noting that AI companies are already working to mitigate biases and harmful outputs post-training.