ChatGPT 是全球使用最广泛的 AI 产品之一，我们已投入资源构建业内一些最先进的系统，用于检测潜在危害、审查可疑活动并升级可信威胁。我们在此宣布两项举措，以推进该领域的工作：一份新的《社区安全保障报告》，以及对未成年人枪支相关内容的更多限制。
保障社区安全仍然是 OpenAI 的首要任务。我们的工作包括将迫在眉睫的暴力威胁提请执法部门注意，2026 年第三季度我们向执法部门转介了 200 多起案件。这些转介正在产生积极（在新窗口中打开）影响（在新窗口中打开），阻止了暴力阴谋，我们继续依赖心理健康和行为专家帮助我们评估疑难案件。
我们认识到，AI 公司与执法部门、家庭成员、心理健康专业人士和政策制定者一样，扮演着独特的角色，后者带来关键的权威、背景信息和现实世界的影响力。我们致力于充分发挥这一作用，持续加强我们的系统，分享我们所学到的经验，并直接与那些最有能力在现实世界中采取行动的人合作。
OpenAI 的社区安全报告
我们不断加强为帮助保护人员和社区所采取的措施。这意味着建立强有力的保障措施，同时也要透明地说明这些措施的效果以及我们正在如何改进它们。我们希望让人们清楚地了解我们的方法、我们的模型如何识别和应对有害行为、我们在实践中看到的风险，以及我们为应对这些风险所采取的步骤。
我们承诺发布《社区安全报告》，内容涉及暴力预防和其他社区安全问题，以及我们保障措施的有效性。该报告将解释我们如何训练系统来检测、调查和应对社区安全问题。我们将分享关于暴力相关危害如何表现的匿名示例以及我们正在学到的经验，重点介绍新的缓解措施和政策变化，并包含支持这些发现的相关数据。
我们将在 2027 年初发布第一份报告，涵盖 2026 年下半年，此后每六个月更新一次。我们的承诺是分享关于我们如何识别、审查和应对潜在有害内容的有意义的发现。
为所有用户设定更强的政策边界
我们的模型经过训练，不会提供有助于他人策划或实施暴力的指令。我们现在对 ChatGPT 如何回应有关过去大规模袭击及实施者的问题施加额外限制，指示模型在这些领域提供较少细节的信息，即使这些信息可从其他在线来源获得。人们应该能够要求 ChatGPT 帮助他们了解这些事件、受害者和社区如何受到影响，以及学习识别警告信号和预防未来暴力。但我们正在收紧政策，限制对过去大规模袭击的重构，包括关于所用武器和装备及其获取方式的细节，以及可能放大袭击者恶名的危害规模详细比较。这些限制适用于所有用户，无论年龄或提问原因。
回应警告信号
有时，一个单独看来普通的请求，结合早先的对话就会呈现出不同的含义。今年 5 月，我们分享了 ChatGPT 如何使用早期警告信号的简短事实摘要来在后续聊天中更谨慎地回应。我们正在此基础上继续推进，当近期聊天显示出可能的暴力警告信号时，ChatGPT 在分享可能被滥用的信息时将更加保守地回应。我们还在努力将那些更严格的限制应用于我们预测某人未满 18 岁的情况，即使他们的聊天没有显示暴力意图的迹象。
我们禁止我们认定违反暴力活动政策的账户。我们还正在与心理健康和暴力预防专家合作，开发额外的自动化方式以更早介入。当对话显示某人正痴迷于暴力或严重伤害他人的想法时，ChatGPT 将能够自动结束聊天，以当场打断这种模式。如果 ChatGPT 因此原因结束多个聊天，我们可能会暂时暂停该账户。这是一项持续的工作，我们继续加强和投资于技术和实际保护措施，包括如何可靠地识别警告信号以及确定何时以及如何介入。
为青少年提供更多保护
面向青少年的 ChatGPT为 18 岁以下用户应用适龄保障措施，提供更强的保护和额外限制，以减少接触敏感内容。我们现在对青少年可以接收的武器相关信息施加进一步限制，即使问题看似无害。
我们的政策已经禁止帮助任何人策划或实施暴力，并限制帮助青少年寻找或获取武器。我们正在扩展这些保护措施，限制可能被滥用以助长暴力的武器和安保系统的实用信息，即使没有暴力意图的迹象。青少年仍然可以使用 ChatGPT 帮助预防和缓和暴力、寻求支持，并与可信赖的成年人联系。
我们去年推出了年龄预测，它分析行为模式等多种信号来估计用户可能未满或已满 18 岁。我们加强了年龄预测系统，以便在人们使用 ChatGPT 时持续评估新信号，并在适当时将青少年转入 18 岁以下体验。被错误置于 18 岁以下体验的用户将始终有一种快速、保护隐私的方式，通过我们的年龄验证合作伙伴 Persona 确认他们是成年人并移除额外的青少年保护。
我们的家长控制功能允许家长将其ChatGPT账户与青少年的账户关联，并共同管理安全设置。关联账户包括安全提醒，当经过专门培训的审核人员发现严重的自残担忧以及与饮食失调可能相关的行为时，可能会发送这些提醒。今年7月，我们将这些提醒扩展至：如果青少年的账户因违反我们关于暴力的使用政策而被停用，则通知家长。我们正在努力引入通知功能，以便在我们因青少年反复讨论暴力而暂停其账户时通知家长，具体如上所述。
展望未来
现实世界中的暴力行为令人心碎且复杂，有许多促成因素。它们也严峻地提醒我们，必须继续设计和迭代我们的系统，以应对人类行为固有的复杂现实。在AI系统中开发有效的暴力防护措施，需要解决识别风险和适当应对方面的新问题，我们将继续投入、迭代并分享关于如何最好地实施我们所追求的保护措施的经验。此处详述的防护措施将在未来几周内开始推出，我们将继续完善它们。我们仍然致力于减轻ChatGPT中的伤害风险，加强我们的分层保护，监测滥用行为，执行我们的规则，并在情况需要时引入现实世界的支持与干预。
ChatGPT is one of the world’s most widely used AI products, and we have invested in building some of the industry’s most advanced systems for detecting potential harm, reviewing concerning activity, and escalating credible threats. We’re making two announcements to further our work in this area: a new Community Safeguards Report and more restrictions around firearms content for minors.
Keeping communities safe continues to be a top priority for OpenAI. Our work includes bringing imminent threats of violence to the attention of law enforcement, and we made over 200 law enforcement referrals in Q3 2026.These referrals are making apositive(opens in a new window)difference(opens in a new window) in disrupting violent plots, and we continue to rely on mental health and behavioral experts to help us assess difficult cases.
We recognize AI companies have a distinct role to play alongside law enforcement, family members, mental-health professionals, and policymakers, who bring critical authority, context and real-world reach. We are committed to making the most of that role by continuously strengthening our systems, sharing what we learn, and working directly with those who are best positioned to act in the real world.
OpenAI’s Community Safety Report
We are constantly strengthening the steps we take to help protect people and communities. That means building strong safeguards, and also being transparent about how well they work and what we’re doing to enhance them. We want to give people a clear picture of our approach, how our models identify and respond to harmful behavior, the risks we see in practice, and the steps we take to address them.
We’re committing to launch the Community Safety Report on violence prevention and other community safety issues, and the effectiveness of our safeguards. The report will explain how we train our systems to detect, investigate, and respond to community safety issues. We’ll share anonymized examples of how violence-related harm manifests and what we’re learning, highlight new mitigations and policy changes, and include relevant data to support these findings.
We will publish our first report in early 2027, covering the second half of 2026, with updates every six months thereafter. Our commitment is to share meaningful findings about how we identify, review, and respond to potentially harmful content.
Stronger policy boundaries for all users
Our models are trained to not give instructions that would help someone plan or carry out violence. We’re now placing additional restrictions on how ChatGPT responds to questions about past mass attacks and the people who carried them out, directing the model to provide less detailed information in those areas even if it is available from other online sources. People should be able to ask ChatGPT to help them understand these events, how victims and communities were impacted, and learn about recognizing warning signs and preventing future violence. But we’re tightening our policies to restrict reconstructions of past mass attacks, including details about the weapons and equipment used and how they were obtained, and detailed comparisons on scale of harm that could amplify an attacker’s notoriety. These limits apply regardless of a user’s age or reason for asking.
Responding to warning signs
Sometimes a request that seems ordinary on its own takes on a different meaning in light of an earlier conversation. In May, we shared how ChatGPT can use short, factual summaries of earlier warning signs to respond more carefully in a later chat. We’re building on that work so that when recent chats show warning signs of possible violence, ChatGPT will respond more conservatively when sharing information that could be misused. We’re also working to apply those stricter limits when we predict someone is under 18, even if their chats show no signs of violent intent.
We ban accounts that we determine have violated our policies on violent activity. We’re also working with mental health and violence prevention experts on additional, automated ways to intervene earlier. When a conversation shows signs that someone is fixating on violence or the idea of seriously harming another person, ChatGPT will be able to automatically end the chat to interrupt that pattern in the moment. If ChatGPT ends multiple chats for that reason, we may temporarily pause the account. This is an ongoing effort and we continue to strengthen and invest in technical and practical protections, including how to recognize warning signs reliably and determine when and how to intervene.
More protections for teens
ChatGPT for Teens applies age-appropriate safeguards for users under 18, with stronger protections and additional limits to reduce exposure to sensitive content. We’re now placing further limits on weapons-related information teens can receive, even when a question appears harmless.
Our policies already prohibit helping anyone plan or carry out violence and restrict helping teens find or obtain weapons. We’re extending those protections to restrict practical information about weapons and security systems that could be misused to facilitate violence, even when there’s no sign of violent intent. Teens can still use ChatGPT to help prevent and de-escalate violence, seek support, and connect with a trusted adult.
We launched age prediction last year, which analyzes multiple signals such as behavior patterns to estimate whether a user may be under or over 18. We strengthened our age prediction system to continually assess new signals as people use ChatGPT and move teens into the under 18 experience when appropriate. Users who are incorrectly placed in the under 18 experience will always have a fast, privacy-protective way to confirm they are adults and have additional teen protections removed through Persona, our age verification partner.
Our parental controls let parents link their ChatGPT accounts to their teens and manage safety settings together. Linked accounts include safety alerts that may be sent when specially trained reviewers identify a serious self-harm concern and behaviors that may be associated with disordered eating. In July, we expanded those alerts to notify parents if their teen’s account has been deactivated for violating our usage policies on violence. We are working to introduce notifications for parents if we pause their teen’s account for repeatedly discussing violence, as detailed above.
Looking ahead
Acts of real-world violence are heartbreaking and complex, with many contributing factors. They are also a stark reminder that we must continue designing and iterating our systems to account for the inherently complicated realities of human behavior. Developing effective violence safeguards in AI systems requires solving novel problems in recognizing risk and responding appropriately, and we will continue to invest in, iterate, and share learnings on how best to implement the protections we’re pursuing. The safeguards detailed here will begin rolling out in the next few weeks and we will continue to refine them. We remain committed to mitigating risks of harm in ChatGPT, strengthening our layered protections, monitoring for misuse, enforcing our rules, and involving real-world support and interventions when situations call for it.
本文内容采集自官方网站，排版和翻译可能与原页面存在差异。
阅读官方全文