我们最新的模型进展为使用COBOL、Fortran和Visual Basic等低资源编程语言构建和迁移软件带来了更强的能力。与Python和JavaScript等语言相比，这些语言可供模型学习的代码和文档较少，但它们支撑着全球关键的基础设施和基本服务。COBOL支持(在新窗口中打开)超过40%的网上银行系统、80%的面对面信用卡交易和95%的ATM交易，而Fortran则用于天气预报(在新窗口中打开)和核反应堆安全分析(在新窗口中打开)。提升AI在这些语言中的工作能力，可以帮助团队维护和现代化基本服务所依赖的软件。
这项工作以开发者和企业客户的反馈为指导，他们告诉我们，编码模型在处理这些语言时往往表现不佳。维护和现代化这类软件可能很困难：文档可能不完整，最初的开发者可能已经离开公司，重要的业务规则可能只存在于代码中。无论团队是在修复缺陷还是迁移到另一种语言，他们都需要保留系统所依赖的行为。
为了衡量进展，我们开发了两项评估。MultiLangBench测试模型能否在20种不同的编程语言中从零构建一个兼容SQLite的数据库引擎。我们的COBOL迁移评估测试模型能否将现有程序迁移到另一种语言，同时保留其行为。两者共同测试了两项互补的技能：构建新软件和保留现有软件的功能。
我们最新的模型在两项评估中都显示出大幅提升，以更低的成本实现了更高的准确率。
在MultiLangBench上，GPT‑6 Astra在20种编程语言中实现了87.4%的平均测试通过率，高于GPT‑5.6 Sol的26.8%，并且在所有测试语言中都有提升。GPT‑6 Luna也优于GPT‑5.6 Sol，达到40.3%，而估计API成本约低78%。
在我们的COBOL迁移评估中，GPT‑6.1 Sol在低推理努力下实现了86.6%的准确率，而GPT‑5.6 Sol仅为5.4%，同时成本低39%。在更高推理努力下，准确率达到97.9%，几乎使该评估饱和。
这些结果共同表明，我们的模型在处理多种编程语言方面正变得显著更好，同时也变得更加成本高效。
1. 从零构建SQLite（MultiLangBench）
MultiLangBench要求模型用指定的编程语言构建一个兼容SQLite的数据库引擎。模型必须组织代码库、管理状态、处理错误，并在保留现有功能的同时添加新功能。
为了比较不同语言的实现，我们使用数据库文件作为共享接口。候选程序接收一个初始数据库和一条SQL查询，然后生成一个修改后的数据库。我们使用参考SQLite实现来检查结果数据库是否具有预期内容。这使我们能够应用相同的测试，无论用于构建引擎的语言是什么。
我们将SQLite现有测试套件中的测试适配为这种格式。该评估衡量模型在长时间编码会话中能构建多少可工作的功能，报告测试通过率和API成本。
在最大推理努力下，GPT‑6 Astra在20种语言中实现了87.4%的平均测试通过率，高于GPT‑5.6 Sol的26.8%。改进扩展到所有测试语言，包括COBOL（10.9%到72.1%）、OCaml（25.6%到85.8%）、Fortran（21.4%到83.5%）和Visual Basic（27.5%到89.4%）。Astra在20种语言中的19种得分超过83%，其中COBOL仍是最难的，为72%。
GPT‑6 Luna提供了一种成本更低的选择，并且在所有测试语言中也优于GPT‑5.6 Sol。在最大推理努力下，它达到40.3%的平均测试通过率。提升包括COBOL（10.9%到17.1%）、OCaml（25.6%到34.1%）、Fortran（21.4%到31.4%）和Visual Basic（27.5%到41.6%）。这些改进的估计平均API成本为每个样本0.77美元，而GPT‑5.6 Sol为3.48美元，约低78%。
我们正在GitHub(在新窗口中打开)上发布MultiLangBench评估设置和测试套件，以及每种语言的结果。开发者可以检查评估测试的内容，查看他们所用语言的性能，并识别对其自身工作重要的差距。
2. 迁移COBOL代码库
我们的第二项评估要求模型将COBOL程序迁移到C++、Java、Kotlin、Python或Rust，同时保留每个程序的功能。
成功的迁移需要理解程序的业务逻辑和COBOL的特定语言行为。COBOL程序通常处理金融交易和业务记录，其中舍入规则和缺失记录处理等细节反映了业务需求。例如，COBOL的定点算术会丢弃多余的小数位，除非明确要求舍入。如果重写时对这些值进行舍入，可能运行无错误，但会产生不同的财务结果。
为了测试这些行为是否被保留，我们在隐藏测试上评估我们的模型，这些测试要求可观察输出逐字节一致。只有当所有测试通过时，迁移才被视为成功。像这样保留行为的能力对于维护和现代化旧系统的团队很有用，不过将其应用于生产软件仍需要专家审查，并针对系统的业务规则和依赖关系进行测试。
在这项评估中，GPT‑6.1 Sol在低推理努力下实现了86.6%的准确率，而GPT‑5.6 Sol在相同设置下为5.4%，且API成本仅为其61%。在更高推理努力下，准确率达到97.9%，使该评估接近饱和。这些评估结果展示了在为现代化业务所依赖软件的团队实现更准确、更经济的代码迁移方面取得的进展。
推进企业软件能力
这些评估，连同开发者和企业客户的直接反馈，帮助我们了解我们的模型在哪些方面正在改进，以及哪些方面仍有不足。
我们的目标是帮助模型将软件工程技能应用到更多语言中：理解不熟悉的代码库、规划变更、使用构建工具、运行测试和调试故障。
您可以通过 API 在团队所用语言的编码任务中试用我们的最新模型。您对哪些方面有效、模型在哪些方面存在困难，以及我们接下来应关注哪些语言或工作流程的反馈，有助于塑造我们训练和评估模型的方式，从而更好地支持您所依赖的软件。
Our latest model advances bring stronger capabilities for building and migrating software in low-resource programming languages such as COBOL, Fortran, and Visual Basic. These languages have less code and documentation available for models to learn from than languages like Python and JavaScript, yet support critical infrastructure and essential services around the world. COBOL supports(opens in a new window) more than 40% of online banking systems, 80% of in-person credit card transactions and 95% of ATM transactions, while Fortran is used in weather forecasting(opens in a new window) and nuclear reactor safety analysis(opens in a new window). Improving AI’s ability to work in these languages could help teams maintain and modernize the software that essential services depend on.
This work is guided by feedback from developers and enterprise customers who have told us that coding models often struggle with these languages. Maintaining and modernizing this software can be difficult: documentation may be incomplete, original developers may have left the company, and important business rules may only exist in code. Whether teams are fixing bugs or migrating to another language, they need to preserve the behavior their systems depend on.
To measure progress, we developed two evaluations. MultiLangBench tests whether a model can build a SQLite-compatible database engine from scratch across 20 different programming languages. Our COBOL migration evaluation tests whether a model can migrate existing programs to another language while preserving their behavior. Together, they test two complementary skills: building new software and preserving what existing software does.
Our latest models show substantial improvements across both evaluations, delivering much higher accuracy at lower cost.
On MultiLangBench, GPT‑6 Astra achieves an average test pass rate of 87.4% across 20 programming languages, up from 26.8% for GPT‑5.6 Sol, with improvements in every language tested. GPT‑6 Luna also outperforms GPT‑5.6 Sol, reaching 40.3% at approximately 78% lower estimated API cost.
On our COBOL migration evaluation, GPT‑6.1 Sol achieves 86.6% accuracy at low reasoning effort, compared with just 5.4% for GPT‑5.6 Sol, while costing 39% less. At higher reasoning effort, accuracy reaches 97.9%, nearly saturating the evaluation.
Together, these results show that our models are becoming significantly better at working with a wide range of programming languages, while also becoming more cost-efficient.
1. Building SQLite from scratch (MultiLangBench)
MultiLangBench asks a model to build a SQLite-compatible database engine in a specified programming language. The model must organize a codebase, manage state, handle errors, and add new features while preserving existing functionality.
To compare implementations across languages, we use database files as a shared interface. The candidate program receives an initial database and a SQL query, then produces a modified database. We use the reference SQLite implementation to check whether the resulting database has the expected contents. This lets us apply the same tests regardless of the language used to build the engine.
We adapted tests from SQLite’s existing test suite to this format. The evaluation measures how much working functionality a model can build over an extended coding session, reporting both test pass rate and API cost.
At maximum reasoning effort, GPT‑6 Astra achieves an average test pass rate of 87.4% across the 20 languages, up from 26.8% for GPT‑5.6 Sol. Improvements extend to every language tested, including COBOL (10.9% to 72.1%), OCaml (25.6% to 85.8%), Fortran (21.4% to 83.5%), and Visual Basic (27.5% to 89.4%). Astra scores above 83% in 19 of the 20 languages, with COBOL remaining the hardest at 72%.
GPT‑6 Luna offers a lower-cost option that also improves on GPT‑5.6 Sol in every language tested. At maximum reasoning effort, it reaches an average test pass rate of 40.3%. Gains include COBOL (10.9% to 17.1%), OCaml (25.6% to 34.1%), Fortran (21.4% to 31.4%), and Visual Basic (27.5% to 41.6%). These improvements come at an estimated average API cost of $0.77 per sample, compared with $3.48 for GPT‑5.6 Sol, approximately 78% lower.
We are releasing the MultiLangBench evaluation setup and test suite on GitHub(opens in a new window), along with results for each language. Developers can inspect what the evaluation tests, review performance in the languages they use, and identify gaps that matter for their own work.
2. Migrating a COBOL codebase
Our second evaluation asks models to migrate COBOL programs to C++, Java, Kotlin, Python, or Rust while preserving what each program does.
Successful migration requires understanding both the program’s business logic and COBOL’s specific language behavior. COBOL programs often process financial transactions and business records, where details such as rounding rules and missing-record handling reflect business requirements. For example, COBOL’s fixed-point arithmetic discards excess decimal digits unless rounding is explicitly requested. A rewrite that rounds those values instead can run without errors while producing different financial results.
To test whether these behaviors are preserved, we evaluate our models on hidden tests that require byte-for-byte parity on observable outputs. A migration is considered successful only if all tests pass. Behavior-preserving capabilities like these are useful for teams maintaining and modernizing older systems, though applying it to production software still requires expert review and testing against the system’s business rules and dependencies.
On this evaluation, GPT‑6.1 Sol achieves 86.6% accuracy at low reasoning effort, compared with GPT‑5.6 Sol’s 5.4% at the same setting at just 61% of the API cost. At higher reasoning effort, accuracy reaches 97.9%, bringing the evaluation close to saturation. These evaluation results demonstrate progress toward more accurate, affordable code migration for teams modernizing the software their businesses depend on.
Advancing capabilities for enterprise software
These evaluations, alongside direct feedback from developers and enterprise customers, help us see where our models are improving and where they still fall short.
Our goal is to help models apply software engineering skills across more languages: understanding unfamiliar repositories, planning changes, using build tools, running tests, and debugging failures.
You can try our latest models through the API on coding tasks in the languages your team uses. Your feedback on what works, where the models struggle, and which languages or workflows we should focus on next helps shape how we train and evaluate models to better support the software you depend on.
本文内容采集自官方网站，排版和翻译可能与原页面存在差异。
阅读官方全文