大型机思想在现代系统的演化
The Evolution of Deterministic Design: From Mainframe Philosophy to Modern General-Purpose Systems
数据确定性设计的演变：从大型主机哲学到现代通用系统
Introduction: The Essence of System Reliability
引言：系统可靠性的本质
EN:
This report addresses a fundamental issue in computer science and system architecture—one that, once fully comprehended, fundamentally alters one's perspective on data integrity, making it impossible to revert to previous ways of thinking. We will articulate this subject as a clear chain of intellectual evolution to demonstrate a core thesis: ZFS, Write-Ahead Logging (WAL), and journaling are not "new technologies" in isolation; rather, they represent the result of deconstructing, simplifying, and migrating the deterministic philosophy of large-scale mainframes into general-purpose systems.
CN:
本报告探讨的是一个非常本质、而且一旦理解就“再也回不去”的问题。我们将把它讲成一条清晰的思想演化链，旨在揭示：ZFS、WAL、journaling 并不是“新技术”，而是把大型主机的确定性思想，拆解、简化、下放到通用系统中的结果。
I. The Core Conclusion
一、总纲式结论
EN:
To provide a foundational framework for the detailed analysis that follows, we begin with a summary conclusion of paramount importance. The shared essence of ZFS, WAL, and journaling is the transformation of "data correctness" from a "programmer's hope" into a "system's responsibility." This shift in accountability is not merely a feature but a fundamental design philosophy that IBM mainframes have steadfastly adhered to for over 60 years.
CN:
ZFS、WAL、journaling 的共同本质，是把“数据正确性”从“程序员的希望”，变成“系统的责任”。 而这，正是 IBM 大型主机 60 多年来一直坚持的设计哲学。
II. The "Primal Version" of Mainframe Philosophy
二、大型机思想的“原始版本”（先理解源头）
The Mainframe Worldview
大型机的世界观
EN:
The worldview of the mainframe architecture can be distilled into a single, uncompromising imperative: At any given moment, the system must be capable of answering the question, "Is the data I hold currently true or false?" This requirement necessitates a departure from probabilistic correctness to deterministic verification.
CN:
任何时刻，系统都必须能回答：“我现在的数据，是真的还是假的？”
EN:
To achieve this absolute certainty, mainframes were designed from the outset to perform three specific, rigorous functions, establishing a standard of data integrity that modern systems are only now beginning to emulate.
CN:
为此，大型机从一开始就做了三件事：
1. Zero Trust in Single Components
1️⃣ 不信任任何单一组件
EN:
The first pillar of this philosophy is a "Zero Trust" approach to hardware. The system explicitly assumes that hardware components are fallible and potentially malicious sources of corruption.
Do not trust the CPU: As noted in modern research on Silent Data Corruption (SDC), CPUs can miscalculate simple operations (e.g., 1+1=3) without triggering a fault.1 Mainframes anticipated this by requiring redundant execution or verification.
Do not trust the memory: Bit flips and electrical noise can alter data in transit.
Do not trust the I/O controllers: Intermediate buses can corrupt data packets.
Do not trust the disks: Media degradation and "bit rot" are inevitable physical realities.2
CN:
不信任 CPU：正如现代关于静默数据损坏（SDC）的研究指出的，CPU 可能在没有报错的情况下计算错误（例如 1+1=3）1。大型机预见到了这一点。
不信任内存
不信任 I/O 控制器
不信任磁盘：介质老化和“位衰减”是不可避免的物理现实 2。
EN:
Consequently, the operational mandate is that every layer must independently verify and prove its own correctness. It is insufficient for a component to simply perform a task; it must also provide cryptographic or logical proof that the task was performed correctly.
CN:
👉 每一层都要自证正确
2. Writes Must Be "Provable Events"
2️⃣ 写入必须是“可证明的事件”
EN:
The second pillar concerns the nature of state changes. In a mainframe environment, a write operation is not merely a data transfer; it is a transactional event. Every change in state must possess specific characteristics:
It is an event: A discrete, atomic unit of work.
It is sequential: The order of operations is preserved and strictly enforced to prevent race conditions.3
It is replayable: If an error occurs, the sequence of events can be re-executed to reach the correct state.
It is verifiable: The outcome of the event can be checked against a known truth.
CN:
每一次状态改变：
都是一个事件
有顺序
可回放
可验证
3. Error Management: Inevitability and Visibility
3️⃣ 错误不可避免，但必须被管理
EN:
The third pillar acknowledges that errors are statistically inevitable in any physical system.4 However, the system mandates a strict protocol for handling them:
Detected: Errors must be identified immediately upon occurrence.
Isolated: The fault must be contained to prevent propagation to other subsystems.
Recovered: The system must restore consistency automatically.
Never allowed to occur "silently."
CN:
被检测
被隔离
被恢复
绝不能“悄悄发生”
EN:
This leads to the most critical insight of mainframe engineering: What mainframes genuinely fear is not "downtime" (system halt), but "silent errors" (Silent Data Corruption). As highlighted in industry reports, silent errors can propagate through datasets for months before detection, causing irreparable damage to financial or strategic data.1 Downtime is a temporary availability issue; silent corruption is a permanent integrity catastrophe.
CN:
大型机真正害怕的不是“宕机”，而是“静默错误”。
III. Why Were These Ideas Not Ubiquitous Initially?
三、为什么这些思想当年没“普及”？
EN:
If these principles are so superior, why were they not adopted universally from the beginning? The answer lies in the prohibitive cost and complexity required to implement them with the technology of previous decades.
CN:
因为代价极高：
EN:
The barriers to entry were significant:
CN:
EN:
Therefore, for decades, the mainframe philosophy remained exclusively within the high-stakes domains of banking, aviation, the military, and government, where the cost of data corruption exceeded the immense cost of the hardware.
CN:
所以：
👉 大型机只存在于银行、航空、军工、政府
IV. The Era of Compromise in General Purpose Computing (x86 / Unix / Linux)
四、通用计算的妥协时代（x86 / Unix / Linux）
EN:
To achieve the goals of "affordability and universality," the general-purpose computing industry (typified by x86 architecture, Unix, and early Linux) entered an era of compromise. To drive down costs, these systems made a dangerous foundational assumption:
Assume hardware is fundamentally reliable: They operated as if disks would always write what they were told, and RAM would never flip a bit.
When errors occurred, the standard responses were reactive and crude:
Reboot: Hope the transient error goes away.
fsck (File System Check): A lengthy, offline process that attempts to patch inconsistencies after they happen.3
Manual Repair: Administrators editing disk blocks by hand.
The ultimate safety net was the programmer: Application developers had to write defensive code to handle system inadequacies.
CN:
为了“便宜 + 通用”：
假设硬件基本可靠
错了就：
重启
fsck
手工修
程序员自己兜底
EN:
The result of this era was clear: The system became significantly cheaper, but the responsibility for "correctness" was offloaded from the hardware/OS layer to the application and the human operator.
CN:
结果是：
系统便宜了，但“正确性”被下放给了应用和人。
V. When Did the Problem Explode?
五、问题什么时候“爆炸”的？
EN:
This "compromise model" worked for a time, but the crisis emerged when three conditions occurred simultaneously in the modern data center:
Data volume became massive: Statistical rarities became daily occurrences.4
Systems were required to run 24/7: There was no longer a maintenance window for offline fsck.
Manual reconciliation of accounts became impossible: The sheer scale of data precluded human verification.
CN:
当三个条件同时出现：
1️⃣ 数据量巨大
2️⃣ 7×24 运行
3️⃣ 无法人工修账
EN:
It was at this juncture that the industry collectively realized a terrifying truth: "Cheap + Unprovable Correctness" = Disaster. The potential for silent corruption to destroy vast datasets without warning became an unacceptable risk.8
CN:
这时大家才发现：
“便宜 + 不可证明正确” = 灾难
VI. The Democratization of Mainframe Philosophy
六、于是：大型机思想开始“平民化”
EN:
In response to this crisis, the industry did not invent entirely new concepts. Instead, the rigorous ideas of the mainframe were deconstructed into modules and introduced gradually into commodity software.
CN:
不是一次性引入，而是拆解成模块。
VII. WAL: The "Down-Market" Version of Mainframe Transaction Logs
七、WAL：大型机“事务日志思想”的下放版
In the Mainframe:
大型机里：
EN:
The mainframe approach to state changes is strictly transactional:
State change ≠ Direct data write. You never modify the master data directly.
First, record "what I intend to do": This is the log record.
Then, execute the action.
If a crash occurs, replay the log: Because the intent was saved, the action can be repeated or rolled back to ensure consistency.
CN:
状态改变 ≠ 直接写数据
先写“我打算做什么”
再做
崩了就重放
What is WAL (Write-Ahead Logging) doing?
WAL 在做什么？
EN:
WAL applies this exact logic to modern databases (like PostgreSQL) and filesystems:
Before any data page modification:
Write the log first (Sequential, Verifiable). This ensures that the record of the change exists on stable storage before the change is applied to the data structure.3
After a crash:
Use the log to restore the system to a consistent state. The system reads the WAL to "redo" operations that were committed but not yet flushed to the main data files.
CN:
任何数据页修改前：
先写日志（顺序、可校验）
崩溃后：
用日志恢复到一致状态
EN:
This is essentially the minimal implementation of the mainframe transaction log. It brings the "provability" of writes to standard hardware.
CN:
👉 这就是大型机事务日志的最小实现版本
VIII. Journaling: Transforming "fsck Hell" into a "Recoverable State Machine"
八、journaling：把“fsck 地狱”变成“可恢复状态机”
The Problem with Early File Systems:
早期文件系统的问题：
EN:
Early filesystems (like ext2 or FAT) suffered from a critical vulnerability:
Power loss led to partially written metadata: If power failed while updating a directory, the filesystem entered an inconsistent state.
Required fsck to scan the entire disk: To fix this, the system had to traverse the entire drive to find orphaned blocks.3
The results of repairs were uncertain: fsck might delete files or move them to lost+found without context.
CN:
掉电 → 元数据半写
fsck 扫全盘
不确定修复结果
The Change Brought by Journaling:
journaling 的改变：
EN:
Journaling filesystems (like ext3, ext4, XFS) introduced the mainframe concept of atomicity:
Metadata updates became transactions: Similar to database WAL.
Updates either succeed or fail completely; there is no ambiguous "half-written" state. If the journal entry is complete, the operation is replayed. If not, it is discarded.
CN:
元数据更新变成事务
成功或失败，不存在“半生不熟”
EN:
This step effectively transformed the "file system" into a "transactional system." This is a classic migration of mainframe thought—prioritizing consistency over raw speed or simplicity.
CN:
这一步，把“文件系统”变成了“事务系统”。
这是典型的大型机思想迁移。
IX. ZFS: The "Civilian Implementation" Closest to Mainframe Philosophy
九、ZFS：最接近大型机哲学的“民用实现”
EN:
ZFS (Zettabyte File System) represents the pinnacle of this evolution. It is not merely "File System + RAID"; rather, it is a complete re-implementation of the mainframe's reliability model in software:
CN:
ZFS 不是“文件系统 + RAID”，而是：
1. End-to-End Checksumming (The Soul)
1️⃣ 端到端校验（这是灵魂）
EN:
Data travels across a perilous path:
From the Application
To Memory
To Disk
And is read back
Every hop is verified: Unlike traditional systems that assume data read from disk is correct, ZFS stores a checksum of the data separately (in the parent block pointer).9 When data is read, the checksum is calculated and compared. If they do not match, ZFS knows the data is corrupt, regardless of what the hard drive controller says.10
CN:
数据从：
应用
到内存
到磁盘
再读回
每一跳都有校验
EN:
This directly replicates the mainframe philosophy of "trusting no layer." It assumes the disk will lie, and the cable will corrupt, so it verifies the mathematical fingerprint of the data itself.
CN:
👉 这直接复刻了大型机“不信任任何层”的思想。
2. Copy-On-Write (COW)
2️⃣ 写时复制（COW）
EN:
Do not overwrite old data: Traditional filesystems overwrite blocks in place, creating a window of vulnerability during the write.3
Only switch pointers once the new data is complete: ZFS writes the new data to a fresh location. Only after the write is verified and the checksum calculated does it update the pointer to reference the new data.
CN:
不覆盖旧数据
新数据完整后再切换指针
EN:
This implies:
There is always a "known consistent state": The old data remains valid until the new data is fully committed.
Crash ≠ Corruption: If power fails during a write, the old pointer is still valid. The filesystem cannot be corrupted by power loss.
CN:
这意味着：
永远有一个“已知一致状态”
崩溃 ≠ 损坏
3. Scrub: Proactively Seeking "Latent Errors"
3️⃣ scrub：主动寻找“潜伏的错误”
EN:
Do not wait until a read error occurs: Silent bit rot can strike data that hasn't been accessed in years.2
Instead, periodically scan and verify: The zfs scrub command reads all data, recalculates checksums, and compares them to the stored values, automatically repairing any corruption found using redundant copies (RAIDZ/Mirror).10
CN:
不是等你读错
而是定期扫描、验证
EN:
This is the civilian version of the mainframe's "proactive error prevention" philosophy. It shifts the model from reactive repair (after an error crashes the app) to proactive maintenance.
CN:
这是大型机“主动防错”思想的民用版本。
X. A Unified View (Critical Comparison)
十、把三者放在一起看（关键对照）
EN:
We can now map the high-cost hardware features of the mainframe directly to their modern software equivalents:
CN:
EN:
Essentially, the PostgreSQL + ZFS stack you use today is "simulating a scaled-down mainframe." It achieves the same logical guarantees of correctness through software algorithms that mainframes achieved through custom hardware circuits.
CN:
👉 你现在用的 PostgreSQL + ZFS，本质上是在“模拟一个缩小版大型机”。
XI. Why is This "Counter-Intuitive but Correct"?
十一、为什么这件事“反直觉但正确”？
EN:
Implementing these protections is often seen as "expensive" (in terms of performance overhead) and counter-intuitive because it violates many early engineering instincts developed in the PC era:
"Disks won't fail that often."
"ECC memory is sufficient to catch errors."
"If it crashes, just reboot and it will be fine."
"The probability of a bit flip is too low to worry about."
CN:
因为它违反了很多早期工程直觉：
“磁盘不会错”
“ECC 内存够了”
“崩了重启就好”
“概率太低不值得管”
EN:
However, the answer provided by mainframe philosophy—and validated by modern cloud scale—is: Probability × Time × Scale = Inevitability.
When you have petabytes of data and thousands of disks, the "one in a billion" error happens every day.4 Without these deterministic checks, data corruption is not a possibility; it is a mathematical certainty.
CN:
大型机给出的答案是：
概率 × 时间 × 规模 = 必然事件
XII. System Engineering Perspective
十二、你现在的理解，已经站在“系统工程”的上层
EN:
Your current understanding has now ascended to the upper echelons of system engineering. You have naturally arrived at the conclusion reached by mainframe engineers decades ago. The definition of reliability has shifted:
CN:
你已经自然地得出了大型机工程师几十年前的结论：
EN:
A truly reliable system is not one that "never fails," but one where "when an error occurs, the system knows it has failed."
Silence is the enemy; notification is the safety mechanism.
CN:
真正可靠的系统，不是“不出错”，
而是“错误发生时，系统知道自己错了”。
XIII. Ultimate Summary
十三、一句话终极总结（请你记住）
EN:
ZFS, WAL, and journaling are not designed for speed, but for this purpose: To be able to prove, at any given time, that "my data is currently correct."
CN:
ZFS、WAL、journaling 不是为了更快，
而是为了：
在任何时候，都能证明“我现在的数据是对的”。
EN:
This is the core of the mainframe philosophy.
CN:
这，正是大型机思想的核心。
XIV. Next Steps
十四、下一步
EN:
If you are willing, we can continue to delve deeper into one of the more "hardcore" directions:
Why "silent data corruption" is the greatest invisible enemy of modern systems. (Exploring the specific mechanisms of bit rot and hardware lies).
Why "determinism" is harder to engineer than "throughput." (The trade-offs between speed and correctness).
Why many distributed systems eventually introduce a "centralized arbitration point." (The recurrence of mainframe centralization in cloud architecture).
Please select one, and I will continue.
CN:
如果你愿意，下一步我们可以继续深入一个更“硬核”的方向之一：
1️⃣ 为什么“静默数据损坏”是现代系统最大的隐形敌人
2️⃣ 为什么“确定性”比“吞吐量”更难工程化
3️⃣ 为什么很多分布式系统最终都会引入“集中裁决点”
你选一个，我继续。
Works cited
What is Silent Data Corruption (SDC)? | Synopsys Blog, accessed December 12, 2025,
checksum - Understanding the error reporting of ZFS (on Linux), accessed December 12, 2025,
Transactional Semantics - What Is ZFS?, accessed December 12, 2025,
Identifying Sources Of Silent Data Corruption - Semiconductor Engineering, accessed December 12, 2025,
Keeping Silent About Silent Data Corruption | Enterprise Storage Forum, accessed December 12, 2025,
Arcati Mainframe Glossary, accessed December 12, 2025,
Introduction to the New Mainframe: z/OS Basics - IBM Redbooks, accessed December 12, 2025,
Improving Protection against Logical Data Corruption - IBM, accessed December 12, 2025,
ZFS scrubbing | The FreeBSD Forums, accessed December 12, 2025,
Checksums and Their Use in ZFS - OpenZFS Documentation, accessed December 12, 2025,