On decentralized clocks: How time became the biggest security threat on blockchain systems
The biggest use case of public blockchains, such as Ethereum, as of now is decentralized finance (DeFi) — being able to exchange, borrow, trade, and earn interest on assets without going through a centralized entity. While the concept of DeFi existed ever since Smart Contracts were a thing, it became a dominating trend within the blockchain industry after the introduction of yield farming and yield aggregators. They promise much higher returns even compared with DeFi leaders Uniswap and Compound, with their most common mechanism involving issuance of new tokens to liquidity providers. Yield aggregators simplify this process, abstracting manual yield farming processes into vaults that users can deposit into, and earn interest. (Shoutout to Andre Cronje for sharing this concept back in 2018 during a Telegram chat 😉)
A key factor behind today’s massive DeFi boom is a concept called composable finance. When a new DeFi protocol is released, most of its source code is also made available to the public. This means that Smart Contract source code from one or more DeFi protocols can be easily composed together to create a new one, like a building block — significantly lowering the barriers required for bringing an idea to a working product.
A major problem with this approach, however, is security.
Smart Contracts are immutable by design, just like anything else posted on a blockchain. They are also incredibly difficult to debug, with them essentially being black boxes modifying state per execution and not emitting helpful logs unless the developer explicitly programs it to do so. The most popular Smart Contract runtime out there, the EVM, barely guarantees state, memory, and runtime-level safety on its own, being required to use external libraries just to prevent integer underflows on calculation or reentrancy attacks. Most Smart Contract audits don’t help — they are more of a “seal of approval” against known forms of attacks only, often involving automated static analysis tools, rather than discovering system-specific flaws that are highly likely to be present in this particular contract. A significant portion of “composed” DeFi protocols (named after food) don’t even include such basic audits at all, being a “use-at-your-own-risk” system in the first place.
All of those factors combined makes on-chain Smart Contracts highly susceptible to novel forms of attacks, most of them previously unknown. While developers are taking precautionary measures against common forms of DeFi attacks due to the increasing number of security breaches, a recent attack on Pickle Finance has demonstrated to the world that: the age of simple flash loans, data corruption and reentrancy-based attacks are now long gone. No Smart Contract system can be perfectly secure against all forms of possible attacks.
Just a few days later, an oracle attack hit Compound, falsely liquidating more than $100M worth of loans. It is believed that the attacker has exploited Compound’s Open Price Feed system by manipulating DAI prices on Coinbase Pro, leading DAI to be off-peg by more than 30%. In a series of events that followed, a malicious attacker has used Uniswap Flash Swaps to borrow, deposit, exchange, withdraw and pay back DAI and cDAI to gain a significant margin, only by calling a single function. The remaining cDAI this liquidation bot has gained was worth more than $3.5M.
Even though the above two attacks were already pretty significant, such attacks on DeFi smart contracts will continue for quite some time at a much larger scale. And it won’t be the developer’s fault for failing to block all possible attack factors. The key reason why attacks on DeFi systems are so difficult to stop is because blockchains themselves were designed to do a bad job on timekeeping — and it’s not just Ethereum.
The Timekeeping Dilemma
Time is an essential concept to any modern computer program. Without the concept of time, it’s impossible to access any TLS-enabled website, create entropy for certain algorithms, exchange secrets, or authenticate Windows licenses. We take timekeeping on computers for granted, but exchanging and keeping track of time is an incredibly difficult problem to solve.
Time is essentially a representation of the universe’s current global state. This means that for any given moment, every single entity must share the exact same value. This is obviously a huge problem for computers: let’s assume there is a huge, highly accurate atomic clock in the middle of the universe, transmitting time data across the Internet. One small problem: transmitting data takes time by itself. When a device receives time data over the Internet and applies it to its own systems, the universe’s global state (time) would have already changed. In other words, it is impossible for a networking device to be perfectly up-to-date with the actual global time value.
Another problem with networked timekeeping is that one cannot perfectly predict time required for data transmission — network conditions constantly change, and no one can guarantee perfect reliability out in the wild. Combined with the first problem, this means that a receiving party also cannot perfectly replicate time value as of its origin, leading to inconsistencies.
The Network Time Protocol, or NTP, solves this problem through a hierarchy of timekeeping servers distributed across the globe (up to 15 Stratums) with their routing paths optimized as much as possible, by constructing a Bellman-Ford shortest-path spanning tree (which reduces both latency and transfer time inconsistencies). Simply put, the NTP time syncing algorithm is defined as:
- A client periodically pings (or polls) remote NTP servers to calculate: time offset θ, and network RTT δ. θ is calculated as
and δ is calculated as
where definitions are: t0 = client timestamp of request packet transmission, t1 = server timestamp of request packet reception, t3 = server timestamp of response packet transmission, and t4 = client timestamp of response timestamp reception.
2. It is trivial that
and
which, solving for θ, results in the current time offset.
3. This is done against multiple time servers, with statistical outliers removed, which then an optimal value of θ is used to continuously adjust the local clock frequency.
This works perfectly fine for personal computers and centralized services, especially with applications that relies on accurate timing — such as encryption. Even though timestamps derived through NTP are mere estimates, they are accurate and mature enough for time-critical applications to rely on.
But what about distributed systems? How does blockchain solve the same problem, especially with time-critical contract code?
Timekeeping on blockchains: Too Loose
Bitcoin requires the concept of timestamps because it is required for its Proof-of-Work consensus mechanism: without a valid timestamp, the network cannot verify whether a particular transaction being mined is trying to revert a previous one. However, the definition of “timestamp” on blockchains is way too loose for contracts to rely on, even though it might have been enough for consensus purposes only.
Bitcoin uses the UNIX timestamp denomination for inclusion within a block. First of all, the UNIX timestamp isn’t really a valid representation of time, even though it is more than enough for command-line applications built back in the day — it doesn’t have the concept of leap seconds, and every day is equivalent to 86400 seconds (which shouldn’t be). This is why UNIX timestamps are not considered to be direct replacements of UTC, but rather a programmatic denomination that simplifies representation of time.
But blocktime under Bitcoin isn’t even an accurate representation of a UNIX timestamp. This is because Bitcoin only uses timestamps as a secondary source to keep PoW secure, not as an actual tool to measure time on-chain:
A timestamp is accepted as valid if it is greater than the median timestamp of previous 11 blocks, and less than the network-adjusted time + 2 hours. “Network-adjusted time” is the median of the timestamps returned by all nodes connected to you. As a result block timestamps are not exactly accurate, and they do not need to be. Block times are accurate only to within an hour or two.
Whenever a node connects to another node, it gets a UTC timestamp from it, and stores its offset from node-local UTC. The network-adjusted time is then the node-local UTC plus the median offset from all connected nodes. Network time is never adjusted more than 70 minutes from local system time, however.
…thus, any on-chain timestamp is considered “valid” if it’s:
- larger than the median timestamp of the previous 11 blocks, and
- smaller than the median timestamp returned by connected nodes, plus 2 hours, but never exceeds more than 70 minutes of local system time
which can have an error margin of more than 1 ~ 2 hours. This is not an accurate clock at all. Bitcoin’s time syncing system is pretty much naive, with just enough accuracy to guarantee security of the blockchain (as an early contributor, I can say with confidence that the whole system is designed this way 😅)
Ethereum is not much different: the only party determining on-chain timestamps (block.timestamp or now under Solidity) are miners. Any miner can submit any arbitrary timestamp, and consensus will technically not block or check against them. The only factors blocking miners from submitting insensible values as a timestamp are:
- technically no one can build on a block from the future, because blocks should be in chronological order. thus, assuming there are more honest miners submitting true timestamp values than malicious ones, a block stamped with an insensible blocktime will not gain enough hash power to take over the blockchain.
- current blocktime cannot be smaller than the previous block.
- mining difficulty is lowest if blocks are stamped with a blocktime that is closest to an actual transaction execution event.
Thus, it is simply against basic network incentives to submit bad timestamps. There’s nothing else stopping miners from submitting insensible blocktime other than consensus-level cryptography — which means that, if a significant portion of miners have a valid incentive to submit an invalid timestamp, they can choose to do so, at any time.
Polkadot and Substrate also don’t enforce timestamp integrity, but only “agree whether blocktime is within a certain delta of local system time”. Tendermint Core is perhaps the only consensus algorithm that actually performs consensus-level votes on timestamps — which is why Tendermint and Cosmos SDK-based chains tend to have a much more robust concept of time compared to others (like being able to match blockheight and its corresponding UTC time with relatively reasonable accuracy).
Simply put, the concept of timestamps on blockchains are bad oracle implementations that just happen to: (i) be relied on by consensus, and (ii) be relied on by millions of contracts to execute important financial transactions. (Chainlink and Band fans rejoice? lol 😅)
But the real problem isn’t just with the accuracy of clocks on blockchains in general: it gets much, much worse. The real problem with blocktime is that — time doesn’t tick within a block.
Time doesn’t tick within a block
Yes, let me say that for the third time: time doesn’t tick within a block. This is important, because this fact alone contributes to a significant portion of DeFi contract attacks that has happened in the past, and will happen in the future.
Checking whether time has passed is important for two primary reasons: for one, in order to be perfectly confident on a state change commitment, a previous transaction that this transaction is building upon should ideally have a smaller blocktime than the current block. Another is that, if we allow multiple operations (or function calls) to happen at the same timestamp, it will mess up our time calculation logic, especially with interest calculation and withdrawals.
On traditional computer systems, time is always continuous. Because of this assumption, we can be confident that one operation that has happened in the past is always finalized, and shouldn’t affect whatever operation I am trying to do right now or in the future (unless we’re defining an asynchronous operation). Unfortunately, under a distributed system this is no longer the case, because every single state-related operation is finalized in batches — in blocks.
As every single operation that occurs within the same block has the same blocktime, it is theoretically possible to do operations that simply were not possible on a typical computer system:
- an external entity taking over control flow within the same block, potentially recursively calling functions to drain funds before the call itself is finalized, if the contract is not properly configured to protect against such attacks (same-block reentrancy)
- borrowing some ether or tokens from a contract, depositing it into another contract, earn rewards or interest from that contract using the borrowed funds, withdraw both the rewards & borrowed assets, and repay the borrowed assets back to the contract, all within the same block — such that the loan itself is never technically finalized, but is still effective only within this block to create a permanent state change that lasts beyond this block (flash loans)
- temporarily impacting oracle prices by buying or selling a significant amount of assets, call an external contract function that depends on that oracle pricing, create a state change within that external contract, and return oracle prices back to normal by repaying bought or sold assets before anything is actually committed (flash swaps)
- anything else that can create a “temporary” state change that can lead to a permanent one — from simple contract calls to assembly memaddr swaps
The sole fact that time doesn’t tick within a block creates this many problems. It’s like a Matrix time freeze — where someone can pause time, only change something that they want, put back everything else, and then simply resume. Except this time, you don’t need superpowers to pause time: the blockchain pauses time for you every block, such that anyone with the right skillset can do anything within that frozen timeframe.
Pretty cool, yet very dangerous. Literally the opposite of a ticking timebomb (pun intended 😉).
Alright, what can we do then?
There are two approaches: short-term and long-term.
The short term approach is to properly guard all external contract entry points against batched transactions, by allowing only one function call to be included in one transaction. This ensures that blocktime always passes after one operation is finalized. While there may be multiple approaches to such a guardian library, some aspects of a contract entry point may not be able to be properly guarded:
- if a contract must allow multiple tx.origin calls at the same time and cannot enforce such restrictions, contract-level workarounds may exist (involving delegatecalls and/or recursive calls).
- checking contract method signatures per transaction and storing this as a mapping is also a supplementary option, while this may still be circumvented with more sophisticated attacks that involve calls to multiple contract systems at the same time.
- using external bots to submit transactions, instead of using contracts to batch everything into one transaction — this is the hard part. as long as an attacker has enough commitment to build an attacker bot and a contract to supply transactions, with the correct nonce and gas pricing, in the correct order any guardian method may be able to be circumvented — as long as this “time pause” trick exists, and access to the mempool by miners is guaranteed.
- we can make things much harder to exploit (mitigation is possible), but cannot completely block such attacks (prevention is difficult).
The fundamental, long term solution would be to: (i) make time tick at all times, (ii) introduce a truly “decentralized clock” that is much more reliable, and (iii) explicitly disallow state commitments through an uncommitted transaction. More specifically,
- assign a minimum “expected execution period” per opcode even when execution is not finalized. this enforces time linearity even when the entire execution flow stays within the same block.
- for this, timekeeping must be way more accurate than what blockchains provide as of now — more specifically, timekeeping processes must be separated from consensus such that block finality doesn’t affect time linearity during execution. since time on blockchains are essentially oracles, time estimation could be drastically improved with an NTP-like timestamp estimation algorithm and a separate oracle voter.
- state commitments must be disallowed when an operation is either still uncommitted, or expected execution time since block genesis is less than current elapsed blocktime. reentrant attacks are blocked on runtimes like CosmWasm through an actor model (which essentially isolates internal state from external entities and only passes messages), but there is still room for improvement.
We still have a long way to go
The world has changed a lot over the last 12 years — a whitepaper transforming into a full-blown industry. While there has been tons of research around improving scalability and usability of blockchains, recent smart contract exploit cases are continuing to show that there are still a lot to uncover in order for this industry as a whole to fully stabilize.
While DeFi as a concept is still very exiting and is a clear, better alternative to legacy finance systems, getting the same level of trust takes time — and it is time for crypto enthusiasts and entrepreneurs to recognize: blockchains are still very much experimental. Pushing experimental technologies to their limits do foster innovation, but do result in unintentional damages — thus further research around security and user protection is crucial, and they must continue even without the DeFi bubble dominating the industry today.
Credits
Special thanks to:
- Byeongsoo Hong (frostornge), who came up with an Ethereum smart contract guardian library that explicitly disallows batched transactions through an attacker contract (disabling flash loans and reentrant attacks entirely for simple contract entry points). Thanks for your meaningful insight around the EVM architecture 😉
- Ryan Park (ryanology045), who provided in-depth analysis around possible DeFi contract exploits (which is also his daily job at the moment)
- Jake Kim (jakeonme), who came up with the Matrix analogy and GIFs plus the courage to write this down on Medium 🥳
