Fulcrum had a $2.5m vulnerability over a month ago and still hasn’t told anyone
DeFi has the potential to provide access to financial services to billions of people. But it is also a young space and vulnerable to errors, occurrences which some parties are all too happy to exploit as proof that DeFi will never work. We feel compelled to disclose the interactions we had with the bZx/Fulcrum team to provide transparency to the community, avoid that more people will lose more money, and to ensure that faith in DeFi isn’t shattered because of the mistakes of individual actors.
The Vulnerability
All started on January 11, 2020, when Fulcrum team released their own Flash Loans feature on the Ethereum Mainnet, and we happened to find a very critical vulnerability in it. We discovered that $2.5M of user funds from 3 pools could be stolen within a single transaction. We prepared our own smart contract to perform a white-hat hack to protect user funds. Since the vulnerable smart contract was published less than 48h before we discovered the issue, there was a very high chance malicious attackers could exploit it, and we wanted to assure that this wouldn’t happen.
At the very last moment, we decided to make proof of concept and transferred only 1 weiDAI (0.000000000000000001 DAI) in two separate transactions to make sure it’s true and give a chance to the Fulcrum team to shut-down their system (we had no idea if they have such functionality and had not enough time to check). This potentially would allow their system to avoid inconsistent state and recover without significant damage. We decided to spend up to half of an hour trying to reach the team by every accessible communication channel, and if we failed to reach them we were standing by to white-hack the funds and immediately disclose it to the DeFi community.
It was 2 AM local time for Fulcrum team, BUT we succeeded, and they responded after a relatively short delay.
We provided a link to the proof-of-hack transactions (first and second) in the hope that they would immediately stop their smart contract. We offered to the Fulcrum team to white-hack their pools at any moment to protect user funds if they had no kill-switch, but they declined. Apparently they thought it it was worth risking user funds during the period of building and queueing a patch in order to avoid integration issues and negative attention.
It took nearly 4 hours for the Fulcrum team to manage the issue, and we got no details from the team about the progress. Additionally, the deployment of the fix took another 12 HOURS, because of special system upgrade timelock in the smart contract.
So there were 16 hours during which anyone could steal $2.5M:
- Buggy source code was publicly available on GitHub and Etherscan
- Anyone could discover and investigate our proof-of-hack txs
- We had a 1-click solution to rescue funds and our finger on the red button
Since we contacted the Fulcrum team and they denied us to white-hack, we were legally unable to help their users and were forced to wait and monitor their contracts for suspicious transactions and Approval
events for 16 hours.
Payoff
Bounty evasion
Finally, the fix was deployed on mainnet. But this story wasn’t over yet. We genuinely feel ashamed that after working through an anxiety-filled night with them, they basically tried to deny us any bounty reward. Please note that it’s usually industry practice to share a percentage of funds saved, while here they are trying to deny us anything based on a technicality.
The Audit Hustle
Then the Fulcrum team tried to order a security audit from us for only the FlashLoan feature. We proposed a $2k price for 1-week audit by three people. They wanted to push the price down to price to a $1.5k because they thought it is just 30 lines of code and no need to check any other code (we do not agree with them).
Imagine having an unaudited smart contract live on mainnet, with millions of dollars of user funds at risk, and your main priority is to skim $500 from the people who just saved your users $2.5m!
The third insult is the charm
Finally, the Fulcrum team proposed us to make this audit as part of the original $5k bounty, wow!
We were very upset and decided not to take responsibility for such low compensation relative to the reputation risks:
The cover-up
After all this, it still got worse. Instead of disclosing the incident to the community as promised, the strategy was now to cover-up. They tried to use the $3.5k to silence us and hide the whole thing. The right thing would have been to share it with their users and community so they can decide whether they want to continue entrusting their money to code that the Fulcrum team released.
But instead, they continued as if nothing happened, leaving more money for the inevitable to happen: more exploits. Since the incident two separate exploits drained around $1m from the Fulcrum system.
We strongly feel that we need to come forward with this information, and honestly wish we had done so earlier. Making mistakes is ok, but denying the truth to users and the commnunity and therefore depriving them of their ability to make infomed decisions, especially when it comes to money, is unacceptable.
The teams intent to silence us with an NDA and wait for months to publish this incident cannot stand.
Two more insults
We were anxious about user funds and tried to help Fulcrum not to sink, while the Fulcrum team was more concerned about their own company instead of user funds, which was possible during a total of 16 hours! If we knew about the 12 hours timelock before reporting to the team, we would have white-hacked all the funds to protect them from being stolen by a malicious attacker.
On top of all, the Fulcrum team started accusing us of the recent exploits.
Since the Fulcrum pools are almost drained because of FlashLoan-related hacks, we are publishing this material to make a clear statement about our relation to Fulcrum. We are not related to any other attacks of Fulcrum, of course. The recent FlashLoan hacks exploited old issues discovered and reported by samczsun. Our hack was related to the buggy implementation of the FlashLoans feature itself in the Fulcrum system.
From several people on the ETHDenver hackathon we heard that Fulcrum starting telling people that they suspect us to be behind the attacks, but frankly I have to say we were very busy building 1x.ag:
We also applied to the Fulcrum bounty on ETHDenver with the 1x.ag project, but they decided to keep prize online for a few weeks/months to wait and check if 1x.ag will bring real amounts to their system (WHAT? IS THIS HOW HACKATHON BOUNTIES WORK?!).
Regarding the hacks, we hope insurance will cover at least part of the Fulcrum system debt.