Changing server architecture is always a big step for any project. If your company has arrived at this point, there is most likely a good reason for...
HOW WE SAVED A STOLEN MILLION
This story is about a big fintech company getting hacked. We'll
call it TwinPeaks to preserve anonymity, or TP for short.
The quieter we become, the more we hear.
It was a cold winter Friday evening. All of a sudden - a phone call coming from the States, you could tell by the area code.
- Daniel, please go online, we have been hacked…
It was TP's CTO. Bad luck seemed to have visited them that day: they got hacked. In short, their project hot-wallet was emptied into external Bitcoin wallets.
What's there to do? A quick look at the transactions made me gasp in horror - the transaction sum equaled to a bit less than 1 000 000 USD.
My first thought was: can you really help there? I guess, you can only offer your condolences.
The only thing Columbus discovered was that he was lost!
One important point, though, - the transactions had zero confirmations. Why? Because Bitcoin network was overloaded at that moment. And that situation gave a chance to take urgent action.
The thing is, as long as a transaction has no confirmation in Blockchain, it is isolated
in a so-called Mempool - a temporary storage. While a
transaction is in Mempool, there is still a chance to influence its fate: to raise
the fee, for example, or even to change the recipient's address.
I think it will be useful to tell about the nuts and balls of a Bitcoin transaction.
The nuts and balls of a Bitcoin transaction
Many are caught in a misconception that it's a kind of message that says how much an addressee should receive, and when a transaction is in progress, the sender's balance goes down, and the recipient's one goes up. But it's not quite like that.
In reality, it's a list of incoming and outgoing transactions, which, when put together, make up the sum that has been sent. And a users' balance is the result of mathematical calculations of incoming and outgoing transactions.
Let me show you the transaction graph:
Dale's balance was 10.5 Bitcoins sent to him by his friend, the FBI chief.
He decided to send Diane 0.5 btc and performed a transaction. At that moment there was a transaction formed which stated that 0.5 btc should go to Diane, and 9.5 btc should come back to another address belonging to Dale. His entire balance is blocked now, but it's not spent yet, as the transaction isn't confirmed yet.
After that, miners' hard work begins. Some miner in American green forests finds this transaction and confirms it. Then five more miners do the same thing and, finally, blockchain.info shows that the transaction has six confirmations, which means it can be considered valid.
Diane had 24.5 btc, which she received from her friend, Federal Security Service chief Albert, and from the deranged Dr. Lawrence. As soon as the operation has been confirmed, her balance has reached an ample semi-round amount of 25 btc, and Dale got his previously blocked 9.5 btc back.
A bit confusing, isn't it? Yes, it is.
But let's imagine that Dale has made a mistake (I myself have trouble imagining that, but let's try anyway) and sent this half a Bitcoin to Bob, not Diane.
All his 10 btc got blocked, and the transaction went to Mempool where thousands of miners
are about to discover it.
Should that happen, Dave wouldn't sit still, since, as long as there are no confirmations, he still is the owner of his 10 Bitcoins. Hypothetically he can send many more transactions, hundreds of them.
"How come?!" you may wonder. "Can't Dave send 0.5 btc to Diane one hundred times and make her 50 btc richer?!"
As soon as any of those transactions is confirmed, the rest of them won't stand a chance. Yes, miners will get to them as well, but they will simply cancel the rest of the transactions once they see those Bitcoins have been spent already.
So, how can Dale save his 0.5 btc and how could I snatch back one million USD out of thugs' paws that evening?
The answer is very simple: I had to send one more transaction which would be discovered by a miner earlier than the initial one. How can it be done? Let me tell you a little bit about...
How miners work
A miner's task is making electricity bring the biggest income possible. One of his tasks
is confirming a transaction online. When they confirm a transaction, they receive a
reward. How big is the reward? Depends on the transaction fee cost. The bigger is the
amount of the fee appointed by a user, the more will a miner get as a reward. That is
why it is more lucrative for a miner to work with transactions which have a bigger
reward. Naturally, working with such transactions will be their priority. The bigger is
the fee, the faster a transaction will be processed.
So, the task narrowed down to sending a transaction to Mempool identical to the one that had been sent previously (with the same 'in', meaning, it should be based on the same incoming transactions), but with a different recipient's address and a bigger fee.
Trying to save the stolen money I created a transaction with an enormous fee (more than $100 instead of initial $7) and changed the recipient's address, having put the address belonging to TwinPeaks. I signed that transaction with a private key and performed its broadcast from several different spots simultaneously - big resources (like, blockchain.info and some other similar resources).
And I started waiting.
It seemed like I'd been waiting for ages.
I nervously paced across kitchen back and forth smoking and glancing at the monitor.
I was also text-chatting with TP's CTO, although, I must admit, his answers were rather curt.
One day the sadness will end.
30 minutes had passed and … I saw the first confirmation of my transaction. In some time, I had all six confirmations of my transaction, and the malefactor's transaction was soon canceled, since the money had been already spent.
I think there is no need to describe how I felt at that moment - many of you have felt this way at some point. Achievement is its own reward, pride obscures it.
Looked like it was time to go to bed, but I decided to describe the entire algorithm in
a simple python script, so that anyone could use this solution if, God forbid, comes
across a similar problem. Or, what seems to be a more frequent case, someone will simply
get tired of waiting until their transaction gets confirmed and will decide to pay for
its processing a bit more having increased the fee. That's why I called the script
upmyfee and uploaded it to Github: https://github.com/dysnix/upmyfee
The script is very simple to use and has a README with use-cases.
Only having made the final git push I went to sleep thinking that owing to cryptocurrency market development and crypto-assets popularization the price of security is growing exponentially. That is why never try to cut corners on the security of your product, and if it happens so that you are in trouble - there is always a way out.
The owls are not what they seem.
Daniel Yavorovych, Dysnix CO-Founder & CTO