ALLCRYPT.COM HACKED INTO OBLIVION THIS COMES STRIGHT FROM THE ALLCRYPT.CO BLOG Posted by AllCrypt Staff on March 18, 2015 Late Sunday night AllCrypt was hacked. We spent the last few days going over logs, speaking with other crypto exchange operators and law enforcement, and here is what we know. Around 8PM on Sunday (all times EDT) our marketing director’s blog account requested a password reset. Up until the writing of this post (Wednesday morning, 10am) we do not know how the thief managed to know the marketing director’s (will refer to this as MD from here) account. Our best guess is it was an educated guess based on info found (more on that in a moment). The MD saw this email come in, and forwarded it to myself, and another team member (a technical lead/temporary assistant support staff), letting us know what happened and that he did not request the password reset. I did not see the email at the time, as I was out, and it was not a huge red flag that would require a phone call. Once I returned home later, I saw the email, and logged into the server to double-check on things. That’s when I discovered the breach. Apparently, the thief had gained access to the tech assistant’s email account. That email was hosted on a private server (not gmail, yahoo, etc). We have no idea how the password was acquired. We spent a lot of time this week downloading password lists from torrents, tor sites, etc, and could find his password in none of the lists. He assures us he did not use the password in multiple places, and that it was a secure password. Our best guess is that it was a brute force attempt. The mail server he uses used the dovecot package for IMAP mail, which, for reasons we cannot comprehend, does NOT log failed password attempts by default. Because of this, at first, we believed that the hacker somehow had the person’s password. But we do not know, and there is no way to know at this point how the password was found. We are not sure if AllCrypt was targeted, or if it was a “fortunate” thing for the hacker. Our hypothesis is this: The user’s email account was breached, and in looking through the emails, saw that he had some admin rights for AllCrypt.com. In that account were emails from myself and our MD. After playing on AllCrypt for some time, the hacker tries to do a WordPress password reset for all of the allcrypt related emails he sees. My personal email was not the email for the main admin blog account, and either was the user who’s email was breached. Our MD, however, did use his personal email. A password reset was issued, and sent to the MD. The massive screwup that led to the loss of funds is when the MD forwarded that email to myself and the tech team member. He forwarded the password reset link. To the breached email account. The hacker reset the MD’s password, and had administrative access to the blog. Access which allows uploading of new files/plugins for WordPress. The hacker first uploaded a file, class.php. After we examined it, we discovered that it grants web-based command line access to any files the web user has access to. Namely, the entire http tree. Quick looks through the web source files is all it takes to see the hostnames, login names, and passwords for the database. The database credentials WERE protected in a ‘hidden’ file in a non-www accessible directory, however, anyone smart enough to read code can find the include lines that point to that file. The www user must have read access to that file, so the class.php the hacker uploaded also had read access. Then, using WordPress, he uploaded adminer.php – a web based database management tool similar to PHPMyAdmin. It was a simple task to then query the database. The hacker created a new user account on AllCrypt, used adminer.php to UPDATE userbalances SET balance=50 WHERE userid=whatever AND symbol=BTC to set his balances to whatever he wanted. Then he began to issue withdrawals. Lots of them. Unfortunately, he was stopped, but only briefly, by our secondary accounting system. Using the wallets for accounting is terrible – we used to do it and realized it killed the wallets performance. So, we created a wrapper around the wallets that stored account balances. Every user had 2 balances. The site balance, which he was able to modify, and the wallet balance, which was in the other system, which he did NOT have access to. It should have stopped him. Except for one small issue: He bought and sold coins. He added fake BTC to his account, and used that to buy DOGE, and LTC, and other coins. When coins are bought and sold, your wallet balances are updated as the coins are moved. By using fake BTC to buy DOGE, he actually had DOGE in his wallet to withdraw. He then sold fake various other coins for BTC and was able to withdraw it. There was a hacking incident months ago regarding JPC that happened on poloniex I believe. Someone’s account on AllCrypt was not protected by 2FA and his password was stolen from the mining pool he mined on (user used the same password everywhere). The hacker withdrew the JPC to poloniex (might have been mintpal now that I think about it?), and proceeded to sell it. The problem was that the user’s account poloniex could not make a withdrawal without clicking an email link. (The coins were not sold entirely on AllCrypt because we did not have the volume, it was better to sell them on the other exchange) What did the thief do? He found a coin with low volume/interest, and with a large buy/sell spread. He made a new account. New account bought some of that coin, then sold it at a really high price. The account with the stolen JPC-sold-for-BTC bought it. Then sold it back at a really low price. Other account bought it, sold it really high. Over and over, until all the BTC in the hacked account was transferred to the hacker’s legitimate account, and it was withdrawn. My point? Even the most secure systems can be circumvented with enough time and ingenuity. The hacker in our case skirted the secondary accounting system by buying the coins he wanted to withdraw, with fake BTC. Through all of this, just about every coin of value we have was emptied. BTC, LTC and DOGE were the first to go. Unfortunately, this was not the first time the hacker had done this we believe – in sorting through the web tree and database, we saw he was able to give his account administrative access. Which allowed him to make backup dumps of the private keys of almost all of the coins we house. Those keys, when backed up, are not stored in the web directory, but as dump files. However – the web user creates the files. Which means the class.php directory explorer software he used could read them. We noticed this only after we were auditing balances, and seeing balances lower (zero, actually) than they had been at first, although we had not moved any coins. We checked the admin areas of the server and saw about 70 dump logs, full of private keys. We began to frantically transfer what we could, but we had not even noticed this was happening until well over a day later. We turned off the pollers, we shut down the site, we changed password and locked everything down. The wallets were safe, right? Well, not if he saved the keys, installed his own wallets, and imported them then moved the funds. So that is what happened. We do not know if it was a targeted attack, or the hacker just “lucked out” by finding the tech’s email password. But that led to a chain of events that ended in the site being nearly liquidated to nothing. What about the coins we still have? We will do our best to return the coins to their rightful owners. It’s not much. And we have to build a tool to do it. We cannot open the site back up – the hacker had full database access. While everyone’s password is encrypted, we don’t trust that completely since it was revealed. In addition, the seed for everyone’s Two-Factor Authentication was seen. Completely invalidating everyone’s 2FA. You can reset a password with a valid 2FA code. So we need to build some tool to email everyone who has any sort of balance left, and let you log in securely, somehow (likely via a key we email you) and provide withdrawal addresses, and we send the coins to that. It kills me that this happened. We had a small hack a year ago, and we recovered near instantly. And we were secure and solid for a year, and decided to shut down because of lack of volume, and lack of profits. Our profits were lost as well. We only made 10.8 BTC over the course of 13 months of operation. Between hardware and operating costs, I am personally down over $15,000. Believe me – I feel your pain as well. No one on the site had as much on the servers as I personally did. Not that I expect pity or compassion, but I think it’s important to know that I’m not retiring to a private island because of this. I also think it’s important to be as open as possible to assuage any fears of an inside job. Worse yet, we had just over 2 BTC in charity collections that were also stolen. We never had the volume to support the monthly donations we planned to make (it felt silly donating $8 to a charity, so we decided to let it build up and make a larger donation at a later date). We may re-open someday. This puts us back months, if not years. Without even that 10.8 BTC to recoup any costs, this is absolutely crushing. So I cannot say. I’ve actually had offers to buy the site. Maybe. I don’t know. This was my baby, and I’d hate to let it go. Anyway, that’s all the news I have for now. I’ll try to post updates about any coin returns, and will be sending emails to those affected when we get those tools in working order. It won’t be a quick process. There’s ton’s of cleanup to be done. Oh – one last thing. The asshole who did it. We traced IP address usage. The password reset link, and the initial uploads and accesses to class.php came from the IP address 128.199.56.120. This IP started changing once our intrusion detection system kicked in. We have a system that scans the web logs, and auto-bans any IP address that does something that looks funny. Well, in using adminer.php, which uses GET requests to access the database, the URL parameters flagged the IDS, and his IP started getting banned. He apparently had access to multiple VPNs (or was using tor, we haven’t checked that yet). Regardless, we looked at user usage for that ip, and back in August, a new user was made. Later that user logged in on that IP. It was also the first user account made that tried to make withdrawals (later, an account was made, then deleted, presumably to cover his tracks, however, while the account itself was deleted, the audit logs were not). The username was: samuel The user also tried to log in with the username: siavash The email address associated with the account is: siavashnobody@gmail.com Additionally, the user made a new account, which, like I said above, he deleted after he was done. But the audit logs were not deleted. I am reserving the information about that account for the time being at the request of law enforcement. It may have information that may be of some importance. The bulk of the BTC withdrawals went to: 17B8qfaeNsv3TZbpycUs6dkzYiJGWNeCw5 Many DOGE withdrawals went to: DJiYqeXZBJXbmsWMefpRY7dEsJDASFAYxX Most of the IP addresses geolocate to Iran. There is one however that does not. That is also being looked at by law enforcement. And now, here’s a Q&A to answer some questions I’m sure the armchair quarterbacks who know better than we do and will jump at the chance to tell us we are so stupid and that “you should have done” might want to ask/scream at us: Q: You idiots! How could you put the database password in the website source files?! A: They have to be, SOMEWHERE. SOMEHOW the website has to be able to authenticate to the database. 99.9% of the time it’s a non issue, it’s in code, or even in a directory not accessible by the user’s browser. But when someone basically has a command line access as the web user, it’s easy as pie to read the file. Q: Idiots! Why weren’t your wallets encrypted?! A: They were. But the hacker made normal withdrawals. The wallets themselves were not hacked. The database was and then tricked into allowing valid withdrawals. Q: You morons! You had a WordPress site that allowed uploading of new files?!?! A: It was the marketing director’s account. Beings that he was constantly updating files, it was necessary for his account to have the ability to upload new files. Q: Your MD forwarded the password reset link in a plain text email?! REALLY?! A: We agree on this one. There was more than one “What the HELL were you thinking, you IDIOT?!” phone call made in the last 72 hours. That absolutely should have never been done. He’s a marketing guy. Not a tech guy. We’ve all been guilty of making stupid mistakes. He thought he was being helpful by letting us know. How was he to know that there was an email account breach? Q: Lies lies lies, it was an inside job. A: Does there exist the possibility that it was an inside job collusion by our MD and tech agent? Maybe. I won’t say it’s 100% impossible. That being said, I trust them. I wouldn’t be in business with them if I did not. Law enforcement has been contacted and some conversations have been had. I am sure they will do due diligence investigating the things I cannot. Q: You’re making this WHOLE thing up. There is no marketing director or tech agent, you stole the coins! A: All I’ll say in response is that if *I* stole them, it’d have been a hell of a lot easier to just pull the power cord and walk away. All this drama and showmanship just to hide something that could never be proven anyway? I have better things to do. Besides, I posted above, and on twitter, who was responsible. Q: Your security sucks! A: I see you running an exchange successfully, I’ll take your advice. Wait, you don’t run an exchange? You’re unemployed? Thanks for the input. Q: You should have had a cold wallet! A: We did. It was user configurable. You could choose how much was in the cold wallet. NO ONE USED IT. (Aside from a few dormant accounts). If we offer the security, and you do not use it, I’m no longer listening to complaints about it. We are not here to diaper you and make sure you aren’t being stupid with your money. That’s all the angry questions I could think to answer at the moment. I’ll update this post when necessary.
ALLCRYPT.COM HACKED INTO OBLIVION THIS COMES STRIGHT FROM THE ALLCRYPT.CO BLOG Posted by AllCrypt Staff on March 18, 2015 Late Sunday night AllCrypt was hacked. We spent the last few days going over logs, speaking with other crypto exchange operators and law enforcement, and here is what we know. Around 8PM on Sunday (all times EDT) our marketing director’s blog account requested a password reset. Up until the writing of this post (Wednesday morning, 10am) we do not know how the thief managed to know the marketing director’s (will refer to this as MD from here) account. Our best guess is it was an educated guess based on info found (more on that in a moment). The MD saw this email come in, and forwarded it to myself, and another team member (a technical lead/temporary assistant support staff), letting us know what happened and that he did not request the password reset. I did not see the email at the time, as I was out, and it was not a huge red flag that would require a phone call. Once I returned home later, I saw the email, and logged into the server to double-check on things. That’s when I discovered the breach. Apparently, the thief had gained access to the tech assistant’s email account. That email was hosted on a private server (not gmail, yahoo, etc). We have no idea how the password was acquired. We spent a lot of time this week downloading password lists from torrents, tor sites, etc, and could find his password in none of the lists. He assures us he did not use the password in multiple places, and that it was a secure password. Our best guess is that it was a brute force attempt. The mail server he uses used the dovecot package for IMAP mail, which, for reasons we cannot comprehend, does NOT log failed password attempts by default. Because of this, at first, we believed that the hacker somehow had the person’s password. But we do not know, and there is no way to know at this point how the password was found. We are not sure if AllCrypt was targeted, or if it was a “fortunate” thing for the hacker. Our hypothesis is this: The user’s email account was breached, and in looking through the emails, saw that he had some admin rights for AllCrypt.com. In that account were emails from myself and our MD. After playing on AllCrypt for some time, the hacker tries to do a WordPress password reset for all of the allcrypt related emails he sees. My personal email was not the email for the main admin blog account, and either was the user who’s email was breached. Our MD, however, did use his personal email. A password reset was issued, and sent to the MD. The massive screwup that led to the loss of funds is when the MD forwarded that email to myself and the tech team member. He forwarded the password reset link. To the breached email account. The hacker reset the MD’s password, and had administrative access to the blog. Access which allows uploading of new files/plugins for WordPress. The hacker first uploaded a file, class.php. After we examined it, we discovered that it grants web-based command line access to any files the web user has access to. Namely, the entire http tree. Quick looks through the web source files is all it takes to see the hostnames, login names, and passwords for the database. The database credentials WERE protected in a ‘hidden’ file in a non-www accessible directory, however, anyone smart enough to read code can find the include lines that point to that file. The www user must have read access to that file, so the class.php the hacker uploaded also had read access. Then, using WordPress, he uploaded adminer.php – a web based database management tool similar to PHPMyAdmin. It was a simple task to then query the database. The hacker created a new user account on AllCrypt, used adminer.php to UPDATE userbalances SET balance=50 WHERE userid=whatever AND symbol=BTC to set his balances to whatever he wanted. Then he began to issue withdrawals. Lots of them. Unfortunately, he was stopped, but only briefly, by our secondary accounting system. Using the wallets for accounting is terrible – we used to do it and realized it killed the wallets performance. So, we created a wrapper around the wallets that stored account balances. Every user had 2 balances. The site balance, which he was able to modify, and the wallet balance, which was in the other system, which he did NOT have access to. It should have stopped him. Except for one small issue: He bought and sold coins. He added fake BTC to his account, and used that to buy DOGE, and LTC, and other coins. When coins are bought and sold, your wallet balances are updated as the coins are moved. By using fake BTC to buy DOGE, he actually had DOGE in his wallet to withdraw. He then sold fake various other coins for BTC and was able to withdraw it. There was a hacking incident months ago regarding JPC that happened on poloniex I believe. Someone’s account on AllCrypt was not protected by 2FA and his password was stolen from the mining pool he mined on (user used the same password everywhere). The hacker withdrew the JPC to poloniex (might have been mintpal now that I think about it?), and proceeded to sell it. The problem was that the user’s account poloniex could not make a withdrawal without clicking an email link. (The coins were not sold entirely on AllCrypt because we did not have the volume, it was better to sell them on the other exchange) What did the thief do? He found a coin with low volume/interest, and with a large buy/sell spread. He made a new account. New account bought some of that coin, then sold it at a really high price. The account with the stolen JPC-sold-for-BTC bought it. Then sold it back at a really low price. Other account bought it, sold it really high. Over and over, until all the BTC in the hacked account was transferred to the hacker’s legitimate account, and it was withdrawn. My point? Even the most secure systems can be circumvented with enough time and ingenuity. The hacker in our case skirted the secondary accounting system by buying the coins he wanted to withdraw, with fake BTC. Through all of this, just about every coin of value we have was emptied. BTC, LTC and DOGE were the first to go. Unfortunately, this was not the first time the hacker had done this we believe – in sorting through the web tree and database, we saw he was able to give his account administrative access. Which allowed him to make backup dumps of the private keys of almost all of the coins we house. Those keys, when backed up, are not stored in the web directory, but as dump files. However – the web user creates the files. Which means the class.php directory explorer software he used could read them. We noticed this only after we were auditing balances, and seeing balances lower (zero, actually) than they had been at first, although we had not moved any coins. We checked the admin areas of the server and saw about 70 dump logs, full of private keys. We began to frantically transfer what we could, but we had not even noticed this was happening until well over a day later. We turned off the pollers, we shut down the site, we changed password and locked everything down. The wallets were safe, right? Well, not if he saved the keys, installed his own wallets, and imported them then moved the funds. So that is what happened. We do not know if it was a targeted attack, or the hacker just “lucked out” by finding the tech’s email password. But that led to a chain of events that ended in the site being nearly liquidated to nothing. What about the coins we still have? We will do our best to return the coins to their rightful owners. It’s not much. And we have to build a tool to do it. We cannot open the site back up – the hacker had full database access. While everyone’s password is encrypted, we don’t trust that completely since it was revealed. In addition, the seed for everyone’s Two-Factor Authentication was seen. Completely invalidating everyone’s 2FA. You can reset a password with a valid 2FA code. So we need to build some tool to email everyone who has any sort of balance left, and let you log in securely, somehow (likely via a key we email you) and provide withdrawal addresses, and we send the coins to that. It kills me that this happened. We had a small hack a year ago, and we recovered near instantly. And we were secure and solid for a year, and decided to shut down because of lack of volume, and lack of profits. Our profits were lost as well. We only made 10.8 BTC over the course of 13 months of operation. Between hardware and operating costs, I am personally down over $15,000. Believe me – I feel your pain as well. No one on the site had as much on the servers as I personally did. Not that I expect pity or compassion, but I think it’s important to know that I’m not retiring to a private island because of this. I also think it’s important to be as open as possible to assuage any fears of an inside job. Worse yet, we had just over 2 BTC in charity collections that were also stolen. We never had the volume to support the monthly donations we planned to make (it felt silly donating $8 to a charity, so we decided to let it build up and make a larger donation at a later date). We may re-open someday. This puts us back months, if not years. Without even that 10.8 BTC to recoup any costs, this is absolutely crushing. So I cannot say. I’ve actually had offers to buy the site. Maybe. I don’t know. This was my baby, and I’d hate to let it go. Anyway, that’s all the news I have for now. I’ll try to post updates about any coin returns, and will be sending emails to those affected when we get those tools in working order. It won’t be a quick process. There’s ton’s of cleanup to be done. Oh – one last thing. The asshole who did it. We traced IP address usage. The password reset link, and the initial uploads and accesses to class.php came from the IP address 128.199.56.120. This IP started changing once our intrusion detection system kicked in. We have a system that scans the web logs, and auto-bans any IP address that does something that looks funny. Well, in using adminer.php, which uses GET requests to access the database, the URL parameters flagged the IDS, and his IP started getting banned. He apparently had access to multiple VPNs (or was using tor, we haven’t checked that yet). Regardless, we looked at user usage for that ip, and back in August, a new user was made. Later that user logged in on that IP. It was also the first user account made that tried to make withdrawals (later, an account was made, then deleted, presumably to cover his tracks, however, while the account itself was deleted, the audit logs were not). The username was: samuel The user also tried to log in with the username: siavash The email address associated with the account is: siavashnobody@gmail.com Additionally, the user made a new account, which, like I said above, he deleted after he was done. But the audit logs were not deleted. I am reserving the information about that account for the time being at the request of law enforcement. It may have information that may be of some importance. The bulk of the BTC withdrawals went to: 17B8qfaeNsv3TZbpycUs6dkzYiJGWNeCw5 Many DOGE withdrawals went to: DJiYqeXZBJXbmsWMefpRY7dEsJDASFAYxX Most of the IP addresses geolocate to Iran. There is one however that does not. That is also being looked at by law enforcement. And now, here’s a Q&A to answer some questions I’m sure the armchair quarterbacks who know better than we do and will jump at the chance to tell us we are so stupid and that “you should have done” might want to ask/scream at us: Q: You idiots! How could you put the database password in the website source files?! A: They have to be, SOMEWHERE. SOMEHOW the website has to be able to authenticate to the database. 99.9% of the time it’s a non issue, it’s in code, or even in a directory not accessible by the user’s browser. But when someone basically has a command line access as the web user, it’s easy as pie to read the file. Q: Idiots! Why weren’t your wallets encrypted?! A: They were. But the hacker made normal withdrawals. The wallets themselves were not hacked. The database was and then tricked into allowing valid withdrawals. Q: You morons! You had a WordPress site that allowed uploading of new files?!?! A: It was the marketing director’s account. Beings that he was constantly updating files, it was necessary for his account to have the ability to upload new files. Q: Your MD forwarded the password reset link in a plain text email?! REALLY?! A: We agree on this one. There was more than one “What the HELL were you thinking, you IDIOT?!” phone call made in the last 72 hours. That absolutely should have never been done. He’s a marketing guy. Not a tech guy. We’ve all been guilty of making stupid mistakes. He thought he was being helpful by letting us know. How was he to know that there was an email account breach? Q: Lies lies lies, it was an inside job. A: Does there exist the possibility that it was an inside job collusion by our MD and tech agent? Maybe. I won’t say it’s 100% impossible. That being said, I trust them. I wouldn’t be in business with them if I did not. Law enforcement has been contacted and some conversations have been had. I am sure they will do due diligence investigating the things I cannot. Q: You’re making this WHOLE thing up. There is no marketing director or tech agent, you stole the coins! A: All I’ll say in response is that if *I* stole them, it’d have been a hell of a lot easier to just pull the power cord and walk away. All this drama and showmanship just to hide something that could never be proven anyway? I have better things to do. Besides, I posted above, and on twitter, who was responsible. Q: Your security sucks! A: I see you running an exchange successfully, I’ll take your advice. Wait, you don’t run an exchange? You’re unemployed? Thanks for the input. Q: You should have had a cold wallet! A: We did. It was user configurable. You could choose how much was in the cold wallet. NO ONE USED IT. (Aside from a few dormant accounts). If we offer the security, and you do not use it, I’m no longer listening to complaints about it. We are not here to diaper you and make sure you aren’t being stupid with your money. That’s all the angry questions I could think to answer at the moment. I’ll update this post when necessary.
Comments
Post a Comment
Go ahead and drop me a line! The Sultan would love to hear what you have to say!