I apologize, once again, for the length of this post but sometimes you just need to get something off your chest. This will be my final word on this subject, but I won’t promise this is my final long post.
I had an interesting conversation via instant messenger with Josh Schnell this evening. It turns out, he felt I was the one being unethical for suspending his accounts when he has paid for service through April 3rd. It was the first I heard from him since our conversation during the evening of January 12th, when he led me to believe his account(s) had been hacked.
Before I get to this evening’s conversation, let’s review the events of January 12th and 13th, some of which I wrote about in my post “A Complete Lack Of Ethics“.
- January 12 – 20:50 : I received a direct message from him asking what was up with the server, that DNS was not working, and that his account alpha-one was not receiving email.
- January 12 – 21:17 : I received an email from PayPal notifying me that his hosting subscription had been cancelled.
- January 12 – 21:28 : He told me that alpha one was not loading and mail was not being forwarded to Google mail.
- January 12 – 21:30 : I check the WHOIS for his domain to find his name servers (DNS) has changed to editdns.net and I let him know.
- January 12 – 21:31 : He says, WTF. He acted shocked that the name servers had been changed.
- January 12 – 21:36 : He leads me to believe that his account has possibly been hacked, and that he is “looking into it” with his registrar
- January 12 – 21:47 : I let him know about the email notification of his account cancellation. He says, WTF, WTF, WTF, leading me to believe (again) that his account has indeed been hacked.
- January 12 – 22:00 : I spend the next five hours checking server logs, comparing IP addresses, looking for signs of “the hackers”.
- January 13 – 09:00 : I notify server techs on my end asking them to make sure everything on my server is secure.
- January 13 – 11:00 : I happen to check WHOIS on his domain again to find it sitting at another hosting provider and different name servers than the night before.
- January 13 – 14:45 : Server techs reply that there has been no apparent breach, all services are running fine, everything looks good on the server, but his passwords “could” have been compromised.
- January 13 – 15:00 : Because I did not hear from him and he was not online, I decide to suspend all his accounts to cut off all external access to his accounts until I hear from him.
- January 13 – 15:45 : I run a WHOIS query on each of his domains to see if the “hack” is widespread” and discover that almost all of his domains were transferred to the new hosting provider.
- January 13 – 16:00 : I finish loading each of his domains in my web browser to find most of them are functioning properly, from the new hosting server. This raised the question, why would someone hack his DNS just to put up his own sites?
- January 13 – 17:00 : After reviewing all of the WHOIS data which show the dates the registration information was last updated, I realize most of the accounts were moved over the course of the past month. At this point I realize I had been made a fool of.
I sent him two emails between 14:00 and 17:00 with no response. He was not online on the instant messenger, so I took action to protect his interests, as well as mine. Yet, tonight he claims I am the unethical one.
Six hours on Tuesday and eight hours on Wednesday were wasted looking for a “hacker” when in reality all I was dealing with was a liar. I’m the unethical one? Please. I was simply doing what I should have been doing to protect my server and my customers.
Before I begin discussing tonight’s conversation (which sheds even more light on the subject), let me be clear about a few items:
- I am not angry that he decided to switch hosting providers. I am angry that he played me for a fool by lying to me and wasting 14 hours of my time.
- I suspended his accounts because he led me to believe he had been hacked, and because he wasn’t online and didn’t respond to the emails I sent him, I did what I needed to do to protect the server.
- I have more respect for hackers than liars.
Now that we’re clear about the events on Tuesday and Wednesday, let’s discuss this evening’s conversation. When he messaged me I couldn’t believe he claimed I was the one being unethical after everything that had transpired. The reason I am posting this conversation is simple. He claimed this evening that he was not lying, but the facts don’t lie. During the conversation I realized some things just were not true. After the conversation I stepped away for a few to get a bite to eat and I realized some things just didn’t add up. So here it is, because I need to get this off my chest as well.
First of all, I find it interesting that I didn’t hear a single word from him for 66 hours. Our last conversation ended on January 12, at 9:53 pm, he finally contacted me this afternoon at 3:59 pm. If your account had been hacked, or you had led your hosting provider to believe your account may have been hacked, wouldn’t it be prudent to contact them as soon as possible to find out if it had indeed been hacked? Oh, that’s right, we all know it wasn’t hacked now, don’t we?
3:59:14 Josh Schnell: can you please explain to me why the account has been suspended when we’ve paid for the next three months?
4:35:55 Josh Schnell: you know. I fail to see how me paying through until march is unethical.
4:36:51 Josh Schnell: asking for support on something you paid for is completely within my rights.
6:15:09 Michael Barrett: asking for support for something that doesn’t exist, leading me to believe everything is kosher with us, is unethical. I spent 14 hours looking into your problem, even contacting others that I know who might help, and for what? You had already transferred the account. The reason email was not working and dns was not working is because it wasn’t assigned to my server anymore, yet you led me to believe (with all your WTF statements) that you had nothing to do with the changes… That sir, is unethical.
As you can see, there is a gap between his messages and my response, as I was away from my desk running errands this afternoon. I found it hilarious at the time that he thinks he has “rights” on my hosting server. He has an account. He pays for that account. He enjoys a few privileges and has access to certain software, but I wouldn’t call those “rights”. His account, like all of the others on my server, is subject to specific terms of service.
There are a few key points of our terms of service which apply to Mr. Schnell’s accounts (especially when I thought his account(s) may have been hacked).
From section 2:
- Failure to respond to email from our abuse department within 48 hours may result in the suspension or termination of your services.
- It is your responsibility to ensure that scripts/programs installed under your account are secure and permissions of directories are set properly, regardless of installation method
Although section 3 applies to spam, one paragraph applies because I was led to believe the account(s) may have been hacked:
- We reserve the right to require changes or disable as necessary any web site, account, database, or other component that does not comply with this policy, at its sole discretion. We also reserve the right to make any such modifications in an emergency at our sole discretion.
From section 6:
- We reserve the right to cancel the account at any time.
Simply paying for hosting service does not imply that your account will not be cancelled or suspended for any reason before the hosting period has expired.
I don’t think anyone would think it was unethical to cut off the “hackers” before they caused more trouble. I don’t think anyone would think it was unethical to suspend the accounts to protect other customers on the server. I received no response from Mr. Schnell for 66 hours. I don’t think anyone, except Mr. Schnell, would attempt to claim I was being unethical in this situation.
The conversation continued.
6:15:50 Josh Schnell: first, we’re planning on staying with you. We’re setting up a backup server, and a load balancing server. Second, I’ve paid until March 2010
6:16:18 Josh Schnell: What’s unethical is taking my 90 dollars, and cancelling my account without discussing any of this with me
6:16:54 Michael Barrett: Yes you have paid through March, and no one said your accounts would not be unsuspended. They were suspended when I realized there was an issue with the DNS, to make sure hackers could not get in. THAT is ethical.
6:16:59 Michael Barrett: I have cancelled nothing
6:17:03 Michael Barrett: I have terminated nothing
6:17:13 Michael Barrett: I was protecting you based on our CONVERSATION here on IM.
6:17:17 Michael Barrett: Nothing more
When we last spoke, I was under the impression he had been “hacked”, yet now we’re discussing a backup server AND a load balancing server.
When you set up a “backup server” you don’t necessarily redirect all of your traffic from your main server to that backup. You can initiate changes much easier than changing your name server at the registrar level to test the backup server. I’ve been in the business long enough to smell bullshit when it spews from someone’s mouth.
Maybe if I had heard from him before 66 hours passed, I wouldn’t have felt the need to suspend (not cancel) his accounts, and I might have been able to discuss it with him. I took the appropriate action with the information (or lack thereof) I had at the time.
The “backup server” story was bad enough, but it didn’t end there.
6:17:37 Josh Schnell: The DNS was faulty, and mail was not being directed.
6:17:45 Josh Schnell: For three websites
6:17:49 Josh Schnell: not just the alpha one server
6:17:51 Michael Barrett: The DNS was working perfectly based on the response I got from Media Temple
6:17:56 Josh Schnell: the eastscene account wasn’t getting mail
6:18:02 Josh Schnell: as well as another
6:18:24 Josh Schnell: there was most certainly a dns server issue
6:18:37 Michael Barrett: Not on my server there wasn’t
6:18:44 Josh Schnell: well, i’m glad to hear that
He claimed his alpha one account was not the only one with an issue. He claimed that he had issues with his eastscene account as well as an unidentified “other” account.
It’s funny how domain registration works. Each time you modify your domain record at the domain registrar, they keep track of those changes. From the moment you create the domain name until long after it’s cancelled, they have a record of each change made to the domain registration. Every change to the domain record is time stamped, and every domain registrar updates their “Last updated” record with that time stamp.
Remember, our initial conversation took place on January 12th, three days after the name server change was complete. The only DNS server issue that could have existed would have originated on his new server, not mine. The domain record clearly indicates that the domain name servers were not pointing to my server three days before this incident occurred.
He went on to state,
6:19:12 Josh Schnell: I wasn’t getting email… so I contacted my hosting company
6:19:42 Josh Schnell: I’m not sure what the heck warranted the post on your blog, or my account being suspended.
6:19:58 Michael Barrett: of course, you could have discussed that with me, rather than pretending that the dns wasn’t working because you had already initiated the change on the DNS. See, I know the dates and times the changes took place too. Our conversation was quite a while after you changed things… yet you pretended to not know why it was changed.
6:20:05 Michael Barrett: you pretended you might have been hacked.
6:20:21 Josh Schnell: you’ll never believe me, and I’m fine with that. But, I swear to god I thought there was a DNS issue
Apparently, he contacted the wrong hosting company, because as you can see in the image above, my server was no longer controlling his DNS. Before I continue any further, let me state that I have screen captures of the WHOIS records for each of his domains. I learned the following from those WHOIS queries:
- alpha-one.ca was updated on January 12th, the day all of this took place, which means the change was submitted before our conversation that evening.
- Seven other domains were changed before the 1st of the year.
- One domain was changed on January 3rd.
- Five domains are still pointing to my server.
Two-thirds of his domains were set to point to his new server days, even months, before he contacted me about this issue. He’s right, I’ll never believe him so he better be fine with that.
6:21:55 Josh Schnell: Which sites are on the server or not, isn’t the issue here. The issue is that I paid until March, and here I am with my money gone, and an inability to access client accounts.
Which sites are on my server was the issue. It still is the issue. I was told there was a DNS and email issue with alpha-one.ca, on my server, when the domain registration was clearly pointing it to another server. I was led to believe his account may have been hacked when he knew full well that he had initiated a change to the domain record that very day. He lied to me and wasted my time which could have been spent helping customers who actually needed something done. Which sites are on the server or not was, and still is, definitely the issue.
6:22:04 Michael Barrett: I could really care less if you were planning to move. I said so in the post, and i have said so with everyone I have discussed it with
6:22:16 Josh Schnell: Except me
6:22:23 Michael Barrett: You leaving is not my concern. What concerns me was the way you lied to me when we chatted the other night
6:22:24 Michael Barrett: Period
6:22:26 Michael Barrett: dude
6:22:35 Michael Barrett: You are acting like a victim. Look at your dns changes.
6:22:41 Michael Barrett: Macgasm was changed in OCTOBER
6:22:43 Josh Schnell: you can play the hurt card, but i’ll say it again…
6:22:48 Michael Barrett: I am not hurt
6:22:50 Michael Barrett: I told you that
6:23:03 Josh Schnell: then please restore my account, and files, in accordance with my payment.
6:23:07 Michael Barrett: Your account will be unsuspended sometime tonight.
6:23:11 Michael Barrett: Excuse me
6:23:18 Michael Barrett: Your payment does not guarantee anything
6:23:35 Michael Barrett: if I suspect something nefarious is taking place I can suspend or terminate accounts
6:23:38 Michael Barrett: read the terms of service
6:23:40 Michael Barrett: BUT
6:23:43 Josh Schnell: Dude, really? I’m hosting with you for over a two years now.
6:23:45 Michael Barrett: I will unsuspend them tonight
6:23:50 Josh Schnell: Thank you
Yeah, that’s waiting for a bit. I know, I know that I promised him his account would be unsuspended tonight but that’s going to have to wait until I remove all of the “special” features he was not paying for, you know, in accordance with his payment. Once I have those changes complete, I will be unsuspending his account(s) even though they don’t point to my server any longer. I’m just not sure what I should do about all that space that is being used and never part of the plan. I wonder what I can do about that in accordance to his payment?
Sadly though, it didn’t end there.
6:25:18 Michael Barrett: The DNS for Macgasm changed in OCTOBER. The change on alpha-one was submitted BEFORE you had the conversation with me. so how do YOU justify all the “WTF’s” when I mentioned the DNS had changed
6:25:33 Josh Schnell: i forgot about editdns…
6:25:40 Josh Schnell: here’s i’ll log in and get you a screencap
6:25:44 Josh Schnell: or at least try to
6:25:48 Michael Barrett: But YOU knew you had submitted the change
6:26:03 Josh Schnell: editdns had been the alpha—one dns since the beginning
6:26:05 Michael Barrett: and YOU led me to believe something nefarious was going on, hence the reason your accounts were suspended
6:26:07 Josh Schnell: beginning
6:26:16 Josh Schnell: i haven’t changed it
6:26:20 Josh Schnell: because they’re charging now
6:26:49 Josh Schnell: restoring my account. I’ll get you a screen cap
6:27:35 Josh Schnell: i don’t even know what email it’s going to
At this point in the conversation he directed me to an image he had uploaded (I am not posting that url here because you know that might not be ethical). Anyway, the dateless image showed a lot of dns entries at editdns some of which pointed to my server but it also had the following disclaimer at the top of the page:
“Your domain is currently suspended and will not resolve. Please contact EditDNS Support for more information”.
Wow. Maybe that’s why DNS and mail were not working for the domain or any of them hosted using his alpha-one.ca name servers? The WHOIS record I pulled today clearly shows that his dns record for alpha one had indeed changed on January 12th.
At this point, I should have ended the conversation. There was no need to keep rehashing the same information. It was clear he was not going to apologize for leading me to believe his account(s) had been hacked, and I wasn’t going to believe another word he said. Then he said,
6:32:53 Josh Schnell: see. i didn’t change the dns.
6:33:00 Josh Schnell: it’s been like this since the beginning
Without dragging this out much further, he went on to say that the reason he moved “a couple” sites over to his “backup” server was because he “panicked”.
6:33:29 Michael Barrett: You panicked, and moved ALL your sites to another server?
6:33:41 Josh Schnell: we have a backup server
6:33:47 Josh Schnell: for a client
6:33:56 Josh Schnell: so i moved a couple sites over there
Funny thing that panic. It took place days and months before January 12th.
6:34:45 Josh Schnell: i changed them after our conversation
6:34:56 Josh Schnell: they were backed up there
The only site moved on January 12th, you know at the time of the “hack”, was alpha-one.ca. Was that some sort of “pre-panic”?
He claimed that if he was attempting to screw me over he would have never paid for three months, and he would have made sure everything was moved before then without saying a word. By the looks of things to me and all these screen captures, just about everything was moved before he uttered one word on the 12th.
At the end of the conversation we were discussing name server changes and backup servers. I pointed out that it didn’t make much sense to have a backup server where you had to update your name server record to allow access to the backup because propagation of the name server change can take up to 72 hours.
6:39:50 Josh Schnell: when some clients rely solely on the Internet for income, relying on one server is a bad practice
6:41:06 Michael Barrett: I know it is. But name server changes can take 72 hours to propagate through the Internet. so using a backup server at a different location rather than at one where the IP for the name server can be changed quickly eliminates that as a suitable “replacement” during down time for one server.
6:41:26 Josh Schnell: “upto” is the key word there
We both agreed it usually doesn’t take that long, in fact it’s much quicker. And then he added,
6:41:33 Josh Schnell: servers update a lot quicker
6:41:37 Michael Barrett: not always
6:41:44 Josh Schnell: alpha-one switched in 20mins for me
Wow. The same alpha-one.ca account he said he didn’t change was switched in 20 minutes when he changed it.
Like I said in my original post, and reiterated at the beginning of this post, I am not angry that he chose to change hosting servers. If was angry about that, I would have just terminated the hosting accounts. All of them. If I was angry that he changed hosting servers I never would have made my original post or this one.
I am angry because I was lied to. I am angry because I wasted 14 hours trying to solve a problem that did not exist. I am angry because I spent two years hosting his sites, kicking in extras along the way, only to be lied to, not once, not twice, but several times over the course of both conversations.
So why should I believe anything he stated in either conversation?
Throughout tonight’s conversation I was told that all this occurred because he was setting up a ‘backup server’ yet it turned out to be a “backup server for a client”.
I was told to restore his account and files in accordance with his payment, even though I had suspended his account(s) in accordance with the terms of service because I was initially under the impression his account(s) had been hacked.
I was told he never made changes to his dns for the alpha one account and that he ‘panicked’ and moved a couple sites on the 12th even though all but one site were moved long before the 12th.
At no time today did he apologize for leading me to believe his account(s) had been hacked. At no time today did he offer any explanation as to why all of the accounts were moved before the 12th. At no time did he apologize for lying to me. That is the most disappointing part of the whole thing.
When confronted about the entire ordeal he disregarded every statement I made about the “hack”, yet he was most adamant about being “paid up until March”.
Forget the fact that I busted my ass to make sure his data was secure. Forget the fact I spent 14 hours searching for clues. Forget the fact that I have spent the previous two years making sure he had the best customer service you’ll find in the web hosting industry.
Well, I’m done.
It’s obvious that he is not going to man up and apologize, but that doesn’t matter anymore. He proved his worth today, and that’s not saying much at all.
I have made significant changes to his account(s) removing every “extra” I have added over the past two years and restoring all of his space and bandwidth limitations per our original agreement. He paid for three months of hosting, and now that I know his account(s) were not hacked, he will get the remainder of his three months (as long as he does not violate our terms of service). He will be billed accordingly for any extras and overages that may occur on his account, just like every other customer hosted on my server.
Since money is so important to him, I will honor his subscription term, at which time his accounts will be terminated. Too bad you can’t buy ethics, I’d gladly refund his money if he promised to buy some.