E-Mail:

Media Temple (mt) Grid Server Outage

Wednesday, November 29th, 2006
at 3:37am

For giggles (and scalability reasons), we set up a Media Temple (mt) “gs” account - setting up some of our sites on their new Grid Server service plan. This morning, it all went to hell. By the afternoon, I started to get antsy - and blitzed a request in their general direction. I received a response quickly, which pretty much said that I wasn’t the only user having issues on the Grid today. Then, in the early evening, I received this update - which I share with you now, in case you’re also a Media Temple gs customer:

This update is intended to summarize the many smaller micro-updates and progress that has been made with this issue since 4:25pm.

- Issues were discovered with the new firmware patch that was provided to (mt) Media Temple this afternoon. This new patch, which was designed to solve the stability issues with our storage segment, introduced new issues. The patch solved the crashing issues, yet introduced performance problems and possible incompatibilities with our GRID technology.

- BlueArc impressively has responded expediently throughout the issue and has escalated (mt)’s issue to the highest level within the company. Constant communication between engineering departments has remained all day.

- During the diagnostic phase (mt) Media Temple found techniques to dramatically reduce disk I/O load from the web segment. This improvement has been scheduled to roll into GRID Master Release (v.1.1). This action has restored the web segment to full performance while the root is issue is discovered.

- Performance issues in the email segment continue to come and go, currently exhaustive work is being done in this area.

- Commitment has been made by both teams to work continuously until permanent resolution is made.

I’m not quite sure what all of that means, but… I sure hope it doesn’t happen again. I mean, I don’t expect much for such a small amount of money, but Media Temple (in my mind) has one of the best Web hosting reputations on the planet - which is largely why we’re dipping our digital toes in their waters. Things seem okay again, although my Comcrap connection has made the Internets close to unbearable.

Tags
Categories

21 Comments

Kiley

November 29th, 2006
at 8:01am

I am a religious reader of many 9rules blogs, one month ago I switched over to the (gs) offered by MT based solely on the fact that 80% of 9rulers use them and that they sounded really reliable.

Well, I gotta say i am really dissapointed and I cancelled my account today, I just cannot afford all of the downtime that I have been experiencing.

I will most likely return when they get all of the (gs) bugs worked out, but until then, they really need to get it together to maintain there reputation.

Joel Burslem

November 29th, 2006
at 9:25am

I recently dived into the (mt) gs plan as well and while I love the concept - I’ve noticed it’s definitely been more buggy than any of my previous shared plans.

My site goes down regularly, not for very long mind you - usually a couple of minutes or less and then it’s back up - but enough to be worrisome nonetheless. Friends of mine have reported the same issues.

It seems Media Temple are still struggling with the grid server concept or maybe the demand. Whatever the problem, I hope they sort it out soon.

John Koetsier

November 29th, 2006
at 11:40am

Chris,

MT has been having on-and-off problems with their new Grid Server service ever since it was brought online a couple of months ago. I guess that’s to be expected with a radically new architecture, but it’s wearing a little thin, to say the least.

Here’s my take:
MediaTemple GridServer is a disaster

Dream hosting... A nightmare? · Textpattern Support Forum

November 29th, 2006
at 12:46pm

[...] Chris Pirillo wrote about the Media Temple Outage. [...]

Christian Burns

November 29th, 2006
at 5:59pm

I am in the process of switching over to MT GS, and I love the concept of the GS. I am hoping that they are past the bugs or soon to be past the bugs.

Mitch

November 29th, 2006
at 8:08pm

I’m a mt customer as well, and the downtime on the grid server has been ridiculous. I’m finding it hard to believe the gs will scale large sites like they say it will (see talkcrunch podcast where they say it can easily handle the load of techcrunch).

Ross

November 29th, 2006
at 10:18pm

I have a test account at MT as well and I’m averaging over 3 hours of downtime a month for static content, I hear if you have a dynamic site you may be expirience outages daily.

You can check out the article i wrote here:
http://www.thehostguru.com/2006/11/29/the-grid-is-fallingthe-grid-is-falling/

Ross
- http://www.thehostguru.com

Grid Server Outage at Media Temple » Mitchelaneous

November 30th, 2006
at 1:28am

[...] Related Hosting Links: + Chris Pirillo Was Burned By the Outage As Well + Here is a Thread About the Situation at Web Hosting Talk + News Coverage of the Event Via Netcraft Tags: mt, media temple, hosting, data center, problems, servers, news Posted by Mitch > November 29th, 2006 at 12:00 pm [...]

Sean

December 2nd, 2006
at 10:28am

The site went down again today and I called MT support and this was caused by the same Issue from Tuesday’s outage…

apparently, it wasn’t fixed…

This is really sad.. and I will definitely cancel my account if this happens again.

Dave Zatz

December 2nd, 2006
at 7:16pm

The theory of the ‘Grid Server’ system is a good one and could be the future of hosting. With that in mind, I registered *today*. My static pages are fine but my blog database continually loses the connection - not sure if the database service is dying or what the problem is. I opened a ticket so we’ll see how long they take to respond and what the issue is. FYI I have *not* changed my DNS settings yet since MT is essentially down for me now.

I also signed up for BlueHost and migrated everything there today as well and the server (with colated MySQL) I’m on has been up and down all day too.

How does someone get good service? Do I need a static IP and a server here in my apartment? It all makes me want to find a new hobby…

Hosting saga: support is everything at Growwwing

December 12th, 2006
at 1:15pm

[...] So Media Temple is not the best hosting choice we have made. Even if it is for our closed beta version, the missing flexibility of being able to be root for your server and the November 28 problems were deal breakers (read the comments in this TechCrunch post to learn more about the product and problems they had). [...]

The Media Temple Blog category they dont want to share with the public

January 11th, 2007
at 7:44pm

Notice how media temple keeps this category off the main blog.

http://www.mediatemple.net/weblog/category/system-incidents/intermittent-latency-and-unavailability-on-gridcluster1/

qatari

January 23rd, 2007
at 2:53pm

Stay away from the Grid! It’s the most unstable platform since Windows 3.11!

Our site http://www.qatarliving.com keeps on dying each night…we were better off on a cheapo shared hosting plan!

robert

April 5th, 2007
at 7:13pm

The Grid Server sucks!!!!!!!!!

4h 8m Down in Jan
4h 40m Down in Feb
15h 10m Down in Mar
8h 43m Down in just the first 5 days in April

Actual (mt) Media Temple (gs) Grid-Server Response Times

http://spreadsheets.google.com/pub?key=p-Ab3UaY590VDhZCTaDinHg

Suburbarazzi

November 27th, 2007
at 7:55pm

ll always be ready for intense bursts of traffic and the growing audience resulting from your online success.” Course, it didn’t quite work like that for us. And, as any quick Google search will tell you, we’renotalone. What was touted as “on-demand” hosting ended up being a lot more like a bunch of old Apple IIC’s strung together with twine and earwax. The good news: We’re finally switching over to a dedicated hosting plan from Media Temple

Harry

December 1st, 2007
at 9:12am

Media Temple Down Again

The Nov 30th “maintenance” was supposed to be 1.5 hours, then they changed it to 6 and now “they don’t know” I am seriously pissed at these guys. I wasted over a hundred dollars in PPC clicks going to a site that isn’t there.

This massive outage is unacceptable I am definetly moving my servers elsewhere…

ATP

December 2nd, 2007
at 1:25pm

This was really a deplorable performance by mediatemple - they already have a really bad customer service and this brings into question their technical competence too:
http://pakistaniat.com/2007/12/01/atps-disappearance-no-we-were-not-blocked-or-hacked-not-yet/

johno

December 3rd, 2007
at 9:54am

One year on, and Media Temple’s Grid Service is probably worse than ever. Yesterday my site was down for almost 8 hours. Today it’s off-line again and so is MT support. Looks like Media Temple is going into meltdown. It’s time to jump ship I think. No web site, no supprt…that’s not hosting, that’s a tvavesty, and I’m furious.

david

April 4th, 2008
at 4:22am

well, it’s march 08, and they are still having “teething” problems with the GS. Every time they “upgrade” us to the mysql burst container, something goes horribly wrong, usually in reverting us, they take out the site for several hours to a day. I’m not sure why I’m still with them.

anyone know if their dedicated service is ok?

What Do You Think?