| spamhotmail@yahoo.com 2006-11-19, 8:10 pm |
|
http://console.hardocp.com/article....CxoY29uc29sZQ==
2:27 PM CST
------------------------------------------------------------------------
Date:
Monday , October 30, 2006Category:
Xbox 360Manufacturers:
MicrosoftAuthor:
Steve LynchEditor:
Kyle Bennett
Inside The Xbox Live Operations Center
[H] Console takes an exclusive behind the scenes look at the Xbox Live
Command Center with Jason Coon, Operations Program Manager for Xbox
Live.
continued...
Why are you located off campus?
Because we didn't want to make the folks at Office and Windows
jealous, I mean, if you are working on serious software would you
really want to be sitting next to a bunch of knuckleheads playing games
all day? Just Kidding The truth is the Xbox Team grew so fast, there
simply wasn't enough room on Campus to house all of us. I hear there
might be plans to move us back on campus eventually, but for now our
ever expanding team will stay where we are.
Is this a 9 to 5 kind of operation?
Not at all, especially not for Xbox Live. We run a service that is in
use 24/7. We have an Operations Center that is manned 24 hours a day,
and in addition we have many technical folks on call during off hours
and weekends. Because we are always working on the next version of Xbox
Live we also support all of our development and test environments too.
We all do whatever it takes, including working late nights and weekends
when necessary.
How large are the Xbox Live facilities?
For office space, Xbox as a group has three full multi-floor buildings,
and shares part of two other buildings in a corporate office park (we
aren't on the main Microsoft Campus) plus we have offices in Mountain
View, CA, as well as London and Tokyo. As far as datacenter space,
that's classified.
How many people work there?
I think that might be classified too.
We learned several interesting acronyms during our trip to the Xbox
Live Operation Center like XOC and MOM, what do these acronyms stand
for?
XOC is the Xbox Operations Center. It is the Nerve Center of Xbox Live,
its staffed 24 hours a day, 7 days a week. They monitor everything
about the service, including the network, the servers, and user
activity such as how many people are logged on at any given time. They
all have Xbox1 and Xbox 360 consoles on their desks, and they have
copies of every game ever published for both consoles, so if users are
complaining of a specific game problem, they can try and reproduce it
so that it is better understood when they escalate the problem to the
product team.
MOM is the Microsoft Operations Manager. This is a Microsoft built
system that allows large IT enterprises to manage many diverse systems.
It provides tools to test applications, as well as automated monitoring
and alerting. If an application throws an event, or writes to a log,
MOM can be configured to monitor it and report when it's behaving
beyond specifications. It's a great toolset, and allows us to manage
a much larger environment with fewer people than would ever be possible
without it.
Let's talk raw numbers...
Generally, raw numbers are classified, but I'll do what I can.
How many overall Live customers are there?
More than four million and growing quite fast every day.
How many people are on Live at any given time?
Closely guarded secret, but I can assure you, it's a lot and this
number keeps increasing.
The most ever?
Just about every day, seriously we don't even celebrate breaking all
time records anymore because it keeps happening so often.
Daily bandwidth usage?
Many gigabits per second.
O.K. then how about the biggest day so far?
38 Gb/s during the influx of people downloading the "Bringing it
Home" content of E3 this year.
What are the primary functions of the two screens at the front of the
room?
The XOC very closely monitors the number of users online at any given
time to ensure that everything is running smoothly at all times. They
also cycle through other various performance counters for different
server types, monitoring load and usage. They even have the ability to
dynamically re-allocate some servers if load is increasing to the point
that it's degrading performance.
------------------------------------------------------------------------
|