Customer Group #1 is the corprorate market who don't care about the price and pay whatever it takes to get the job done. The XServe doesn't serve those guy's well as there's not a lot of corprorate software from Apple besides fileserving, webserving. The XSan and video editing market is probably all where Apple is really good at. But corporates want to run big clusters of Oracle databases (or MySQL or Postgres... etc). The XServe and MacOS X is not really well supported there by the vendors. Telecommunications market is (or was at least) in the hands of Solaris. MacOS X with its lack of SCTP has no future there. The corporate server market needs availability, uptime and guarantees. The first XServe's (still PPC) had a lot of missing things which are vital in the server market such as lights out management, remote console, redundant power supplies, RAID etc. The newest ones have a lot of that fixed, althought a remote console is still missing and sometimes even lights out management fails to work (seen that once or twice in last 5 years). So for the corporate market the XServe's price is "ok" but its feature set is limited. You can't run your 200GB database in memory as you can't put enough memory in it. There's no path to the high end.
Customer Group #2 is the hosting market. Those are the guys who buy 10'000's of servers at the time. The guys who host websites for millions of people and who run datacenters which take 8megawatts of power and the like. For those folks, offering MacOS X Server hosting would be an option as it can offer the small and medium enterprises options to use MacOS X Server features. Even if its only in a virtual machine but that's good enough for a lot of small to medium size customers. However when I look around what those folks are doing you immediately understand why they don't buy DELL, HP, IBM, SUN or Apple's XServe. They pile 100'000's of machines. Price DOES matter to those folks. So if you look at those guy's datacenter's you find a lot of noname boxes. They are inexpensive, can be ordered by the palette and if it fails, throw away and buy a new one. Its affordable. They are happy with the "limited" functionality of MacOS X but they would go for a Linux as it's not giving them an advantage to run MacOS X Server for 99.999% of their customers.
Customer Group #3 are the folks who develop under MacOS X and want to run their software as service for their clients. They buy XServes as it suits well into their datacenter. they are ready to pay the premium (+30% or so more compared to a normal Rackmount PC of same specs) because they use MacOS X's features which Linux can't give them. They use Cocoa, CoreData etc. etc. Those guys just love MacOS X and promote MacOS X heavily. They don't really need MacOS X Server, normal MacOS X will do. Some of them live of this exclusively. Those are the guys which Apple has just slapped into their face.
Conclusion: for corporate, apple's hardware is not good enough yet, for hosting its not cheap enough yet and the remaining market is too small for Apple to care about.
What Apple however forgets is that that "small" market very heavily promote MacOS X. Thats why the desktop is increasing because those guys make great software for it. This is what's making MacOS X unique. Its objectiveC, Cocoa and the whole environment. Apple now killed the opportunity to have its development environment enter the server market. Using MacMini's or MacPro's in the datacenter is just a joke. No serious business will depend on that kind of solution.
There is however a way out for Apple. Apple has to admit it is good at making software and laptops and desktops but not really good at making servers. Others do better servers such as Supermicro, IBM, HP, Dell, Sun and even Cisco with its UCS. For every type of workload, you find the perfect workhorse and you can run Windows, Linux, Solaris, FreeBSD etc on it. But you can't run MacOS X or MacOS X Server on it. The solution to Apple's disaster would be to license MacOS X/MacOS X Sever to hardware vendors who are doing a very good job in producing servers. As Apple can't serve that market (otherwise the XServe would not be pulled), others can very well and instead of loosing everything in that market, they could gain more market than they had before. Market they where not able to address before. So Apple could only win.
But as they fear their MacOS X would leak out to John Doe's Desktop that way and everyone would just steal MacOS X instead, hell has to freeze before Job's would allow this to happen.
Customer Group #1 will never even consider MacOS X for their corporation.
Customer Group #2 will never consider MacOS X neither
Customer Group #3 will be pissed off heavily and move away from Apple towards Linux
Result #4: Desktop sales of MacOS X will be reduced. This might not happen over night but a couple of years later.
I'm myself in Group #2 and #3. We now move our 500'000 lines of server code over to Linux. I wish I wouldn't have to do that. I started to love Cocoa over the last few years due to the fact it made me lots of things simpler which are a pain to do in C.
You might want to consider looking at http://www.savethexserve.com/ if you where thinking of buying an XServe.