I've been watching many top-end raiding guilds' progression through Naxx, mainly on worldofraids.com.
All of the bosses in each wing have been killed fairly quickly, but I haven't heard of anyone killing the Four Horsemen.

What makes this encounter so difficult?

Just go read THIS and don't pollute these forums with this garbage.