As more facts, figures and graphs emerge from the organizations that both to compile such things, here's a bit more detail on what shouldn't really be called the "Krugman-Ferguson Debate," but for the sake of simplicity will be called that anyway.
Essentially the argument boils down to this: we're in a nasty recession and the government wants to spent a lot of money to make the recession a little bit less nasty. At the same time, the government already owes a lot of people a lot of money. Recently, interest rates on government debt have gone up a bit. So, as Martin Wolf summarizes:
Prof Ferguson made three propositions: first, the recent rise in US government bond rates shows that the bond market is “quailing” before the government’s huge issuance [i.e. those who buy and sell government debt are worried that all these fresh IOUs aren't going to find enough reciprocal lenders willing to pony up, which would push up interest rates and push down bond values, so as a self-fulfilling collective response, bond-traders are hesitant to buy up the new debt, worried that these investments would devalue]; second, huge fiscal deficits are both unnecessary and counterproductive [i.e. since U.S. government debt securities sit at the center of the world bond market, any increase in U.S. treasury rates ripple throughout the economy. Higher rates across the board make it harder for businesses to expand, home-owners to refinance and, in short, for the economy to recover]; and, finally, there is reason to fear an inflationary outcome [First, because all the government spending will push more money into circulation which is by definition inflationary and second, because when bond-holders start selling U.S. government debt and buying the debt of other countries, they are selling U.S. dollars and buying foreign dollars, which devalues the dollar, and pushes up the prices of imports and most commodities in general]. These are widely held views. Are they right? (Source: FT)
Martin Wolf says "No." So does Paul Krugman. And Brad Delong. And Brad Setser.
First, the concern that all this public deficit spending is pushing up on our overall deficit levels. Not to worry, says Setser, and he has the graph to prove it.

In fact, according to Setser, the increase in government largess is
more than offset by the decrease in private debt. In other words, companies are continuing to "deleverage" and households continue to be "responsible" faster than the Treasury can issue its securities.
Of course, the simple fact that we're only borrowing slightly less than last year isn't necessarily encouraging. This is especially true if many of our foreign friends are loosing their appetite for our collective dead-beatishness. Are they? Setser says:
Foreign demand for Treasuries — as we have discussed extensively — hasn’t disappeared, unlike foreign demand for other kinds of US debt. But foreign demand hasn’t increased at the same pace as the Treasury’s need to place debt. The gap was filled largely by a rise in demand for Treasuries from US households. (Source: CFR)
Which is where Krugman comes in. According to him, it would be one thing if China and Friends were becoming less willing to lend
and U.S. households were saving at 2006 levels (the technical term is "zilch"). But that isn't the case. U.S. households are suddenly, frantically saving a lot. That's kind of the key reason for this whole recession thing. So we have a lot more saving than usual and, because borrowing has decreased so severely, less private demand for it. Filling in that gap is Uncle Sam.
This is not to say that chronic deficit spending is something to shrug off, nor is it to say that we shouldn't start planning to slow things down once private borrowing starts to pick back up. But the idea that the recent increase (though by historical standards, it's really a normalization) or interest rates is a text-book case of "crowding out" is pretty questionable.
Martin Wolf again:
The jump in bond rates is a desirable normalisation after a panic. Investors rushed into the dollar and government bonds. Now they are rushing out again. Welcome to the giddy world of financial markets.
Which is to say that, yes, investors are becoming somewhat less willing to lend to the U.S. government, but this is relative to late 2008, the height of the financial panic, when, seeing potential default and economic demise in every stock, commodity, derivative and commercial bond, every Tom, Dick and Mutual Fund was suddenly willing to take close to nothing in return just as long as their cash was held by the most reliable borrower there is.
There could of course be some considerable undershooting in the bond market, since that market is only as saavy as its participants. If everyone starts to freak out about inflation and devaluation, rates could keep climbing. But if that is the case, I would imagine that the Fed will keep buying up Treasury bonds to keep rates low and wait until investors remember how high the unemployment figures really are.