He was worried about inflation, and the minutes show that other members agreed with him in discussion (though they didn't vote for a rise in interest rates). And remember this was before they BoE members knew what was in the budget.
The budget will have made him even more hawkish - as I said in my previous post on VAT, it will add to inflationary pressures that are building due to the falling pound (and sterling as been falling since the election because the markets believe that the Coalition is going to tip us back into recession).

I have some sympathy with the need to normalise interest rates. If you look at the graph left, during the global recession of 2001-3, the UK dealt with the crisis by loosening fiscal policy, and while interest rates were cut, they didn't fall too low, and the monetary stimulus was removed pretty promptly. The USA though not only loosened fiscal policy, but turned on the monetary taps too wide, causing the massive commodity speculation that did so much to hurt the world economy, plus of course a housing bubble, which led to the credit crunch.
It might be smarter for all economies to have higher interest rates and a looser fiscal policy rather than the situation we have now, where everyone in Europe is tightening fiscal policy, but money is still extremely cheap, which means we have all manner of speculation going on, from attacks on sovereign countries to an oil price that is artificially high given global economic conditions.
The nightmare scenario is if uber-loose money causes inflation, and you end up with a toxic situation of tightening monetary policy at the same time as tightening fiscal policy. The Coalition are expecting the BoE to keep monetary policy loose in response to their fiscal tightening (see the comments from the OBR), but the BoE might not be able to deliver, especially when the government recklessly puts up VAT. You'd think they would have learnt from Thatcher's experience of jacking up VAT from 8.5% to 15% in the early 80's, but no!
Like most Labour people, I hate VAT with a purple passion; it slows consumption, it's regressive, it contributes to price inflation, and worst of all, once you put it up you can't cut it without the agreement of all 27 members of the EU (and that's unlikely to happen when many EU countries have higher VAT than us).
I've no idea why Tories are so enamoured of it. I think they believe that it's a stealth tax that no one notices - but it actually causes a lot of damage to the economy, (a lot of the ills of the previous Tory govt can be laid at the door of their VAT policy). I guess we are heading back to an era of low growth under Conservatives