I'm working a lot with refactoring at job and it's clearly made a difference in my programming style and efficiency. Yes, refactoring is longer currently, especially that a lot of code haven't been refactored for a long time, but it clearly made the result more predictable and help to keep the ratio of bug/LOC really low; I have introduced only two bugs in all my work up to now, and have changes more than a thousand lines of code with really big changes and no unit testing. If I have those unit testing, I will have certainly catch the bugs sooner, since it was very simple typo that completly change the behavior of the application.
One thing that can both accelerate and make refactoring more efficient are good refactoring tools. Those tools are more and more common but and I'm looking for some good ones. When I was looking for one of those, I came accross this page, which talk about a tutorial called Refactoring Scripts 101. The concept is very interesting and I would like to see it integrated into a version control system. This could solve one of the problem of merge patches, which are particularly exacerbated with refactoring: line changes is just an approximation of the real change made to a file. The real change really concern more stuff like renaming a variable, or moving a code from one file to another. If we can describe those change instead, this would make merging far more easier. I think that Darcs try to do something similar but I have to check that.
For sure, not all changes can be describe like this. But if we can combined both, e.g. by incorporating an "apply patch" script, this would make our life really easier. I have an hard time however to imagine a diff tool able to identify the refactored difference between two repositories. Even with a common format, without such thing, we wouldn't achieved the Unix's rule of modularity.