I agree, to some extent. Last year, I thought of a scenario I would like to see as a reality: your phone holds all the documents, code, pictures, … you care to have. It has a basic computing package: a small, reasonably high res display; a decent CPU and GPU; a usable amount of RAM.
You come home, arrive at work, you drop it into a desktop docking station that provides extra memory, a replacement CPU & GPU (same instruction set, different performance), and you’re good to go. You could have tablet and laptop docks as well, much like the ASUS PadFone and Motorola Atrix, but with a decent software and some standardized hardware interface backing them.
Also, I’m not sure if performance is still leapfrogging as fast as that graph seems to indicate, or maybe the software is not taking advantage of it. I bought an LG Optimus 2X about 18 months ago, and I am yet to see a phone that is overwhelmingly better in terms of performance. The only thing I miss on that phone - as a phone, not a desktop replacement - is a good keyboard.
Perhaps this new generation of 2GB memory phones with quad-core Cortex-A15’s will make a convincing upgrade, but I’m not sure they need to be, until phones can grow beyond their current physical size constraints.