Simon - the longer lenses, such as the 30-110mm on the Nikon 1 is still very compact, so it doesn't feel unbalanced at all.
I have to agree though, kind of. They are great for consumers and people wanting slightly more control/quality compared to a compact. But they aren't compact really. They don't fit into a pocket, and when they are coupled with a selection of lenses and accessories, you may as well be carrying a DSLR. They are also quite expensive for what you get, and the lens selection is small and also rather pricey.
I can definitely see them growing and beginning to take a larger chunk of the market, but i don't think they'll ever replace DSLRs in both the professional and serious amateur categories.