I moved out my WTF and Interface Folders elsewhere so everything became clean and on defaults. Downloaded http://files.wowace.com/Cartographer/Cartographer-r74405.zip and it still happens, clicking the Castle icon of Orgrimmar on the Kalimdor map still sends me all over the place, as far as viewing Silithus instead.
Windows XP, and it was definitely empty since I had default UI and all as if I just reformatted :)
Tested some more, by splitting up those Cartographer folders with the included batch file, and then disabling them all at the char screen addon's menu, leaving just Cartographer intact (Battlegrounds, Foglight, guild positions, etc all unticked), and it still happens, The Cartographer tab on the World Map only had Profiles options there. Disabled Cartographer and clicking the Orgrimmar Castle icon brings me to Orgrimmar 100% now.
I don't understand why this isn't affecting anyone else. Can't be hardware affecting Cartographer now, cos that would just be silly. Even UI scale and video settings was at defaults