That was correct, when all variables got saved in one savedvariables.lua ... now that addons use their own addonname.lua in a savedvariables subfolder nothing gets deleted if an addon doesn't exist anymore.
My solution is, every now and then... I just open the savedvariables folder, sort by date/time and delete everything thats not from the day of the last wow login.
Now, there is SavedVariables per character too, some addons use it allready... so you have to check
The good thing is, now that every addon has it's own little savedvariables file, you no longer need to search through a really big savedvariables.lua and search what you need, eg for backups - you just can edit/copy the small file.