Every time I click on a folder and sometimes even icons on the desktop, windows installer starts and then tries to install Symantec Antivirus, which isn't even on my system anymore. If I click "cancel", my normal right click menu of choices appears.

Download the MSI clean up utility from HERE
Double-click the file to install the utility.

On the Windows taskbar, click Start > Programs > Windows Installer CleanUp Utility to run the utility

For each entry that begins with "CC," "cc," "Norton," "Symantec," "Sym," or "MSRedist," click the entry, and then click Remove.

Reboot - done