A policy is a setting that changes keys in a special area of the Registry. If a GPO that sets a policy is removed, the affected Registry key is automatically removed on the next GPO refresh, in essence, reverting the Registry setting to its default state.
A preference is a setting that changes a Registry key that is not in the policy area of the Registry, and is persistent. If a GPO that sets a preference is removed, the affected Registry key IS NOT removed automatically during the next GPO refresh. (This is also referred to as tattooing the Registry.) In order to disable the effect of the Registry key set by a preference, the GPO must be set to Disabled, and allowed to take effect on the clients. Once this has occurred, the GPO can be removed.
A policy is indicated in the Group Policy Management Console, and the Group Policy Editor, by an icon with a blue dot, whereas a preference is indicated by an icon with a red dot.