There is actually a specific policy to do exactly what you are asking. Look under User Configuration | Administrative Templates | System. The setting you are looking for is ‘Prevent access to registry editing tools’
If you enable this setting on your Windows XP systems, users will not be able to open up the registry.
Of course, if there are other specific applications you do not want your users to run, Software Restriction Policies (XP) and Applocker (Win7 and later) to the rescue!
Jeremy has a whole section dedicated to Software Restriction Policies and also Applocker in Chapter 8 of his book (third edition, with the gear cover), and it’s also covered with hands-on labs in his training classes.