Sometimes the problem isn't that you can't delete a file, it's that you want to make sure that it's really, really gone. If you need to securely delete sensitive documents or files, use Eraser--an application for securely erasing and overwriting files and directories stored on standard mechanical hard drives. You can also user Eraser to overwrite all the empty space on a disk, making sure anything you've deleted in the past is truly gone.
Revo Uninstaller Portable
One of the simplest ways to get a gridlocked computer running faster is to remove programs that you don't need. This has two benefits: you clear up hard disk space, which can speed up your computer, and you reduce the number of apps running at startup, which can make your boot time shorter. You can uninstall programs by hand, but that's slow going and the programs aren't always completely removed. Instead, keep a portable copy of Revo Uninstaller handy to quickly and completely uninstall as many programs as you want.
One of the main reasons that older systems gradually slow down is the accumulation of programs and services set to automatically run at startup. Autoruns is a powerful app from Sysinternals that shows you every single process that will start with your computer, along with other common sources of trouble, including browser toolbars and shell extensions. Telling a process to not run at startup is as simple as unchecking a box.
If you use KeePass to store your passwords (and if you don't, you should think about it), it's not a bad idea to keep your KeePass install and password database on a thumb drive. That way, you'll always be able to access your online accounts, no matter what computer you're on. Even better, you don't have to worry about identity theft if you lose the USB drive, because the Keepass database is encrypted and requires a master password. Be careful about logging in to potentially infected computers with KeePass, though.
The other USB drive: SystemRescueCD
If you want to be really, truly prepared for a PC disaster, you can go one step further than just creating an emergency app thumb drive. By loading up a second drive with a Live install of the SystemRescueCD operating system, you'll be prepared even if your computer's OS is so FUBARed that you can't even log in.
SystemRescueCD is a free live operating system, meaning you don't have to install it on a hard drive. Just insert the SystemRescueCD USB drive into the computer, restart, and boot from the thumb drive. The Linux-based operating comes equipped with software that you can use to access the data on your hard drives and back it up across the network, along with various other handy-dandy utilities--basically, everything you need to fix (or at least recover) as much of your broken OS as possible. The makers of the distribution were even kind enough to provide step-by-step instructions on how to get a SystemRescueCD live drive up and running.