A third choice is to put an entire bootable PC session on a USB drive itself or on a Live DVD, so that you can run whatever OS and associated programs you like. The trick is figuring out what OS is small enough to fit on a removable drive, although now that USB drives can be found with 16 GB or more capacity that isn't so much of an issue. You also have to make sure that your borrowed PC can boot from a USB or DVD drive too, which is usually set up in the PC BIOS settings.
There are versions of Linux, including Damn Small Linux, that are very tiny and capable.
And you can also make use of Google's Chrome OS too. Chrome OS is not to be confused with Google's Chrome Web browser, although there are some similarities. The Chrome OS is fairly limited, and is really what you get when you just have a browser as your entire OS. Here is one place that can offer some advice and sample instructions on how to do this.
Then there are products that are designed to use a virtual PC session on a USB key drive that can be setup to your liking. MokaFive and MojoPac.com are two such tools. To assemble your virtual PC, you will need the MokaFive Creator software, which is Windows-only. You start with a base physical machine, such as from the Windows distribution DVD or an ISO image file. You convert this into the LivePC MokaFive format and then run it, making any other configuration changes and adding or deleting any applications, then finally package and upload it to the management service. Once you go through these steps, you can run the virtual PC on either Windows or Macs with the MokaFive player software.