You can alter or install anything onto your phone or tablet such as UI hacks (like the 5-icon dock) or unofficial apps (like classic console emulators) that would not otherwise be accepted into the App Store. Saurik's alternative app store, called Cydia, is arguably the definitive repository of such apps or tweaks. Cydia is so ubiquitous in iOS jailbreaking circles that most jailbreak tools install it automatically.
iPhone hackers first coined the term "jailbreaking" in reference to breaking the iPhone out of Apple's iTunes "jail." The term has since been (somewhat erroneously) used in a more general sense to refer to similar hacks made to other platforms like Android, where it's also known as "rooting". In any case, the basic concept is the same, regardless of what you call it.
For better or for worse, iOS is a closed environment: Apps need to adhere to Apple's stringent guidelines to make it onto the App Store. To a certain extent, this restriction exists for your own protection and to maintain the overall quality of apps. There are some apps, however, that don't make the cut but arguably ought to--the WikiLeaks app for iOS comes to mind.
"When you go to the App Store and install an app, you click [the icon], it opens, you do stuff with it, you hit the home button to close it, it goes away," saurik says. "An app is a self-controlled universe hidden behind a single icon." Anything outside this simple design--like saurik's UI theming mod called Winterboard, or any other alterations to built-in apps--is strictly forbidden.
To Apple, an app is like a sandbox. Your app can only play within the sandbox, but if you want to roam outside of it, jailbreaking is your only option.
Why Not Jailbreak?
Jailbreaking has some inherent risks. Root access is necessary for modding, but it can also open the floodgates for malicious attacks. Although there haven't been many documented cases of this, one hacker did create a harmless Rick-roll worm known as iKee to emphasize the vulnerability of jailbroken devices that haven't been properly secured by users.
Additionally, because jailbreak apps don't have to adhere to Apple's guidelines, some users complain about general instability, increased data usage, or decreased battery life.