The Nexus 4's battery is technically nonremovable, by the way -- the back panel doesn't come off by design -- but if you don't mind tinkering and taking things into your own hands, you might find that opening the device up and swapping out its battery isn't as impossible as it seems.
If the Nexus 4 has one glaring Achilles heel, it's storage: The device has no SD card slot, and with internal storage going only as high as 16GB -- which comes out to about 12GB to 13GB of actual usable space -- that doesn't leave you with a whole lot of room. Google's focus here is clearly on moving people to the cloud and to a habit of streaming over storing, but not everyone's going to be happy with that setup. If high storage is a priority to you, the Nexus 4 may not meet your needs.
The Nexus 4 has an 8-megapixel rear-facing camera that, thanks in large part to its Sony BSI sensor, is capable of capturing great-looking shots both indoors and out. If you're coming from the Galaxy Nexus, it's going to be a major improvement.
The Android 4.2 Camera application introduces a refreshingly clean new minimalist interface.
I found the Nexus 4's camera to be excellent in well-lit conditions and darker environments (an area where the Galaxy Nexus's performance is particularly weak). The phone has a bright LED flash as well, and images taken with the flash looked natural and not at all washed out.
Shutter speed on the Nexus 4 is decent enough: I typically experienced one- to three-second delays between shots, depending on how much focusing was required. By default, the Nexus 4's camera takes photos in HDR mode, which quickly snaps shots at different light exposures and then combines them into a single image. The camera has settings for adjusting the exposure and white balance as well as a small selection of preset "scene modes." It can also record video at 1080p resolution.
The Nexus 4 has a 1.3-megapixel front-facing camera, too, for video chatting and vanity pics.
All considered, the Nexus 4 doesn't have the best camera you'll find in Android Land -- I'd say that honor still lies with HTC's One phones -- but it certainly has a very good camera that'll more than meet most users' needs.