Neither device is particularly comfortable to hold when not in a case. In fact the iPad feels slippery and unwieldy when naked. I didn't buy a case for the Kindle because I didn't think I'd need one. I'll be rectifying that shortly. Pictures of the Kindle make the bezel look ridiculously large, but in practice it feels a bit narrow. I tend to kind of cradle the Kindle in my right hand with the lower right corner jammed into the center of my palm, my fingers on the back of the Kindle and my thumb on top on the page turn button (if you've seen the TV ad of the couple at the beach reading their Kindles, they hold theirs the same way). I haven't dropped it yet but I always feel like I easily could, and after a while my hand starts feeling cramped because the thing is so small. If the page turn buttons pressed in from the edges, rather than down from above, you could 'pinch' the Kindle with your fingers along one edge and your thumb along the other and it would feel very secure. Sadly it doesn't work that way.
Still, you can hold the Kindle with one hand for a long period of time. The iPad is too heavy to hold up unsupported for very long. And you can hold the Kindle very securely with two hands and not have to let go. With the iPad you need to move a hand to swipe/tap in order to turn pages. The iPad's body is slippery aluminum. The Kindle is some kind of matte plastic; it isn't as sexy but seems much more practical and it feels good under your fingers.
Now let's talk about the software. On the iPad, your Kindle home page is a colorful display of book covers. Tap a book cover to start reading. On the Kindle, the home page is a drab mass of text. You use the directional button to move a selection cursor around, then a center button to choose that selection. On the actual Kindle you can set up 'Collections' to organize your reading materials. For instance my girlfriend and I each have a collection to put the books we want to read into. A title can be in more than one collection, so if there's something we both want to read it can appear in both our collections. The Kindle app, at least at this time, has no similar organizational tools that I'm aware of.
Once you're actually reading (and that's ultimately what we're here for, no?) the differences are much more subtle. I was concerned that the page turn time of the Kindle would be slow enough to bother me, but it isn't. The 'bonus features' of the two experiences are very similar. Looking up a word, for instance. On the actual Kindle, you use the directional buttons to place the cursor at the start of any word and a definition will pop up at the top or bottom of the screen (whichever is less intrusive). On the Kindle for iPad app, you press and hold on a word to get the iPad selection cursor in order to do the same thing. Both devices allow you to bookmark and add notes to selections (the physical Kindle has a hardware Menu button that pops up a command list to do these things and more). Kindle for iPad has a cleaner display; you can get rid of everything but the text of the book you're reading. The actual Kindle always has a menu bar with the name of the book, battery indicator and so forth at the top, and a gauge at the bottom showing how far along in the book you are. Granted, this is a pretty minor point.
This is a silly thing, but one aspect of the Kindle I really like: when you put it in Standby Mode (which as far as I can tell, just disables the buttons) a random image appears on the screen. Generally it's a portrait of some major author. Right now it's Mark Twain. When your Kindle is sitting on your bedside table it looks more like a piece of art than a gadget.
So, actual Kindle vs Kindle app for iPad. Which is the better option? Which will I use more? I'm going to cop out a little bit and say that it depends. In bed at night I might lean towards the iPad. I don't need to have a bright light on so I won't disturb my girlfriend if I stay up late to finish a book. (I should note that Amazon sells a Kindle case with a light built into it, but it's $60 and the light doesn't look all that bright. I need good light to read comfortably at my age.) But during the day if I'm sitting in an easy chair or outside on the balcony it'll be really nice to have the Kindle.
There's also the price consideration: $500 for a base level iPad vs $139 for a base level Kindle. Of course the iPad can do much more (in fact I have the apps for the e-bookstores of Borders, Barnes & Noble and iBooks on my iPad, in addition to Kindle) so it's hard to judge based on price. If all you want to do is read e-books then by far the Kindle is the right choice. It is, unsurprisingly, very good at delivering text to you in a small, lightway package. And if you have an iPad, I'll go out on a limb and say you really don't need a Kindle unless reading off the iPad tires your eyes or you're a big outdoor reader. If I knew then what I know now, I might not have ordered the Kindle. On the other hand I do like it enough that I'm going to keep it. It's such an elegant little device that it's hard not to love it.