I'm not sure there's much of a security hole with sending information to New Relic; most of it will be dreadfully boring and useless information about which URL responds within how many milliseconds. New Relic does encrypt the data during the trip, but when it gets there, it's still in the control of another company. And then most of this data is recordable by outside visitors to your site, and they already know full well whether your website is working.
But some of the holes can be more subtle, and code can reveal secrets by mistake. For example, a supposedly confidential password might be used in the URL of some internal Web service call that would normally stay secure inside the firewall. Some JDBC URLs, for instance, include database passwords. Or the programmer might punt on adding more security in the name of time. Will this information find its way to New Relic's servers? It may be hard to know because the person implementing the monitoring may just pop in the agent JAR and never think about the sensitive data.
Compared to AppDynamics, the style of New Relic's performance data presented by the website seems a bit more traditional. The main page tracks the average response time, the throughput, and the Apdex score, a stat that roughly measures the percentage of users who get their information quickly. The speediness of each Web page is also tallied and graphed on another table. The free version shows the last 30 minutes. As with AppDynamics, if you want more information -- both more metrics and a longer history -- you'll need to upgrade.
More profiling information can be found by starting up a separate session. This records the amount of time spent on each of the major routines and subroutines in your system. In the past, I've usually relied upon the profiler with Eclipse to gather this information, but that's not always a good indication of what part of the system is exercised most with real client data in a working environment.
Some advantages of a central server become more apparent when you dig into deeper features, such as Web integration and the API. New Relic's central server will export your incident reports to other websites such as Campfire, Lighthouse, or Pivotal Tracker. The monitoring server can start up a discussion on its own. There's also a connection with Twitter, although why anyone would want to share deployment and incident notifications with the entire world is something I'll never understand. Kids today want to get their server information mixed up with invitations to drink and reminders from Mom that the root word in "smartphone" is a hint that there's a voice feature buried among those apps.