I am going to start off with the company I do have a lot of respect for. New Relic emerged on the APM scene with the launch of it's SaaS portal in 2013 (it was founded in 2008). The main talking points anyone positioning New Relic will be all focused around data. It is pretty simple to see that their main target is what they have aptly named "data nerds". Data science is a huge driver in terms of arenas such as big data. Data and understanding end users helps increase conversion rate, quickly identify performance problems and all around makes organizations more efficient.
New Relic positions its platform into 7 components. They are APM, Mobile, Insights*, Server, Browser, Plugins, Synthetics. Insights is the component I am personally excited to see go GA since it will potentially be big data analytics as a service. So without further ado, here are some of the questions you should ask New Relic:
How do you correlate different data points?
This can be phrased in multiple different ways. Another way of thinking of this would be: if a uses calls in and reports a problem, can I find that end to end transaction? Answer will be "no" to that second question. Most vendors struggle with this and use time-stamp based correlation to "follow" call stacks or service calls from one area to another.How do you license data?
With the onset of Insights (again, I saw the platform and it is pretty slick), there are often questions around what can be done with this big data analytics. As of right now, you will need to pay for each of the 7 components listed above. It does appear that New Relic will be double dipping when it comes to Insights. If you have an APM agent (which you will have to pay for) you will also have to pay for storage of those specific calls in Insights. Its your data, but you gotta pay the toll for using it?How do you grab custom metrics?
Everyone says they can grab custom metrics. Have these guys show you what is needed. The biggest headache I hear from New Relic customers is the complexity it requires to do something as simple as grab a method argument or create a custom dashboard. Outside of the insights platform, I have even heard grumblings around alerting. This is a real scenario that happened at a customer. Lets call the customer George:George: "I just bought New Relic. Can Dynatrace do custom dashboards and alerts?"
Bill: (Looks confused because this is peanuts when it comes to features/functionality of Dynatrace) "Yes."
George: "Great! Prove it!"
Bill: "K."
15 minutes later
George: "I'll buy"
One of the corner stones New Relic stands on is all focused around data, but without Insights (AKA additional costs) the platform does not appear to do any sort of customized dashboard.
How do you integrate with non-production environments?
This is key for companies who want to promote better code to production (otherwise known as every company). One of the biggest cons I see with the platform as a whole is the fact is its focus on production environments. This is due to the fact that the data collection is only powerful for large scales of data which typically do not happen in non-production environments. Yes, New Relic may have integrations with CI tools or even load generation tools. But fundamentally, is the value shown enough to warrant an additional license purchase? Always be weary of vendors who give away test licenses for free. That means they have conceded to the fact the value prop of their solution just is not there.Final Rant
New Relic can help identify a significant amount of performance problems BUT I have not seen it used in a triage scenario. The problem is, as much as they preach their data collection a lot of the data is filtered out and not stored within the environment. I am of the opinion to not trust the computer overlords with the ability to determine what is important to store or not. Not yet anyways.Back to intro
Continue to AppDynamics
Continue to Dynatrace
No comments:
Post a Comment