What do you consider an appropriate "Error Rate"? I'm working with a client's IT department to develop a framework to improve its web and mobile digital properties. A recurring topic is the metrics that the business areas (mainly marketing) measure; one of them is "Error Rate," within the context of a customer-facing web and mobile application. The Error Rate can be a broad metric, but in general, within this context, it refers to the total of valid HTTP requests that return an error, divided by the total of requests. The client's IT department wants to define an initial achievable target for error rate, they want to know if there are any benchmarks or references of what is an "acceptable" error rate range.
Sort By:
Oldest
Global Chief Cybersecurity Strategist & CISO in Healthcare and Biotecha year ago
There isn't a one-size-fits-all answer to what constitutes the "best" or "appreciate" error rate for web and mobile applications because it can vary depending on factors like the application's complexity, user expectations, and the criticality of the errors. Ensure that you get with the legal department to see what is mandated in the client contracts. Get with the CFO to see if there is a mandate in the Business or Cyber Liability Insurance. Get with Compliance to see if there are other mandatory requirements within certifications, the sector, or where they do business. NIST and others have frameworks based upon best practices.VP of IT in Softwarea year ago
Tough question. Ideally, the visible error rate would be pretty close to zero. There may be errors that occur but these really should be effectively hidden due to retries (of things that caused) that happen automatically. Chief Technology Officer in Softwarea year ago
Depends on various factors.User Expectations: Start by understanding the expectations of your client's users. Consider conducting user surveys or collecting feedback to gauge what level of error rate would be acceptable to them. User satisfaction should be a primary consideration.
Industry Benchmarks: While there may not be a single "acceptable" error rate, you can research industry benchmarks for error rates in similar applications or websites. Benchmarking against competitors or industry leaders can provide some context.
Application Criticality: The criticality of the application should play a significant role in determining the acceptable error rate. For example, an e-commerce platform may have a lower tolerance for errors compared to a news website.
Impact of Errors: Consider the impact of errors on user experience, data integrity, and the overall functionality of the application. More critical functions may require lower error rates.
Service Level Agreements (SLAs): If your client has SLAs in place with customers or partners, the error rate target should align with these agreements.
Historical Data: Analyze historical data on error rates to understand past performance. This can serve as a baseline for setting improvement targets.
Risk Tolerance: Assess your client's risk tolerance. Some organizations may be more risk-averse and aim for lower error rates, while others may accept slightly higher error rates to prioritize other aspects of the application.
Continuous Improvement: Keep in mind that setting an error rate target is not a one-time exercise. It should be part of an ongoing process of monitoring, analysis, and continuous improvement.
User Impact: Consider the impact of errors on users. Some errors may be minor and have minimal impact, while others could be critical and highly disruptive.
Technical Constraints: Evaluate the technical limitations and constraints of the infrastructure and systems supporting the application. Sometimes, achieving extremely low error rates may be technically challenging or costly.
Cost-Benefit Analysis: Assess the cost associated with reducing the error rate. Achieving a near-zero error rate may require significant resources, so it's important to weigh the costs against the benefits.
Load and Traffic: Consider how error rates may fluctuate under different loads and traffic conditions. An acceptable error rate during normal usage may differ from peak usage.
Sr. Director, Head of Global Omnichannel Capabilities Delivery Center in Manufacturinga year ago
This is a hard one because not every website and mobile app are the same. For validated/GXP applications, acceptable error rate is 0% for us. However, for more marketing/sales related apps and sites (non-direct revenue impacting), I think the allowable threshold should be lower. I typically manage to a 99.9% uptime for these types of apps. Again, every scenario is different, and the business needs to provide some comfort level knowing that in a high demand and high change request environment (which most sales/marketing applications are), there is room for error if strict change control practices are not followed like with validated applications. So, the more agile and quick, the more errors that are prone to happen. In the end, both IT and business need to agree and be comfortable with the allowable error rate.Global CIO in Consumer Goodsa year ago
Percentage of requests is not the right way to look at it as it depends entirely what the failure is. If you have one failure that exposes customer credit card information out of a million requests, that is utterly unacceptable. On the other hand, 10% of calls resulting in a failure to show the correctly updated time is probably ok.