End user monitoring – Top priority, but metrics unclear…

12 Dec 11 End user monitoring – Top priority, but metrics unclear…

Let me describe a recurring situation in almost every organization. When companies deploy critical business applications requiring 99.9% uptime, it’s because users expect this level of delivery. But this is not the reality. The system is up and running and it is close to 99.9% uptime, but the end users are still complaining that the application delivery is delayed.

Monitoring end user experience is vital because the main objective of any application is to enrich the user experience. This is crucially important in banking and financial services companies which are highly dependent on technology and are characterized by
• Large networks with branches reaching tier III and rural areas
• Real time operation and transactions that take place 24*7
• Increasing customer demands
• Constant compliance to regulations by RBI, SEBI and IRDA
• Increasing competition
• Multiple modes of service delivery (technology drivers)

However, in a recent study carried out by ValueNotes and Anunta among CTOs and CIOs of bank and financial companies, we found that less than half could establish the link between IT and end user measurements.

end user mointoring

It is clear that IT heads understand the importance of end user monitoring, and most of them are attempting to measure end user metrics. Most of the metrics defined were vague while others had no objective measures but relied on user feedback. A CIO at a public sector bank had an interesting view on the reliability of metrics. He said,” If I look at the no. of problem tickets as one of the metrics, then we cannot gauge the end user experience correctly, as all the tickets may not be relevant.”

Some of the trends observed in our study were:

• Application performance measurements are very broad and the assessment is not detailed
• Most of the metrics signifies a reactive approach towards monitoring and they are not monitored regularly
• The metrics around user experience are gathered for incident reporting and problem solving rather than performance improvement
• Difference in IT and end-user measurement is a clear sign of end user dissatisfaction
• In case of companies who had outsourced their application delivery, very few vendors promised SLAs around application performance from the end user side

I believe that the Indian banking and financial services sector is way past the debate of ‘End user monitoring- a necessity or a beneficial option.’ Application performance management is getting more complex by the day while the expectations of the end user are rising exponentially. End user monitoring has to become an integral part of application performance management. I think that the job doesn’t end at measuring the end user metrics but it should further expand to
• Approaching end user monitoring proactively and on a real time basis
• Linking the end user metrics to business metrics
• Isolating problems and resolving them immediately before the end
users are affected
• Ensuring continued business process

Not monitoring end user performance is not an option. An IT Head of a brokerage firm said,” We are in an industry that needs to be active 24*7, if we don’t monitor end users metrics, it is like we are sitting on a ticking bomb.” Unhappy users’ means unhappy end customers and no financial services company can afford that.

To download a complimentary copy of the white paper, “State of application performance management in the Indian BFSI sector”, please follow this link

Sayli Vaidya

Sayli was a research analyst at ValueNotes, with a focus on healthcare and pharma.

2 Comments
  • Ambarish Kubair
    Posted at 13:48h, 14 December Reply

    The article is well presented and clearly puts down the rationale of end user performance monitoring. However considering your argument, it becomes evident that it represents only one side of the coin. The flowery words of this write up do dominantly morph the consequences of such end user performance monitoring. The important facets which are totally ignored in this article belong to the development, deployment and support of any such core banking application. Having an in hand experience of working in this domain, I would like to add few more things that could broaden the view in context of this topic.

    Point 1 : Working of any system does not just depend upon the uptime of the core application. There are several different factors like network, user end infrastructure; time bound service dependence and so on affecting the end user performance. These factors play major role in determining performance on user end. Rather based on an in-house study conducted in my previous organisation, it was found that majority of tickets raised (around 85 to 90%) were results of these factors. Aggregating this fact with required resources to cope with the ever increasing demand (as confirmed by your article) it becomes a near impossible task for any organisation to implement such monitoring steps to ensure 100% perfect service to end user.

    Point 2 : Banks and other financial services don’t work using single system. They have multiple legacy systems, sometime developed and supported by different third party partners. However they do have single interface using which, customers could access various services they provide. Behind the scenes, this interface triggers processes of different systems, supported by different companies, using dedicated different networks. Hence from the point of clicking a small button from user pc, till affecting the system of main financial institution, this process includes traversing from different networks and applications so far supported by different companies. Some of these companies don’t even have sufficient infrastructure and system in place to handle the huge influx of requests. Hence this makes it again, a near impossible task to implement such monitoring steps to ensure 100% perfect service to end user.

    Point 3 : Any organisation working on the face of this planet always consider utilitarian approach to perform their work. Even regulatory bodies like RBI and SEBI officially accept this fact and allow the flexibility in service quality. Remember the six sigma method in place for ensuring service quality? So if the organisation successfully provides clean service to 99994 requests in every 1 million requests, they are still within the limits of regulatory board. Also the criteria in place for this rating system, supports only those request that make up to the core system of the financial and banking organisation. Hence in context of point 2 above, if the requests is gone missing or rejected anywhere before entering the bank network, it is not counted as failure on their part. In other words, these regulatory bodies in a way encourage the non-implementation of such monitoring steps to ensure 100% perfect service to end user.

    Point 4 : The finance and legal costs required by any financial organisation to ensure 100% service to end user or implement monitoring is way more than even the profit margins of that organisation. Even the big players like CITI Group, Goldman Sachs, Morgan Stanley and J P Morgan have publically accepted that they fail to provide 100% perfect service to everyone all the time (in context of the technology, network, systems and third party application in use). Indian financial services and banking industry is way behind these organisations, hence it is really hard to even consider them to have relevant resources and finance to ensure 100% service.

    Sagar made a good point of having RUM in place to bridge this gap. However the main issue he did not point out was the fact that RUM plays its part only when any services FAIL. Systems like RUM are monitoring applications which in a way only count the request calls and number of failures. But there are two main disadvantages to them. They fail to precisely predict the future failures and they are used by organisation only in testing phase of SDLC. Hence when any system goes in production, these services could not provide any conclusive evidence which could indirectly help in improving the service. Thus again making it clear that it is near impossible task to implement such monitoring steps to ensure 100% perfect service to end user.

  • Sagar Kulkarni
    Posted at 19:36h, 12 December Reply

    Very well written. As an IT professional, I agree that monitoring end user performance is the key for building successful application. Technologies like Real user monitoring (RUM) can surely bridge the gap between IT and the end user. Really a good one!

Post A Comment

*

Show Buttons
Hide Buttons