This corporation runs a high volume identity verification system, servicing AML and sanctions compliance, point-of-sale identification, and gambling compliance needs.
Having previously suffered outages at times of high volume, they asked us to identify and eliminate the causes. The installed hardware was apparrently sufficient, so the system was analysed completely from end to end.
We analysed the behaviour of individual components of the system:
Using existing load tests to drive a profiler, we identified hot-spots of high processor usage. These were rewritten or optimised where identified, and this resulted in a 30% reduction in processor usage for application servers at any given load.
The most time-consuming SQL queries in the system were identified. These were each analysed and rewritten, sometimes in several different ways, and query plans, IO usage, CPU usage and elapsed time analysed to identify the most efficient plan. Where possible this was done without creating additional indexes.
A legacy C++ system suffered occasional crashes, and slow startup.
We analysed the code using static analysis tools, and identified unsafe casts, off-by-one errors, use-after-free, overflow, and such typical issues which were fixed.
Slow startup was improved by preloading data where possible.
A complex interaction between thread pools and request queues in re-entrant ASP.Net code was identified.
This was mitigated by empirically adjusting the thread pool, incoming and outgoing request queue sizes, while using load tests to determine throughput.
Investigation determined that SQL IO was a significant bottleneck. This was mitigated in several ways:
The project was a complete success.
We will do our very best to help you to solve your problems, no matter what the cause.
Contact us now on info@beetlesoft.co.uk
