Trade transactions data is pulled from Enterprise Oracle database into IIP using Sqoop and loaded into HDFS. Necessary processing logic and business rules were applied on the data to eliminate improper trades. Data deduplication, data validation, data enrichment and data derivation operations were also applied on the data to complete the data processing and analysis.
Report was generated and the data from the regulatory report was pushed back to Oracle. Various analytics reports can be displayed by Tableau.
PoC was generated with data volume of 596 million trade transactions on a 100-node cluster in AWS stack. Message insertion rate into IIP was achieved at 130,000 records/ second or 18.22 MB/ second. Report execution was done in 35 seconds for 30,000 trade records and corresponding 120,000 trade line items including end-to-end processing.
Bank currently needs 10 minutes on an average to report a trade. In 15% of the instances the 15-minute window has been overshot, which attracts a penalty. With IIP, the end-to-end processing and reporting just takes 35 seconds, avoiding the non-compliance and related penalties to the bank. IIP's solution is elastic and scalable supporting future increase in trade volumes with minimal cost increase to the bank.
IIP helped predict if an ATM is functioning or may fail in a week's time (over 80% confidence level) through alerts and incident data from XMS and machine data.
Ticketing data from 8500 ATMs / 4M records were ingested into Spark and then the cleaning was carried out (date, spaces, null fields, etc.) in just 27 seconds on 10 node AWS cluster (32 CPU, 64GB RAM, 640GB SSD storage).
Enriched the data to create parameters required for analysis. Spark Logistic Regression ran on top of this data with data prediction in 60mseconds.
The data was ingested to Oracle and the visualization was done using Tableau.
There was a 14.3% efficiency increase (from 3.5 to service calls per technician per day).
Client saw an 18% cost reduction from increasing the mix to 40% staged calls.
The time taken for chronic/ defect identification came down from weeks to days and then, ours.