Neural Network for Emulating Credit Risk Assessment

Ilona Jagielska and Janusz Jaworski
Department of Information Systems, Monash University, Australia
ijagiels@cfit.monash.edu.au

Abstract

This paper reports on an application that uses neural networks to emulate and analyse the decisions of the current credit card risk assessment system of a major financial institution. The application is a part of a larger project that investigates the use of neural networks to support decisions in the area of credit risk assessment. The main objective of this project was to develop a neural network system capable of helping in credit card application assessment and analysing the performance of the credit card application assessment. Two prototype neural network systems have been developed. One, described in Jagielska and Jaworski [Jagielska and Jaworski1996], attempts to predict the performance of credit card accounts based on the accounts historical data, another, a neural network assessment emulator described below, is a neural network trained on past applications, that attempts to mimic decisions of human experts and the existing credit scoring automated system.

The purpose of the neural network emulator was to provide the branch managers with assistance in their preliminary assessment of the credit card applications. The research was conducted on three credit card populations Visa Gold, Visa Classic and Mastercard. Since Visa Gold applications in this institution were assessed by experts this population gave the opportunity to compare the performance of neural networks with the performance of human expert. Credit risk of Visa Classic and Mastercard was assessed by an automated system using credit score cards developed by an external agency. In this case the emulator was seen as a chance to independently evaluate the risk of the applications.

Application

Neural networks' ability to learn and generalise from examples has been utilised in building the neural network emulator. The network paradigm used in this application was backpropagation. The neural network software used in this research was HNC ExploreNet and HNC KnowledgeNet. It was selected because KnowledgeNet is capable of some explanation of a network's decisions.

The variables selected as input to the neural model were similar to the information normally used for credit scoring risk assessment in the institution. The output consisted of a simple classification of good and bad accounts. The samples of 2300 Visa Gold, 4000 Visa Classic and 4000 Mastercard have been selected randomly from the cardholder database. The method of sampling in this study was similar to the method used to develop score cards for the institution. The selected data was subdivided into training, validation, and testing sets. Several neural network architectures were tried for each credit card population. After training and testing the decisions made by the networks were compared with the decisions made by the existing system. An application was considered to be classified correctly if the classification of a neural network was in agreement with the decision of the credit scoring automated system in the case of Visa Classic and Mastercard, and with expert judgements in the case of Visa Gold.

The best accuracy achieved was: for Visa Classic 92.23% for accepted applications and 78.43% for rejected applications, for Visa Gold 82.41% for accepted and 80.47% for rejected applications. For Mastercard the accuracy was only 70.14% for accepted and 67.62% for rejected applications. The analysis of disagreements between the decisions of the current system and the neural networks was performed using decisiveness and certainty indicators provided by the KnowledgeNet software and detailed analysis of the data and history of the misclassified cases.

The analysis for Visa Classic revealed among others that in many borderline cases the managers pre-declined or pre-approved the applications. The accuracy of the neural network classification would have been 97% if the pre-approved and pre-declined applications had been removed from the testing sample.

For Visa Gold the discrepancy between decisions of the emulator and the experts was in many cases the result of missing financial data. In such cases the financial experts who had assessed the applications must have had more information than that available in the cardholder applications. In some cases the neural network was more consistent in its decisions than human experts as applications with similar characteristics to those already accepted were rejected by human experts and accepted by the neural network.

The cause of the poor results achieved for the Mastercard was the consequence of using a score card designed for Visa Classic applications. Experts confirmed that the populations targeted by both cards were different and the same score card should not been used for both populations. Further they suggested that in such situation the applications which should been accepted could have been rejected and vice versa.

Conclusions

The neural network credit risk assessment emulator has satisfied the main requirement to successfully emulate decisions of either the credit scoring automated system in the case of Visa Classic or expert judgement in case of Visa Gold. The emulator was also useful in analysis of decisions made by both decision sources. The research proved that it is feasible to build a neural-network based tool which can be a valuable aid in credit card risk assessment.

References

Jagielska and Jaworski1996

Jagielska I. and J. Jaworski, 1996. `Neural Network for Predicting the Performance of Credit Card Accounts', Computational Economics 9(1), 77-82.