Global Metrics has a proven methodology for model development and analytic services that we have refined over many projects of the selected technique or technology. We view the model development process as an interactive one in which teams from both Global Metrics and the client work closely together for the successful completion of the project and transfer of knowledge.
This team approach helps us to ensure that the models we deliver are both technically sound and appropriate for your business needs, and that your staff has the information and understanding necessary to make them an effective tool for improving your business decisions.
Our model development projects consist of the following phases:
Select analytic approach / technology
During any single project, a number of techniques can be used to analyze data, discover interactions, reduce the dimensionality, determine predictors, and develop a final model. The following table illustrates the types of technical approaches in which Global Metrics has expertise. This is not an exhaustive checklist, and the decision of which types of analysis are appropriate for the project is made by the analyst.
-
Type of Analysis
- Data Displays and Visualization
-
Data Structure Investigation and detection
-
Data Reduction / Pre-processin
-
Clustering and Profiling
-
Predictive Variable Generation
- Performance Inference
-
Predictive Variable Selection
-
Predictive Model
-
Validation
-
Statistical Tools
-
Techniques
- Log-linear, Chi-Square testing
- Factor Analysis and Principal Component Analysis, Interactions
- Cluster Analysis, Hierarchal Analysis, two step analysis, K-Means Analysis, Kohonen Mapping.
- Transformations, Derivation’s(Ratio Formulation),Optimal Binning and Tiling
- Traditional performance inference, Dual Score performance inference.
- CHAID, C5.0, CRT, QUEST, Random Forest etc
- Stepwise Regression - Logistic or Standard, Global Metrics Proprietary Techniques
- Logistic Regression, GLM, GL Mixed, Standard Regression Bernoulli Likelihood, Global Metrics Proprietary Techniques, Neural Networks - Back Propagation or variant, Decision Tree.
- Traditional holdout sample, bootstrap validation, on-going validation and monitoring
- SAS Base, E Miner, ENT Guide, SPSS Statistics, SPSS Modeler, MATLAB... R. Weka, KNIME , Rapid Miner.
- Processing of Data
- Model development
- Model implementation
- Validation
- Tracking, reporting, and model usage guidance
- Model Adoption Programme (Workshop)
Processing Data
Upon initiation of the project, Global Metrics and Customer will discuss data requirements based on the project goals and data availability. Customer will then send Global Metrics the data in the agreed upon format and with the documentation necessary to proceed with the project. For Offshore Global Metrics may request de-personalized data.
Model development
Generate predictive characteristics
Once Global Metrics has identified the development sample, the analytic team generates an extensive list of predictive characteristics which will be available for inclusion in the models. This information may come from any data source supplied that will also be available at the time new accounts or data points are scored. The analyst will employ a variety of statistical techniques to generate characteristics and evaluate their contribution to the predictive power of the models. The specific techniques chosen will depend on the type of model, the client’s objectives and the specifics of the project.
Perform segmentation analysis
Segmentation analysis helps determine the effectiveness of multiple models for an applicant population. This is done by partitioning the development sample into combinations of mutually exclusive and exhaustive groups (subpopulations or segments) and determining which model “family” produces the best results. Determining the "best" result incorporates quantitative analysis as well as operational considerations.
Select variables for models
Initial models are built and tested for completeness and, where required, intuitiveness.
Weights of Evidence and Strategy Design
The client may choose to be involved in Weights assignment meeting where Global Metrics will work with the client to finalize model weights. This meeting is used to finalize model development decisions and facilitate handoff of the final model. If requested, Global Metrics will provide the statistical formulas used to measure the predictive value of characteristics and of score distributions. These could include the K-S statistic, weight of evidence, information value, divergence, and lift curves.
Model Implementation
Global Metrics delivers all the instructions and backup material that you will need for a successful implementation. After these are programmed in excel or codes generated for SAS, SPSS or any other Open source, should be uploaded into a test region on your system, and some non-production applications and bureau reports run through the code.
Validation
In this step the analytic team validates the model’s performance to show that it is making consistent, correct and precise predictions. Typically we will hold out an independent sample from the development sample to perform the validation.
Tracking, reporting, and model usage guidance
When the models are used in production there are several critical tracking reports that must be maintained. Some of these are intended for immediate tracking of all applicants, and others are for studying delinquent accounts after they are booked. We will supply our Scorecard User Documentation, which contains templates and performance definitions. Whenever a change in data, sourcing, or applicant is detected, we strongly recommend that customers produce and carefully review their tracking reports. It may be necessary to modify your procedures, policies, or management parameters.
Model Adoption Programme (Workshop)
Global Metrics is pleased to provide a description of our Model Adoption Program which is a comprehensive knowledge transfer program that enables a client to develop their own team of modellers that upon completion of the program would be able to develop new models, validate and analyze existing models, etc in accordance to the Global Metrics Methodology. The price for this program is based on a custom quote depending on number of client personnel trained and duration of the training. A comprehensive price quote can be provided to Customer based on further review and discussion.