Modular Machine Learning for Model Validation

QuantUniversity partnered with PRMIA to do the first QuantUniversity summer school in Machine Learning and AI in Finance. We had more than 1000 participants from more than 20 countries including India, China, Australia, UK, Turkey, South Africa etc. attend the summer school series.

This year, we offered 3 courses in Data Science, Machine Learning, and Model Risk Management:

  1. Just Enough Python for Data Science
  2. Machine Learning and AI for Financial Professionals
  3. Model Risk Management for Machine Learning Models

In addition, we had 10 lectures from eminent quants, innovators, and thinkers on various topics in AI/ML and Fintech related topics.

Lecture 2

In Week 2, we had Dr. Joseph Simonian from Autonomous Investment Solutions discussed his work on Modular Machine Learning for Model Validation. Here is a summary of the workshop.


Implementing model validation through a set of interdependent modules that utilize both traditional econometrics and data science techniques can produce robust assessments of the predictive effectiveness of investment signals in an economically intuitive manner.

The proposed methodology, modular machine learning, also answers a number of practical questions that arise when applying block time-series cross-validation such as what number of folds to use and what block size to use between folds.

It is possible to re-interpret the Fundamental Law of Active Management into a model validation framework by expressing its fundamental concepts, information coefficient, and breadth, using the formal language of data science.

In this talk, we introduce an approach towards model validation which we call modular machine learning (MML), and use it to build a methodology that can be applied to the evaluation of investment signals within the conceptual scheme provided by the FL. Our framework is modular in two respects: (1) It is comprised of independent computational components, each using the output of another as its input, and (2) It is characterized by the distinct role played by traditional econometric and data science methodologies.

Slides, demos and videos at:

If you want to try out the demos yourselves on the QuAcademy:
Use ‘QUSUMMERSCHOOL’ as Registration Code



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store