Automating Mortgage Application Workflow With AI Agents for Direct Mortgage

Summary

Direct Mortgage Corp. (DMC) specializes in residential mortgage lending and loans and has been in the business for 29 years.

Their mortgage application workflow, however, was predominantly manual and time-consuming, resulting in a labor-intensive process for both customers and employees.

They partnered with Multimodal in order to automate it. We delivered the first working prototype, powered by Generative AI, in less than a month.

Today, DMC uses two AI Agents to automatically extract data and classify a wide range of documents. This lets them serve more customers, reduce costs, and increase employee satisfaction.

Mortgage Companies Need Intelligent, Multi-Purpose AI Agents

Mortgage companies require a lot of customer data to make informed lending decisions. This data is usually submitted in the form of different document types, including bank statements, paystubs, hazard insurance documents, and many others.

This is exactly how DMC received customer data.

• Customers would upload their documents and fill in additional information. They also needed to upload the right document in the right field, essentially classifying the documents themselves. 
• Employees then needed to manually extract, review, and verify that data before proceeding to the next step.
  

The workflow was mainly manual and extremely labor-intensive, for both employees and customers. The CEO, Jim Beech, has been trying to automate it for years, but with little to no success and underwhelming ROI.

“I've tried different people with [suitable] skillsets and several different entities. We even built our own programming team. But they could only get so far, and it took forever. (...) A 1040 tax return, for example. [It] took us about a year and a half to actually get that done.” - Jim Beech

It was virtually impossible to accurately process documents, especially unstructured documents, using pre-AI systems. 

Even tax returns, which are structured, presented huge challenges for DMC’s programming team and available technologies. Unstructured, or more free-form documents, like paystubs, required even more advanced capabilities.

This is so because one unstructured document can look widely different from another document of the same type, with the data often being found in varied places. 

‍

DMC needed a system that could do much more than just follow predefined rules. They needed an intelligent system capable of understanding the context the data is in, learning on its own, and processing a wide range of documents — even if they don’t perfectly match its expected templates. 

With that in mind, DMC had two main goals:

• Adopt versatile AI solutions capable of processing unstructured data and a variety of document types, with an emphasis on document classification and data extraction.
• Streamline the entire mortgage application workflow with a single system, instead of a number of disparate tools.
  

However, the mortgage industry is heavily regulated, and the stakes are incredibly high for both clients and companies; so, any automation systems need to be highly accurate and trustworthy.

They partnered with us to develop such a system, powered by Generative AI.

Model Customization Is Key to Better Accuracy, Versatility, and Trust

In order to develop a powerful AI system for DMC, we decided to customize OpenAI’s GPT-3.5. We chose it for several reasons:

• GPT-3.5 achieves a satisfying level of accuracy on finance-related tasks. 
• GPT-3.5 is also about 1.5x faster than its successor, GPT-4. 
• Additionally, it is also 30x cheaper than GPT-4. 
  

GPT-4 is generally more accurate than GPT-3.5. However, the difference between the two is negligible when it comes to data extraction and document classification — the two most important tasks for DMC. 

We felt that, overall, GPT-3.5 was the more attractive choice. 

However, in order to increase its accuracy and reliability, we needed to train it on DMC’s internal documents, especially unstructured documents. To do so, we performed in-context learning.

In-context learning

→ allows models to solve tasks they haven’t been previously trained on
→ aligns the model with the task at hand by providing examples of previous successful task completions

A brief overview of the process:

1. Feed the model a batch of relevant input-output examples

2. Define the context for the model (e.g., explain what type of document you want to process and what data you need to extract)

3. Repeat 

4. Feed the model the input document + ask for the output

5. Evaluate performance

6. Repeat the process if needed

Some documents we initially received from DMC were mislabeled. For example, several paystubs were labeled as bank statements, which led to a relatively poor performance of our first classifier model. 

To increase the accuracy, we needed to go back a few steps and ensure we were working with high-quality data. We were primarily focused on accurately labeling the documents. Here are the exact steps we took:

Data preparation

1. Data cleaning

2. Data labeling

3. Data validation

This work resulted in two models; one specializing in document classification, and the other specializing in both classification and data extraction.

AI Agents Automate Mortgage Applications at Human-Level Performance

At the moment, our AI Agents are handling over 200 different document types for DMC.

• Our classification model can currently process a total of 173 document types (including invoices, bank statements, and approval letters) + 9 rate sheets.
  
• Our classification and extraction model is, at the moment, processing 23 different document types, including various tax return forms, paystubs, and hazard insurance files. 

173 + 9 + 23 = a total of 205 document types

The models achieve human or even superhuman-level performance despite a relatively short customization period. Our first prototype, an AI Agent for paystubs, achieved incredible accuracy after just 30 days of development.

That’s why DMC decided to partner with us for a bigger AI project in the first place.

“We gave [Multimodal] the most difficult form to process, a paystub. That was our test pilot. I couldn’t throw anything more difficult at them than that. And in 30 days, they had it resolved.” - Jim Beech

According to DMC, this automation has led to a 20x faster time-to-approval and a cost reduction averaging around 80% per processed document. However, this is only the beginning of DMC’s venture into Generative AI. 

“We plan to expand this into many aspects of our business; other types of business processes, like marketing. It seems like Multimodal could allow us to make that happen.” - Jim Beech

DMC intends to keep adding functionalities until they achieve an almost-perfect all-in-one solution. The goal is to automate the workflows of different departments instead of just core business, maximizing efficiency and reaching true wide-scale automation.

While DMC may be pioneering such a forward-thinking strategy in their industry, complete enterprise AI is the likely future for many organizations. 

AI Agents work in synergy, and can enable departments to work synergistically as well. Sales and marketing departments, for example, can constantly exchange data with each other, and AI Agents can trigger actions based on data from both. 

This aligns perfectly with our big vision at Multimodal: stacking AI Agents for compounding gains, until an enterprise truly becomes an AI-powered organization. Based on current results, we confidently assume this will only lead to further increases in both productivity and customer satisfaction.