Lead 0574

Outreach

Hi [PERSON], found [ORGANIZATION] researching how FinTech AI teams run document workflows in production. Curious: is the bottleneck still extraction accuracy, or has it shifted to traceability, eval & edge cases? Would value your take.

Lead

I intended [ORGANIZATION] to be a dSaaS(data science as a service) to aid companies eliminate decision latency. But having worked with a heavily regulated client(not finance) in my previous role before, I would say the biggest problem is the LLM math problem, and traceability for the numbers it uses. I personally don't recommend a client spend a lot of effort chasing the performance on edge-cases as the reward ratio is a diminishing return after a certain point. I believe with the current standard of LLMs right now and AGI being atleast 5-10 years away, we need to focus on striking the right balance between classic ML, traditional code and LLMs to address each other's shortcomings.

Lead

I feel it's just enough to classify edge-cases well and use them and the borderline edge-cases by classifying conservatively to feed a retraining loop. In sensitive fields like finance it's very vital not to lose the trust of users and regulatory agencies alike.

Lead

Also on extraction accuracy l, it would only be challenging if they have need a pipeline for new documents(types) coming in continuously. Else it's an easy nut to crack by using a hybrid approach.

Lead

For me the only reason I use documents are when I need to consider contracts/SLAs etc to enforce business rules and limitations, all my docs are in a vector database and are fetched depending on the ask. And I always route the fetched chunks to the agents to give them business context to help them make decisions.

Lead

I have an eval framework that saves things to the background now and it's a manual retraining loop I haven't automated yet

Lead

And it's not just business rules validation, I also check every number(like pricing for suppliers) from contract data twice to make sure we are not using any numbers created out of thin air

Lead

I also salt hash the supplier identities before they hit the LLM and rehydrate them at the UI layer again as an additional privacy measure for folks who are concerned about privacy in a secure cloud private deployment.

Coverage note. This page shows only messages available in the outreach export. Later replies or follow-ups on LinkedIn may not be represented. Personal, organizational, and contact details have been removed.