Since our underlying product offering is around shipping copilots for business functions in financial services (wealth, asset management, insurance), there are possible step-changes we want to explore in every component and for that reason we are exposing our framework.

At OnFinance, we wanted a clear path to technical permeation for generative AI within our financial services clients along with expanding the scope for upsell opportunities at the enterprise level. So we built out a generalized copilot framework for Gen AI copilots in finance in order to guide both our attention and energy when it comes to building and shipping for new business functions. The most recent version of this framework is illustrated below:

       Fig 1: Our generalized copilot framework for all BFSI clients. Focus on what is underlined and in bold

To showcase the power of this framework, let me show you how straightforward it is to build out a copilot for financial research just by implementing components in the abstraction shown above**.**

• Public Data: For financial research, public data includes price feeds, company filings (see here), opinionated content (news and social media), educational content, rating agency reporting, and regulator/exchange guidelines on investor compliance. To curate this information, we built a generic crawler that both discovers new domains and scrapes them regularly.

  • Parameterization: From this type of content, we want to parameterize the following information: financial entity, financial topic, investor sentiment, market impact projection, source credibility, investor sentiment, author credibility, and impact duration. For this, we built out our NLP stack that has 94% accuracy on this task.

• Enterprise Datasets: For financial research, analysts normally work with 3 types of datasets: customer financial profiles (current holdings, past transactions), customer interactions (email, mobile app, call, WhatsApp, and so on), and in-house curated datasets (research data, risk models, customer reports, and so on). Considering both the sensitivity and the datasets’ centrality to the company’s moat, our customers won’t share this data for model training.

  • Synthetic Replication: What we can end up doing here is straightforward, we can take the client’s data catalog from them in an annotated format where relationships, semantic meaning, value ranges, and significance are defined for each column in an Excel or a page in a PDF. We can then use that to query the client’s data with an API at runtime and use relevant chunks of context to answer the analysts’ questions.

• Large Documents: Most documents in finance are very long in size i.e. 100+ pages (>300k tokens) and therefore holding context across such a large number of pages is very important if accurate inference needs to be delivered from far away chunks.

  • Ingestion and Parsing: Once we break information down into chunks, we rearrange them to build a sort of B+ tree in a bottom-up manner that preserves order and topic semantics across different levels in the B+ tree. This makes sure that relevant information in discrete chunks is correctly retrieved through an LCA approach.

• Source of Truth: The source of truth will be a sprawling knowledge graph with foundational-level concepts for financial research at the center of the graph and more transient entities like corporations, financial products, and key promoters at the edges of the graph. In addition, the most frequently used entity-relationship links in the graph are also embedded in vector space to allow for a more linear search approach and then depth can be further tuned.

• Initialize Task Context: While you could operate the LLM for a 0-shot inference, it makes a lot of sense to leverage in-context learning for an n-shot inference. So you can use the system prompt to initialize the context of a task for an LLM.

  • In-Context Learning (ICL) Examples: It makes sense to reduce the number of tokens used for ICL by training a multilabel classifier to pre-classify the task into 1 of 5 types (multi-document inference, real-time price, user-context, private data, opinionated data) and maintain separate ICL templates for each of them.

• nth Inference: Interfacing between the research analyst and the LLM happens across a chat history of roughly 12-18 charts per report generated depending on how specific the analyst is in generating any report. So at every intermediate step, certain checks need to be implemented.

  • Source Grounding: The general approach we have is mentioned on the following page but for a research analyst it is mainly focused on building data lineage and data provenance for every numeric data and number-driven inference returned by the LLM.

  Answer Grounding

  • Guardrail Interactions: Within any 2 interactions the main thing that needs to be curtailed post-response generation is making sure wrong information or wrong files do not get included in the chat history including inference level.
  • Outcome Alignment: Between any 2 questions detect whether the analyst is staying on topic and if they are not, the LLM will generate 2 follow-up questions to ensure that they do not get lost from their report generation tasks.
  • Output Parsing: The formats that are must-haves for financial research are CSV, SQL, code (charts), and JSON. We can start with langchain’s Pydantic parser for the v1.

• Post-Task Generation: This is relatively straightforward as it just involves the analyst providing NeoGPT with a list of relevant question-answer pairs to generate relevant PPT MACROs and convert it into a PDF once the analyst marks it as complete and all compliance checks pass.