In finance, data is gold but it’s also sensitive. Keeping this data secure and private is not just a priority, it’s often a legal requirement. On-premises AI models live on the company’s own servers, meaning all that valuable data doesn’t need to venture into the wild.

Financial services companies deal with sensitive end-customer datasets and research findings that must be handled with utmost care and compliance. It is crucial for these organizations to have on-premise machine learning (ML) models to enable financial services organizations to expose their enterprise data fully within their controlled environment.

<aside> 💡 Most Indian customers are currently backing down by looking at the cost of single-tenant on-premise deployment for the Gen AI stack. However, as hardware improves and the gen AI stack for finance verticalizes further, the maths will be in favor of private deployments.

</aside>

But the concern most startups including us also have is that, if I deploy the model on their premises I don't get a long-term license purchase from them. They can easily replace me with a third-party model customizer and keep access to my model. So I would ideally want a deployment strategy that allows for seamless inference but renders the model completely unstable in retraining.

This is where fully homomorphic encryption (FHE) comes into play. This is a technique which says that states that assume there is an encryption function f if it satisfies the following 2 criteria any operation performed on the encrypted data is the same as that performed on plaintext:

• f(a+b) = f(a) + f(b)
• f(a.b) = f(a) . f(b)

So, why don't we just build this out internally? Well, there are problems here that abound. The first one is that the first FHE version was discovered very recently in 2009. Here is a 2016 summary of it. Since then there have been multiple versions of FHE since then with the latest being in 2016 which is an approximate implementation on block floating point (read this as integers). This adds several nondeterministic and deterministic approximation errors, that need handling in implementation.

So the basic idea is we can effectively encrypt the weights of a quantized LLM and deploy it in production. Since the encryption in fully homomorphic, it will not hurt accuracy greatly.

Now there are additional constraints ofc for this function which makes its implementation in production far from trivial:

• Computational Inefficiency: The non-approximate implementations of FHE are actually 10^12 times slower than performing an operation on plain text which renders any high-performance operation completely useless due to its exponential latency.