Instruction tuning is a supervised training method used for improving an LLM on a new type of task using a dataset that contains natural language instructions along with the task examples. This is unlike fine tuning which is an unsupervised approach.

Collecting an instruction-tuning dataset is often the most expensive task that an AI company needs to do. Normally this is done as a combination of crowdsourcing, customer usage data, and hiring third-party annotators on platforms like AWS Mechanical Turk. This method is slow, expensive, and often insecure for vertical models as it more often than not exposes the company’s unique insight.

<aside> 💡 This is especially problematic in finance because the cheapest of the above 3 approaches i.e. customer usage data is heavily monitored by the regulators and the stakeholders at major banks, insurance firms, wealth managers, and asset management companies.

</aside>

A very nice solution to this is something called unnatural instructions which refers to a large dataset of creative and diverse instructions, collected with virtually no human labor. The first paper on this was published in Dec 2022 by Meta and productized in October 2023 as unnatural code LLaMa. The paper concludes that despite containing a fair amount of noise, training on Unnatural Instructions rivals the effectiveness of training on open-source manually-curated datasets.

Fig 1: Showing the benchmark beating capacity of unnatural code LLaMa vs. manual instruction tuning the model

LLM Augmented Dataset Generation: This approach of unnatural instruction tuning uses an LLM to generate a very large dataset of instructions from a very small dataset. The meta team gave the LLM 3 instruction-input-constraint triplets and asked it to generate the 4th triplet. This was done until a dataset of 64,000 triplets was produced from 15 manual instructions. After this, they asked the LLM to make variations of each triplet that preserved the original content to end up with ~240K total examples across 11 task categories and 57% factual correctness.

     *Fig 2: The pipeline for unnatural instruction generation using an LLM and then using outputs for training*

NLP Augmented Dataset Generation: This is an alternate approach devised by us to prune wrong question-answer pairs from our instruction-tuning dataset wherein we generate a much larger seed dataset (~12K) of instructions using our NLP stack. We then allow an LLM to pick any combination of tasks from the seed dataset and synthesize it into a new question-answer pair. Finally, we ask it for semantic variations to end up with ~1,000,000 examples across 5 complex financial task types.

This dataset pruning by instruction mining using finance-specific natural language indicators like financial entity, market event, investor sentiment, market impact, and impact duration allows us to ensure our dataset quality is relatively high even for unseen task categories. It is normally observed across the industry that models trained on instruction-mined datasets perform better than models trained on unfiltered data in at least 42% of benchmarks.