Updates

[January 2024] New Model Release: Arithmo2-Mistral-7B

Arithmo2-Mistral-7B model improves initially released Arithmo-Mistral-7B model on both GSM8K and MATH benchmarks. Specifically, there is absolute improvement of +1.7% on GSM8K, +3.0% on GSM8K PoT, and +1.9% on MATH benchmarks. We release both merged model and LoRA Adapter.

• Arithmo2-Mistral-7B is trained on same data as Arithmo-Mistral-7B except that we removed both validation and test set of lila ood subset to avoid possibility of data leakage.
• Added NEFTune
• Enabled sample packing = true for faster training.

Arithmo Models

Both Arithmo2-Mistral-7B and Arithmo-Mistral-7B models are trained to reason and answer mathematical problems and is also capable of writing a Python program that upon execution prints answer to the question. We used Mistral-7B as a base model and used QLoRA to fine-tune it on a single RTX 4090 GPU.

Benchmark Results

Arithmo2-Mistral-7B model is fine-tuned with 4-bit QLoRA on single GPU and is competitive with supervised full-finetuned state-of-the-art Mathematical Reasoning models. Refer to Comparing Arithmo models with other SFT LLM models section for more details.

• Zero-Shot CoT: On providing a question as prompt, model generates reasoning steps to solve the question along with answer. We check if answer matches with ground-truth.
• Zero-Shot PoT: We prompt the model to generate a Python program for the given question. During inference, we execute the Python program generated by the model and check if the program output matches with ground-truth answer. Visit Model Card to see few PoT examples.

Installation

pip install transformers >=4.34.0
pip install accelerate
pip install sentencepiece
pip install protobuf

# If you are GPU poor like me
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu

# If you have a GPU.
pip install --pre torch --index-url https://download.pytorch.org/whl/nightly/cu118
pip install scipy
pip install bitsandbytes

How to query the model

# Set `run_model_on_gpu` to `False` if you are running on CPU. Model will generate reasoning steps with answer for your question. If you want to generate Python program, uncomment line-69 that adds a Python prompt.
# This script automatically does formatting for you, so you just need to type question (eg: `What is 2+2?`) without any prefix like `Question:`, etc.**

$ python query_model.py

Note: Above script automatically does formatting for you, so you just need to type question (eg: What is 2+2?) without any prefix like Question:, etc. Checkout query_model.py for more details.

Sample Input:

Question: There are total 10 children. I have to give 1 apple to first child, 2 apples to second child, 3 apples to third child, and so on. How many apples do I need?

Model Output:

Answer: The total number of apples needed is the sum of the first 10 positive integers.
This can be calculated using the formula for the sum of an arithmetic series:
\[S = \frac{n}{2}(a_1 + a_n),\]
where $S$ is the sum, $n$ is the number of terms, $a_1$ is the first term, and $a_n$ is the last term.
In this case, $n = 10$, $a_1 = 1$, and $a_n = 10$.
Plugging these values into the formula, we get:
\[S = \frac{10}{2}(1 + 10) = 5(11) = \boxed{55}.\]
The answer is: 55

Arithmo-Mistral-7B is trained with the following format:

CoT Format (generate reasoning steps with answer):

Question: <question>

Answer:

PoT Format (generate a python program):

Question: <question> <python_prompt>

Answer:

It will perform best if queried in this way with your own script.

Model Finetuning Details

Due to limited compute budget, Mistral-7B model is fine-tuned with QLoRA using Single RTX 4090 GPU. We plan to do a full finetuning of Mistral-7B model on this dataset to further improve performance.

Reproducing Results

Model Training Data

Model training data is prepared by combining MetaMathQA (train split), lila OOD (train, validation, and test splits), and MathInstruct (train split) datasets. We have verified that our training data has no overlap with GSM8K and MATH test set. Further post-processing steps are applied such as 1) deduplication, 2) randomly lower-casing x% inputs, 3) adding diverse set of Python prompts for PoT, and 4) standardizing answer format. Final dataset is of size ~540,000. Also, to train Arithmo2-Mistral-7B model, we removed both validation and test set of lila ood subset to avoid possibility of data leakage.

# This script generates train and eval sets.
$ python data_prep/prepare_model_traininig_data.py

Here is Huggingface link for our dataset.

Answer/Response Generation

Prediction on GSM8K Test set

Zero-Shot with CoT:

# This script saves output to `data/predictions/gsm8k/Arithmo-Mistral-7B/predictions_Arithmo_gsm8k_zero_shot_CoT.json` path.
$ python eval/gsm8k/gsm8k_generate_response_zero_shot_CoT.py

Zero-Shot with PoT:

# This script saves output to `data/predictions/gsm8k/Arithmo-Mistral-7B/predictions_Arithmo_gsm8k_zero_shot_PoT.json` path.
$ python eval/gsm8k/gsm8k_generate_response_zero_shot_PoT.py

Prediction on MATH Test set

Zero-Shot with CoT:

# This script saves output to `data/predictions/gsm8k/Arithmo-Mistral-7B/predictions_Arithmo_MATH_zero_shot_CoT.json` path.
$ python eval/MATH/MATH_generate_response_zero_shot_CoT.py

Zero-Shot with PoT: Answers in MATH test set consist of expressions like (x+2)/5 instead of a numeric value. Currently, Arithmo-Mistral-7B's PoT training data doesn't contain expressions as answers. Hence, we don't run PoT based inference on MATH dataset.

Metrics Computation

GSM8K Test set

Zero-Shot with CoT:

$ python eval/gsm8k/gsm8k_compute_metric_zero_shot_CoT.py

Expected output: Total Instances: 1319, Correct Count: 985, Accuracy (Correct Count/Total Instances): 0.7467

Zero-Shot with PoT:

# Step-1: This script executes generated python programs and saves results into a file.
$ python eval/gsm8k/gsm8k_write_zero_shot_PoT_outputs.py > data/predictions/gsm8k/Arithmo-Mistral-7B/gsm8k_zero_shot_PoT_results.txt

# Step-2: This script computes accuracy by taking above file as input.
$ python eval/gsm8k/gsm8k_compute_metric_zero_shot_PoT.py

Expected output: Total Instances: 1309, Correct Count: 932, Accuracy: 0.7119

MATH Test set

Zero-Shot with CoT:

$ python eval/MATH/MATH_compute_metric_zero_shot_CoT.py

Script is borrowed from official math repository
Expected output: Total Instances: 5000, Correct Count: 1266, Accuracy (Correct Count/Total Instances): 0.2532

Comparing Arithmo models with other SFT LLM models

Results for all models except Arithmo2-Mistral-7B and Arithmo-Mistral-7B are taken from MetaMath repository.

Citation

To cite Arithmo models:

@misc{jindal_2023_arithmo,
  author = {Jindal, Ashvini},
  title = {Arithmo-Mistral-7B: Mathematical Reasoning Model},
  howpublished = {Hugging Face},
  month = {October},
  year = {2023},
  url = {https://huggingface.co/akjindal53244/Arithmo-Mistral-7B}
}

Support My Work

Building LLMs takes time and resources; if you find my work interesting, your support would be epic!

P.S.: If you are interested in providing compute support, please reach out to Ashvini Jindal

References

@article{yu2023metamath,
  title={MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models},
  author={Yu, Longhui and Jiang, Weisen and Shi, Han and Yu, Jincheng and Liu, Zhengying and Zhang, Yu and Kwok, James T and Li, Zhenguo and Weller, Adrian and Liu, Weiyang},
  journal={arXiv preprint arXiv:2309.12284},
  year={2023}
}

@article{Yue2023mammoth,
  title={MAmmoTH: Building math generalist models through hybrid instruction tuning},
  author={Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen},
  journal={arXiv preprint arXiv:2309.05653},
  year={2023}
}

@article{mishra2022lila,
  title={Lila: A unified benchmark for mathematical reasoning},
  author={Swaroop Mishra, Matthew Finlayson, Pan Lu, Leonard Tang, Sean Welleck, Chitta Baral, Tanmay Rajpurohit, Oyvind Tafjord, Ashish Sabharwal, Peter Clark, and Ashwin Kalyan},
  journal={arXiv preprint arXiv:2210.17517},
  year={2022}
}

Todos