Fortune is a framework for symbolic table reasoning using large language models (LLMs). Unlike prior approaches that rely heavily on supervised fine-tuning (SFT) with annotated formulas, Fortune introduces a reinforcement learning (RL) framework that learns directly from binary answer supervision. This formula-driven design leads to stronger generalization, higher reasoning accuracy, and more robust table understanding capabilities.
Notice: This repository provides only the core Fortune component implementations. It is not a complete standalone training framework. These components must be integrated into the original Verl codebase and placed in the corresponding Verl project directories to function correctly.
conda create -n fortune python=3.11
conda activate fortune
# Install torch (adjust CUDA version if needed)
pip install torch==2.4.0 --index-url https://download.pytorch.org/whl/cu124
# Flash attention (for efficient training)
pip install flash-attn --no-build-isolation
# vLLM inference engine
pip install vllm==0.8.3
# Excel file support
pip install openpyxl
# External formula executor
pip install formulas
# Additional utilities
pip install nvitop beautifulsoup4
Our implementation builds upon the Verl framework.
Note: Our codebase cannot be executed directly. It first requires the installation of Verl, along with additions or modifications to the original Verl framework code. Please be aware that different releases of Verl may require different implementation changes.
# 1. Clone Verl (our implementation is based on Verl v0.3.0post1, released on 04/04/2025)
git clone https://github.com/volcengine/verl
# 2. Apply our patches to Verl
## 2.1 Manually update the following files as instructed
# e.g., modify the contents of verl/verl/trainer/ppo/metric_utils.py
# by applying the changes provided in verl_patches/trainer/ppo/metric_utils_u.py
verl_patches/trainer/ppo/metric_utils_u.py
verl_patches/trainer/ppo/ray_trainer_u.py
verl_patches/utils/dataset/rl_dataset_u.py
verl_patches/utils/dataset/sft_dataset_u.py
verl_patches/utils/reward_score/__init__u.py
## 2.2 Add the remaining files from verl_patches/*
cp -r verl_patches/* verl/verl # overwrite the original files
# 3. Install Verl
cd verl
pip install -e .Follow the instructions in: data/raw_data/README.md.
Baseline Evaluation
# Evaluate GPT-based baseline
sh scripts/eval/run_eval_gpt.sh
# Evaluate local LLM baseline
sh scripts/eval/run_eval_local_llm.shTraining
# Supervised Fine-Tuning (SFT)
sh scripts/train/run_sft_all.sh
# Reinforcement Learning (RL)
sh scripts/train/run_rl_all.sh
# Cold-Start Reinforcement Learning
sh scripts/train/run_rl_all_cold_start.shIf you find this project helpful, please consider citing:
@article{cao2025fortune,
title = {Fortune: Formula-Driven Reinforcement Learning for Symbolic Table Reasoning in Language Models},
author = {Cao, Lang and Xu, Jingxian and Liu, Hanbing and Wang, Jinyu and Zhou, Mengyu and Dong, Haoyu and Han, Shi and Zhang, Dongmei},
journal = {arXiv preprint arXiv:2505.23667},
year = {2025}
}
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.