Uneven split func

[noahho]

Motivation and Context

---

Public API Changes

• No Public API changes
• Yes, Public API changes (Details below)

---

How Has This Been Tested?

---

Checklist

• The changes have been tested locally.
• Documentation has been updated (if the public API or usage changes).
• A entry has been added to CHANGELOG.md (if relevant for users).
• The code follows the project's style guidelines.
• I have considered the impact of these changes on the public API.

---

[gemini-code-assist]

Summary of Changes

Hello @noahho, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request refines the data batching mechanism used internally by the Classifier and Regressor dataloaders. The primary goal is to provide more flexible and potentially more efficient data processing by allowing batches to be unevenly sized, ensuring that most batches are full while handling remaining data gracefully, and enforcing a minimum batch size to prevent very small, inefficient batches.

Highlights

• Data Batching Logic Refinement: I've updated the split_large_data utility function to generate 'uneven' data batches. This means most batches will be of the specified max_data_size, and the final batch will contain any remaining samples. A new minimum batch size of 2 has been introduced, and any final batch smaller than this minimum will be dropped.
• Changelog Update: I've added an entry to CHANGELOG.md to document the change in how preprocessed data batches are generated by the Classifier and Regressor dataloaders, noting that they now produce uneven, fully-filled batches with the last batch containing the remainder.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[Copilot]

Pull Request Overview

This PR modifies the split_large_data function to use an uneven chunking strategy instead of the previous even split approach. The change simplifies the splitting logic while ensuring minimum chunk sizes are maintained.

• Replaces even chunk distribution with simple sequential chunking of max_data_size
• Adds minimum batch size validation and drops chunks smaller than 2 samples
• Updates documentation and changelog to reflect the new uneven splitting behavior

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
src/tabpfn/utils.py	Refactors split_large_data function to use uneven chunking with minimum size validation
CHANGELOG.md	Documents the change to uneven batch generation in preprocessed data

[gemini-code-assist]

Code Review

This pull request refactors the split_large_data function to create unevenly sized data chunks. This is a good simplification over the previous implementation which created evenly sized chunks, and the new logic is clearer and more maintainable. The change seems to better align with standard data batching strategies.

I've added one comment to CHANGELOG.md to correct a typo and improve the clarity of the new entry.