convert column table to arrow if arrow present

[shivam2680]

What type of PR is this?

• Refactor
• Feature
• Bug Fix
• Oth

Description

Added logic in fetchall_arrow() to convert ColumnTable results to a pyarrow.Table if pyarrow is installed, maintaining consistency with behavior prior to version 3.5.0. This behavior makes result set aligned with the docstring of fetchall_pyarrow method

How is this tested?

• Unit tests
• E2E Tests
• Manually
• N/A

Introduced a new test case, test_catalogs_returns_arrow_table, in tests/e2e/test_driver.py to validate that the fetchall_arrow method returns a pyarrow.Table when used with cursor.catalogs(). This test is conditionally executed if Arrow support is available.

Related Tickets & Documents

Resolves #550

[github-actions]

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

[Copilot]

Pull Request Overview

This PR updates the behavior of fetching query results by converting a ColumnTable to a PyArrow Table when PyArrow is available. It adds a new end-to-end test to validate that the catalogs() API returns a PyArrow Table and updates the fetchall_arrow() method in the client to merge columnar results when appropriate.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
tests/e2e/test_driver.py	Added test_catalogs_returns_arrow_table to verify arrow table conversion.
src/databricks/sql/client.py	Updated fetchall_arrow() to merge ColumnTables and convert the final result where PyArrow is present.

[jackyhu-db]

can you add more description in the PR to explain why you make this change? Is there any disruption risk of this change?

[shivam2680]

can you add more description in the PR to explain why you make this change? Is there any disruption risk of this change?

Updated description