Skip to content

Added the ability to open a ZarrTable from a store path.#56

Closed
alxmrs wants to merge 1 commit into
mainfrom
open_table
Closed

Added the ability to open a ZarrTable from a store path.#56
alxmrs wants to merge 1 commit into
mainfrom
open_table

Conversation

@alxmrs

@alxmrs alxmrs commented Sep 24, 2025

Copy link
Copy Markdown
Collaborator

I needed this for xarray-sql#69, please let me know if there is a better way to go about implementing this.

I needed this for xarray-sql#69, please let me know if there is a better way to go about implementing this.
@alxmrs
alxmrs requested a review from maximedion2 September 24, 2025 15:48
@alxmrs
alxmrs marked this pull request as ready for review September 24, 2025 15:50
@maximedion2

Copy link
Copy Markdown
Collaborator

So I have some changes to how that part works in PR #54, would you rather merge this now and I will rebase and modify once I merge my PR, or wait until I merge and you rebase and make the necessary changes? I'm fine with either way.

Also I didn't ask for your review on PR #54 I wasn't sure if you were working on anything in this repo but of course feel free to take a look and let me know if you have any thoughts! And I have a second PR in the works that will allow reading from s3 buckets, is that something you'll want to pick up once I merge (the PR isn't ready yet)?

@alxmrs

alxmrs commented Sep 25, 2025

Copy link
Copy Markdown
Collaborator Author

I'm on a vacation for the next two weeks and have much more down time than normal, so I'm game to review #54 and make other contributions! Let me look at that PR now to see if we should merge this and rebase or wait to land this change. :)

I'm excited to get these sources in xarray-sql sometime during this trip. That would be very exciting to me.

@alxmrs alxmrs mentioned this pull request Sep 25, 2025
@alxmrs

alxmrs commented Sep 25, 2025

Copy link
Copy Markdown
Collaborator Author

I'd be happy to review the S3/Cloud buckets PR, for sure. I just made a PR into your new feature branch here: #57. Happy to merge that one instead of this one.

@maximedion2

Copy link
Copy Markdown
Collaborator

Okay should we close this one now that you opened #57?

@maximedion2

Copy link
Copy Markdown
Collaborator

Oh I've been wanting to ask you, are you basically using some of the functionalities here without the datafusion "hookup"? I was briefly discussing with Kyle (on his earlier PR) about if we should have the arrow + zarr logic be completely stand alone or if it's fine to have it fully integrated with datafusion.

@alxmrs

alxmrs commented Sep 26, 2025

Copy link
Copy Markdown
Collaborator Author

Okay should we close this one now that you opened #57?

Yes, happy to close this one.

are you basically using some of the functionalities here without the datafusion "hookup"?

I specifically need something like the ZarrTable or something that implements DataFusion's TableProvider interface that can read subsets of arrays from a group into a table. I've just written up what I'm looking for here: #58. I would be happy to move such logic somewhere else (the "zarrquet" library in xqlsystems/xarray-sql#69 would make me happy!) and use this library for the core scan operations on Zarr.

@alxmrs alxmrs closed this Sep 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants