Exiting with fsspec.open_files(...) leaves the returned readers open when using simplecache or filecache.
Reproduced on master be32e2b2bb775104bc3f2bf291a87bbfedd8a273, Linux, Python 3.10, with only an in-memory source and temporary cache:
import tempfile
import fsspec
memory = fsspec.filesystem("memory")
memory.pipe("/example", b"data")
with tempfile.TemporaryDirectory() as cache:
with fsspec.open_files("simplecache::memory:///example", cache_storage=cache) as files:
assert files[0].read() == b"data"
print(files[0].closed) # False; expected True
files[0].close()
This also happens with filecache, a warm cache, and an exception inside the context. OpenFiles.__enter__ returns the handles from open_many, but __exit__ only closes the unpopulated OpenFile wrappers. Keeping the returned list alive therefore retains open file descriptors after the context ends.
AI assistance was used to investigate and run this reproducer.
Exiting
with fsspec.open_files(...)leaves the returned readers open when usingsimplecacheorfilecache.Reproduced on master
be32e2b2bb775104bc3f2bf291a87bbfedd8a273, Linux, Python 3.10, with only an in-memory source and temporary cache:This also happens with
filecache, a warm cache, and an exception inside the context.OpenFiles.__enter__returns the handles fromopen_many, but__exit__only closes the unpopulatedOpenFilewrappers. Keeping the returned list alive therefore retains open file descriptors after the context ends.AI assistance was used to investigate and run this reproducer.