You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
At the moment, because SSHFS is considered cachable, creating a new filesystem doesn't create a new connection, and maintains the old one.
That's probably not a good fit for SFTP connections, since they're likely to timeout and be closed off after a while, so almost any long running process using SSHFS is eventually going to hit an issue where the filesystem continues to reference a closed off connection, and the cache needs to be manually cleared.
To fix that, I've just set "cachable" to False here, so that creating a new filesystem would also bring in a new connection.
Honestly, I don't have the knowledge to know the full implications of this, but my main thoughts are:
Does this mean it's worth adding in something to clean up / close the connection when the SSHFS object goes out of memory? (otherwise there are potentially open but unusable connections floating around)
Is there a case where you'd want it to be cachable? In which case, the "cachable" setting could theoretically be put behind an __init__ argument? (my guess is that'd cause some kind of side effects nobody is expecting though)
Thanks for digging into this @benrutter — the diagnosis (cached instances outliving their connections) is right, but after testing this change we're closing it, for two reasons:
As written it's a no-op: fsspec's instance cache lives in the _Cached metaclass, which checks the class attribute cls.cachable both when looking up and when storing instances — an instance attribute set inside __init__ is invisible to it. Verified empirically: with this patch applied, constructing SSHFileSystem twice with the same arguments still returns the same cached object.
The working variant (cachable = False as a class attribute) would be the wrong default: every construction would pay a fresh SSH handshake (hundreds of ms), a heavy ecosystem-wide cost — DVC and fsspec.open() construct filesystems repeatedly and rely on the instance cache.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes issue #42
At the moment, because SSHFS is considered
cachable, creating a new filesystem doesn't create a new connection, and maintains the old one.That's probably not a good fit for SFTP connections, since they're likely to timeout and be closed off after a while, so almost any long running process using SSHFS is eventually going to hit an issue where the filesystem continues to reference a closed off connection, and the cache needs to be manually cleared.
To fix that, I've just set "cachable" to False here, so that creating a new filesystem would also bring in a new connection.
Honestly, I don't have the knowledge to know the full implications of this, but my main thoughts are:
__init__argument? (my guess is that'd cause some kind of side effects nobody is expecting though)