cmd/upspinfs: replace timed refeshes in upspinfs with Watch based refresh

The previous version had a very short valid time for anything it
gave the kernel to limit the periods of inconsistency between
the kernel and the directory servers. It also refreshed any
cached DirEntries fairly often to limit its own inconsistency.
This created performance issues where we could only trade
performance against duration of inconsistency.

This version uses Watch to maintain consistency. There is still
a window of inconsistency as the Watch events percolate across
the network but a much smaller one. Also since we know when things
change, we can force the kernel to invalidate cached information
and thereby feel safe in increasing the kernel cache validity time.

If Watch is not supported by a DirServer, it falls back to refresh
entries once a minute. That gives us the benefit of caching but
with a larger window of inconsistency.

A somewhat related change in dircache removes an unnecessary check
that was proportional to the size of a directory. This too was
affecting performance, though only in very large directories.

The new watch.go is very similar to proxied.go in dircache. I
should eventually merge them but not until I have had more

experience with this version.

The one thing I learned is that one must drain the Watch channel
after closing done or future RPCs on the same connection can
hang.

Fixes #549

Change-Id: I8df66be9d606f7d43dfd4abd4bc3d9f389267f57
Reviewed-on: https://upspin-review.googlesource.com/17880
Reviewed-by: Rob Pike <r@golang.org>
5 files changed
tree: 411bf51ff8ab541ad8d382efabd42e9ce25d37ee
  1. .travis.yml
  2. AUTHORS
  3. CONDUCT.md
  4. CONTRIBUTING.md
  5. CONTRIBUTORS
  6. Gopkg.lock
  7. Gopkg.toml
  8. LICENSE
  9. PATENTS
  10. README.binary
  11. README.md
  12. access/
  13. bind/
  14. cache/
  15. client/
  16. cloud/
  17. cmd/
  18. config/
  19. dir/
  20. doc/
  21. errors/
  22. factotum/
  23. flags/
  24. key/
  25. log/
  26. metric/
  27. pack/
  28. path/
  29. rpc/
  30. serverutil/
  31. shutdown/
  32. store/
  33. subcmd/
  34. test/
  35. transports/
  36. upbox/
  37. upspin/
  38. user/
  39. valid/
  40. vendor/
  41. version/
README.md

Upspin

Augie

Documentation: upspin.io

About the project

Upspin is an experimental project to build a framework for naming and sharing files and other data securely, uniformly, and globally: a global name system of sorts.

It is not a file system, but a set of protocols and reference implementations that can be used to join things like file systems and other storage services to the name space.

Performance is not a primary goal. Uniformity and security are.

Upspin is not an official Google product.

Status

Upspin has rough edges, and is not yet suitable for non-technical users.

Build Status

Contributing

The code repository lives at upspin.googlesource.com and is mirrored to GitHub.

Note that the Upspin project does not use GitHub pull requests, and that we use the issue tracker for bug reports and proposals only.

See the Contribution Guidelines for more information on contributing to the project.

Reporting issues

Please report issues through our issue tracker.

Community

All Upspin users should subscribe to the Upspin Announcements mailing list to receive critical information about the project.

Use the Upspin mailing list for discussion about Upspin use and development.

Code of Conduct

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.

The Upspin mascot is Copyright 2017 Renee French. All Rights Reserved.