HashSize is the number of bytes in a hash.
const HashSize = 32
DebugTest is set when GODEBUG=gocachetest=1 is in the environment.
var DebugTest = false
func DefaultDir() (string, bool)
DefaultDir returns the effective GOCACHE setting. It returns "off" if the cache is disabled, and reports whether the effective value differs from GOCACHE.
func FileHash(file string) ([HashSize]byte, error)
FileHash returns the hash of the named file. It caches repeated lookups for a given file, and the cache entry for a file can be initialized using SetFileHash. The hash used by FileHash is not the same as the hash used by NewHash.
func PutBytes(c Cache, id ActionID, data []byte) error
PutBytes stores the given bytes in the cache as the output for the action ID.
func SetFileHash(file string, sum [HashSize]byte)
SetFileHash sets the hash returned by FileHash for file.
An ActionID is a cache action key, the hash of a complete description of a repeatable computation (command line, environment variables, input file contents, executable contents).
type ActionID [HashSize]byte
func Subkey(parent ActionID, desc string) ActionID
Subkey returns an action ID corresponding to mixing a parent action ID with a string description of the subkey.
Cache is the interface as used by the cmd/go.
type Cache interface {
// Get returns the cache entry for the provided ActionID.
// On miss, the error type should be of type *entryNotFoundError.
//
// After a success call to Get, OutputFile(Entry.OutputID) must
// exist on disk for until Close is called (at the end of the process).
Get(ActionID) (Entry, error)
// Put adds an item to the cache.
//
// The seeker is only used to seek to the beginning. After a call to Put,
// the seek position is not guaranteed to be in any particular state.
//
// As a special case, if the ReadSeeker is of type noVerifyReadSeeker,
// the verification from GODEBUG=goverifycache=1 is skipped.
//
// After a success call to Get, OutputFile(Entry.OutputID) must
// exist on disk for until Close is called (at the end of the process).
Put(ActionID, io.ReadSeeker) (_ OutputID, size int64, _ error)
// Close is called at the end of the go process. Implementations can do
// cache cleanup work at this phase, or wait for and report any errors from
// background cleanup work started earlier. Any cache trimming should in one
// process should not violate cause the invariants of this interface to be
// violated in another process. Namely, a cache trim from one process should
// not delete an ObjectID from disk that was recently Get or Put from
// another process. As a rule of thumb, don't trim things used in the last
// day.
Close() error
// OutputFile returns the path on disk where OutputID is stored.
//
// It's only called after a successful get or put call so it doesn't need
// to return an error; it's assumed that if the previous get or put succeeded,
// it's already on disk.
OutputFile(OutputID) string
// FuzzDir returns where fuzz files are stored.
FuzzDir() string
}
func Default() Cache
Default returns the default cache to use. It never returns nil.
A Cache is a package cache, backed by a file system directory tree.
type DiskCache struct {
// contains filtered or unexported fields
}
func Open(dir string) (*DiskCache, error)
Open opens and returns the cache in the given directory.
It is safe for multiple processes on a single machine to use the same cache directory in a local file system simultaneously. They will coordinate using operating system file locks and may duplicate effort but will not corrupt the cache.
However, it is NOT safe for multiple processes on different machines to share a cache directory (for example, if the directory were stored in a network file system). File locking is notoriously unreliable in network file systems and may not suffice to protect the cache.
func (c *DiskCache) Close() error
func (c *DiskCache) FuzzDir() string
FuzzDir returns a subdirectory within the cache for storing fuzzing data. The subdirectory may not exist.
This directory is managed by the internal/fuzz package. Files in this directory aren't removed by the 'go clean -cache' command or by Trim. They may be removed with 'go clean -fuzzcache'.
TODO(#48526): make Trim remove unused files from this directory.
func (c *DiskCache) Get(id ActionID) (Entry, error)
Get looks up the action ID in the cache, returning the corresponding output ID and file size, if any. Note that finding an output ID does not guarantee that the saved file for that output ID is still available.
func (c *DiskCache) OutputFile(out OutputID) string
OutputFile returns the name of the cache file storing output with the given OutputID.
func (c *DiskCache) Put(id ActionID, file io.ReadSeeker) (OutputID, int64, error)
Put stores the given output in the cache as the output for the action ID. It may read file twice. The content of file must not change between the two passes.
func (c *DiskCache) Trim() error
Trim removes old cache entries that are likely not to be reused.
type Entry struct {
OutputID OutputID
Size int64
Time time.Time // when added to cache
}
func GetBytes(c Cache, id ActionID) ([]byte, Entry, error)
GetBytes looks up the action ID in the cache and returns the corresponding output bytes. GetBytes should only be used for data that can be expected to fit in memory.
func GetFile(c Cache, id ActionID) (file string, entry Entry, err error)
GetFile looks up the action ID in the cache and returns the name of the corresponding data file.
func GetMmap(c Cache, id ActionID) ([]byte, Entry, error)
GetMmap looks up the action ID in the cache and returns the corresponding output bytes. GetMmap should only be used for data that can be expected to fit in memory.
A Hash provides access to the canonical hash function used to index the cache. The current implementation uses salted SHA256, but clients must not assume this.
type Hash struct {
// contains filtered or unexported fields
}
func NewHash(name string) *Hash
NewHash returns a new Hash. The caller is expected to Write data to it and then call Sum.
func (h *Hash) Sum() [HashSize]byte
Sum returns the hash of the data written previously.
func (h *Hash) Write(b []byte) (int, error)
Write writes data to the running hash.
An OutputID is a cache output key, the hash of an output of a computation.
type OutputID [HashSize]byte
func PutNoVerify(c Cache, id ActionID, file io.ReadSeeker) (OutputID, int64, error)
PutNoVerify is like Put but disables the verify check when GODEBUG=goverifycache=1 is set. It is meant for data that is OK to cache but that we expect to vary slightly from run to run, like test output containing times and the like.
ProgCache implements Cache via JSON messages over stdin/stdout to a child helper process which can then implement whatever caching policy/mechanism it wants.
See https://github.com/golang/go/issues/59719
type ProgCache struct {
// contains filtered or unexported fields
}
func (c *ProgCache) Close() error
func (c *ProgCache) FuzzDir() string
func (c *ProgCache) Get(a ActionID) (Entry, error)
func (c *ProgCache) OutputFile(o OutputID) string
func (c *ProgCache) Put(a ActionID, file io.ReadSeeker) (_ OutputID, size int64, _ error)
ProgCmd is a command that can be issued to a child process.
If the interface needs to grow, we can add new commands or new versioned commands like "get2".
type ProgCmd string
ProgRequest is the JSON-encoded message that's sent from cmd/go to the GOCACHEPROG child process over stdin. Each JSON object is on its own line. A ProgRequest of Type "put" with BodySize > 0 will be followed by a line containing a base64-encoded JSON string literal of the body.
type ProgRequest struct {
// ID is a unique number per process across all requests.
// It must be echoed in the ProgResponse from the child.
ID int64
// Command is the type of request.
// The cmd/go tool will only send commands that were declared
// as supported by the child.
Command ProgCmd
// ActionID is non-nil for get and puts.
ActionID []byte `json:",omitempty"` // or nil if not used
// ObjectID is set for Type "put" and "output-file".
ObjectID []byte `json:",omitempty"` // or nil if not used
// Body is the body for "put" requests. It's sent after the JSON object
// as a base64-encoded JSON string when BodySize is non-zero.
// It's sent as a separate JSON value instead of being a struct field
// send in this JSON object so large values can be streamed in both directions.
// The base64 string body of a ProgRequest will always be written
// immediately after the JSON object and a newline.
Body io.Reader `json:"-"`
// BodySize is the number of bytes of Body. If zero, the body isn't written.
BodySize int64 `json:",omitempty"`
}
ProgResponse is the JSON response from the child process to cmd/go.
With the exception of the first protocol message that the child writes to its stdout with ID==0 and KnownCommands populated, these are only sent in response to a ProgRequest from cmd/go.
ProgResponses can be sent in any order. The ID must match the request they're replying to.
type ProgResponse struct {
ID int64 // that corresponds to ProgRequest; they can be answered out of order
Err string `json:",omitempty"` // if non-empty, the error
// KnownCommands is included in the first message that cache helper program
// writes to stdout on startup (with ID==0). It includes the
// ProgRequest.Command types that are supported by the program.
//
// This lets us extend the protocol gracefully over time (adding "get2",
// etc), or fail gracefully when needed. It also lets us verify the program
// wants to be a cache helper.
KnownCommands []ProgCmd `json:",omitempty"`
Miss bool `json:",omitempty"` // cache miss
OutputID []byte `json:",omitempty"`
Size int64 `json:",omitempty"` // in bytes
Time *time.Time `json:",omitempty"` // an Entry.Time; when the object was added to the docs
// DiskPath is the absolute path on disk of the ObjectID corresponding
// a "get" request's ActionID (on cache hit) or a "put" request's
// provided ObjectID.
DiskPath string `json:",omitempty"`
}