Why does go.sum include so many older packages

I've been going though a set of projects, dealing with updating dependencies and there is one thing I don't have a clear answer to and that is why the generated sum file lists so many older versions of each dependency.

in our project we had some vulnerabilities introduced though an older version of golang.org/x/crypto that we resolved though a replace directive to a package release with security fixes but this doesn't feel very correct and could lock us into an insecure version of a package.

now I have gone though and updated the package which depended on an older version of golang.org/x/crypto and looped back to the package with the the replace directive and attempted an update but I still see the older packages listed.

I'm wondering what this means for our project and how I can find why these are included in the first place?

running a simple go mod why -m golang.org/x/crypto reveals that the only project dependent on golang.org/x/crypto was the one that I had updated.

Solution

@JimB offered some documentation on go sum with the following statement

The go.sum file may contain hashes for multiple versions of a module. The go command may need to load go.mod files from multiple versions of a dependency in order to perform minimal version selection. go.sum may also contain hashes for module versions that aren’t needed anymore (for example, after an upgrade). go mod tidy will add missing hashes and will remove unnecessary hashes from go.sum.

And the resulting set of packages defined in a go sum comes from the minimal version selection process which seems to be a deep topic.

An example would be importing "google.golang.org/grpc/metadata" into a module, running go mod tidy in the module and a small portion of a resulting sum file would be the following:

github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=

each reference to a version indictates a node in the minimal version selection algorithm graph

adding the following to the mod file

replace github.com/golang/protobuf => github.com/golang/protobuf v1.4.3

and running a go mod tidy, the resulting sum entry for protobuf changes to:

github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=

because the replace directive indicates a replacement of all versions so all nodes in the dependency graph are replaced with v1.4.3 which just simplifies to the inclusion of the single version of the dependency v1.4.3

As for the issue I had with the vulnerability scanner, it seems the author of it was unaware of of how dependencies should be checked for Golang and upgrading the module to go 1.17 where indirect dependencies are listed in the mod file didn't stop the sum entries from flagging the project for vulnerabilities.