go gitlab gitlab-ci gitlab-ci-runner go-modules

How do I make golang project module accessible to another project pipeline on gitlab?

I have built 2 projects on my local machine. One called middleware and another called authentication.

both have modules. module for middleware project is called gitlab.com/nrs16/util, and module for project authentication is called gitlab.com/nrs16/authentication.

project authentication imports module gitlab.com/nrs16/util and uses some of it's function. when I run it locally all works fine

Both projects have their respective repositories on gitlab. for authentication I created a pipeline to build it's binary however it's failing on go mod tidy with the below error

go: gitlab.com/nrs16/authentication/src imports
    gitlab.com/nrs16/util: module gitlab.com/nrs16/util: git ls-remote -q origin in /go/pkg/mod/cache/vcs/93c72a4658a85380430fcf63ad9a84a56291aad8ea2eb7f78ecda66ceb9f7e58: exit status 128:
    fatal: could not read Username for 'https://gitlab.com': terminal prompts disabled
Confirm the import path was entered correctly.
If this is a private repository, see https://golang.org/doc/faq#git_https for additional information.

the executor is shell

This is the .gitlab-ci.yml content:

stages:
  - build

variables:
  GO111MODULE: "on"

build:
  stage: build
  image: golang:1.21.0
  script:
    - go mod tidy
    - go mod download
    - go build -o authentication ./src/*.go
  artifacts:
    paths: 
      - src/.authentication

middleware go.mod:

module gitlab.com/nrs16/util

go 1.21.0

require (
    github.com/dgrijalva/jwt-go v3.2.0+incompatible
    github.com/spf13/viper v1.16.0
    golang.org/x/crypto v0.13.0
)

require (
    github.com/fsnotify/fsnotify v1.6.0 // indirect
    github.com/hashicorp/hcl v1.0.0 // indirect
    github.com/magiconair/properties v1.8.7 // indirect
    github.com/mitchellh/mapstructure v1.5.0 // indirect
    github.com/pelletier/go-toml/v2 v2.0.8 // indirect
    github.com/spf13/afero v1.9.5 // indirect
    github.com/spf13/cast v1.5.1 // indirect
    github.com/spf13/jwalterweatherman v1.1.0 // indirect
    github.com/spf13/pflag v1.0.5 // indirect
    github.com/subosito/gotenv v1.4.2 // indirect
    golang.org/x/sys v0.12.0 // indirect
    golang.org/x/text v0.13.0 // indirect
    gopkg.in/ini.v1 v1.67.0 // indirect
    gopkg.in/yaml.v3 v3.0.1 // indirect
)

import in main.go file in authentication/src/ directory

package main

import (
    "database/sql"
    _ "encoding/json"
    "fmt"
    "log"
    "net/http"

    middleware "gitlab.com/nrs16/util"

    "github.com/gorilla/mux"
    _ "github.com/lib/pq"
)

Notes:

Middleware project is public on gitlab, I am not sure if this is enough to create its module on the runner's environment.

How can I make util module accessible to the runner so binary for project authentication builds? also runner is shared runner works fine on my local machine but that's expected since the module is created on my local PC but just thought I'd share this as well

I am new to this so I tried checking on chatGPT. it says that I should just push project middleware with it's go.mod file , which I did, and make it public, which I did.

I also changed the module name from gitlab.com/nrs16/util to util, it also worked fine on my local PC. But I still got this error on go build command in pipeline:

package util is not in std (/usr/local/go/src/util)

full .gitlab-ci.yml content:

stages:
  - build

variables:
  GO111MODULE: "on"

build:
  stage: build
  image: golang:1.21.0
  script:
    - go build -o authentication ./src/*.go
  artifacts:
    paths: 
      - src/.authentication

main.go:

package main

import (
    "database/sql"
    _ "encoding/json"
    "fmt"
    "log"
    "net/http"

    middleware "util"

    "github.com/gorilla/mux"
    _ "github.com/lib/pq"
)

Solution

Well, this is not an actual answer but it should help you move forward. Unfotunately there seem to be quite same major issues in how you try to work with Go that are error so this is too much for just comment. Bulletpoints as no real order here.

If you have problems it0's best to avoid everything complicated, special, exceptional, doable but unusual. The almost every advice below is technically wrong, it's just the simples, sanest, less-error prone way.
The fundamental building block of Go programs is the "package": You build packages, you test packages. Not files. A "Go Module" might not be what you are used to from other languages: A "module" is a set of packages versioned together. i.e. have the same lifecycle.
Stop stuff like go build -o authentication ./src/*.go It is wrong to bild Go code like this: 1. Do not put sources in a src folder. That might be common in other languages, don't do it in Go. Never. 2. You do not call go build withe file arguments ever. You build Go programs by running go build in the folder of the package you want to build.
Use proper names of the folders! Per convention the folder name should be the package name or the program name (for package main) and nothing else. (No src you remember?) Do not make "util" packages. Especially if the seem to have some domain (middleware). Call them "middleware" or "fsabstraction" or "payment". Do not create packages of unrelated stuff.
Read (and if possible work through) https://go.dev/doc/tutorial/create-module, https://go.dev/doc/tutorial/workspaces and https://go.dev/doc/code . It explains everything in detail.
Always include the whole go.mod of all modules in the question
If you want to use several modules (e.g. for educational purpose) and you use gitlab you must name your modules in the form gitlab.com/<UserOrg>/<Project>. Never ever name a module "test" or "main". Never name a module "util", "strings" or "foo". Always use a name like "foo.bar.nil/whatever" or "example.org/myfirst/try" (this has the only reason to give you better error messages; but you are having problems and better error messages help solving problems).
Always use full qualified import path like "example.org/myfirst/try/some/package/in/there". Never ever use relative imports.
go mod tidy is run before checking the code in. If you want a dedicated "source download step" (it's unneeded!) you can do go mod download on the CI. But again, no need to, go build will take care of downloding anyway.
If one CI jobs wants to use code from a different repo it must be able to download it from that repo. You didn't allow gitlab.com/nrs16/authentication to checkout gitlab.com/nrs16/util, thus the error. Note the last line: "If this is a private repository, see https://golang.org/doc/faq#git_https for additional information." If your repo isn't public you need to do smth like git config "url.ssh://git@gitlab.com/.insteadof" "https://gitlab.com/" on the CI.
Access to a private repo is a common question here, you'll find several answers. One way to allow Gitlab repo A access repo B is configuring a Deploy Key on B and setting SSH_PRIVATE_KEY variable on A to that key.