rpc.ServerCodec Still Serving?

Question

I was performing some RPC tests, and stumbled across a problem I can't seem to solve. In my testing I create three separate RPC servers, all of which I try to close and shutdown. However upon performing my last test (TestRpcCodecServerClientComm), it seems my client connection is connecting to the first RPC server I started (I know this because I at some point attached IDs to the RPCHandlers), even though I attempted everything I could to make sure it was shutdown. Though the code is not there I have attempted to inspect every single error I could, but that did not bring about anything.

rpc.go

package rbot

import (
    "io"
    "net"
    "net/rpc"
    "net/rpc/jsonrpc"
)

func RpcCodecClientWithPort(port string) (rpc.ClientCodec, error) {
    conn, err := net.Dial("tcp", "localhost:"+port)
    if err != nil {
        return nil, err
    }
    return jsonrpc.NewClientCodec(conn), nil
}

func RpcCodecServer(conn io.ReadWriteCloser) rpc.ServerCodec {
    return jsonrpc.NewServerCodec(conn)
}

rpc_test.go

package rbot

import (
    "errors"
    "fmt"
    "net"
    "net/rpc"
    "testing"
)

type RPCHandler struct {
    RPCServer net.Listener
    conn      rpc.ServerCodec
    done      chan bool
    TestPort  string
    stop      bool
    GotRPC    bool
}

func (r *RPCHandler) SetupTest() {
    r.stop = false
    r.GotRPC = false
    r.done = make(chan bool)
    r.TestPort = "5556"
}

// TODO: Create separate function to handle erroring
func (r *RPCHandler) CreateRPCServer() error {
    rpc.RegisterName("TestMaster", TestAPI{r})

    var err error
    r.RPCServer, err = net.Listen("tcp", ":"+r.TestPort)

    if err != nil {
        return err
    }

    go func() {
        for {
            conn, err := r.RPCServer.Accept()
            if err != nil || r.stop {
                r.done <- true
                return
            }
            r.conn = RpcCodecServer(conn)
            rpc.ServeCodec(r.conn)
        }
    }()
    return nil
}

func (r *RPCHandler) CloseRPCServer() error {
    r.stop = true
    if r.conn != nil {
        err := r.conn.Close()
        if err != nil {
            fmt.Println(err)
        }
    }
    err := r.RPCServer.Close()
    <-r.done
    return err
}

type TestAPI struct {
    t *RPCHandler
}

func (tapi TestAPI) Send(msg string, result *string) error {
    if msg == "Got RPC?" {
        tapi.t.GotRPC = true
        return nil
    }
    return errors.New("Didn't receive right message")
}

// Check if we can create and close an RPC server successfully using the RPC server codec.
func TestRpcCodecServer(t *testing.T) {
    r := RPCHandler{}
    r.SetupTest()

    err := r.CreateRPCServer()
    if err != nil {
        t.Fatalf("Could not create rpc server! %s:", err.Error())
    }

    err = r.CloseRPCServer()
    if err != nil {
        t.Fatalf("Could not close RPC server! %s:", err.Error())
    }
}

// Check if we can create a client without erroring.
func TestRpcCodecClientWithPortt(t *testing.T) {
    r := RPCHandler{}
    r.SetupTest()
    r.CreateRPCServer()
    defer r.CloseRPCServer()

    RPCClient, err := RpcCodecClientWithPort(r.TestPort)
    defer RPCClient.Close()
    if err != nil {
        t.Fatalf("Could not create an RPC client! %s:", err.Error())
    }
}

// Let's double check and make sure our server and client can speak to each other
func TestRpcCodecServerClientComm(t *testing.T) {
    r := RPCHandler{}
    r.SetupTest()
    r.CreateRPCServer()
    defer r.CloseRPCServer()

    RPCCodec, _ := RpcCodecClientWithPort(r.TestPort)
    RPCClient := rpc.NewClientWithCodec(RPCCodec)
    defer RPCClient.Close()

    var result string
    err := RPCClient.Call("TestMaster.Send", "Got RPC?", &result)
    if err != nil {
        t.Fatalf("Error while trying to send RPC message: %s", err.Error())
    }

    if !r.GotRPC {
        t.Fatalf("Could not send correct message over RPC")
    }
}

Not sure if I'm just mishandling the connection or something of the like, any help would be much appreciated.

For the Record The RPC api does receive the correct string message

Answer 1

While not the source of your problems, your test configuration has a few race conditions which you should take care of before they cause problems. Always check for issues with the -race option. You should also let the OS allocate the port so you don't run into conflicts. See for example how httptest.Server works.

Your failure here is that you're not creating a new rpc.Server for each test, you're reusing the rpc.DefaultServer. The first call to CreateRPCServer registers a TestAPI under the name TestMaster. Each subsequent call uses the already registered instance.

If you create a new rpc.Server each time you setup the test and register a new TestAPI, the final test will pass.

srv := rpc.NewServer()
srv.RegisterName("TestMaster", testAPI)

...
// and then use srv to handle the new connection
srv.ServeCodec(RpcCodecServer(conn))