Does shapeless play well with type members?

Question

I'm trying to use shapeless to derive a Generic for a type member defined in a trait, but am not having luck. I have made as simple of a reproduction of the issue as I can think of while keeping it close enough to the original code. I'm taking inspiration from this blog post, and trying to expand (bastardize) it to be a bit more generic. It probably won't make sense why I have code that looks like this from this example alone, but hopefully that doesn't take away from this question :)

I have a trait that declares a type member, a case class representing some common set of fields, and another wrapper case class that combines an instance of both:

object A {
    trait TheTrait {
        type TheType
    }

    case class CommonFields(height: Double, isTall: Boolean)
    case class Wrapper[T <: TheTrait](commonFields: CommonFields, t: T#TheType)
}

I also have an implementation of the trait:

trait Obj extends TheTrait
object Obj extends Obj {
    case class Source(name: String, other: Int)

    override type TheType = Source 
}

My goal is to be able to take a tuple with values for both CommonFields and TheTrait#TheType for some instance of TheTrait, and to use shapeless to turn that into an instance of a Wrapper. So for the example so far, I'd like to go from something like (5.1, false, "sub", 10) to Wrapper[Obj](CommonFields(5.1, false), Source("other", 10)). Here's what I've come up with:

object Test {
    class Constructor[T <: TheTrait] {
        // take a tuple of all the fields of CommonFields and T#Trait and produce an instance of each in a A.Wrapper
        def apply[In <: Product, All <: HList, ORep <: HList, CRep <: HList, N <: Nat](in: In)(implicit
            cGen: Generic.Aux[A.CommonFields, CRep], // generic for CommonFields
            cLen: Length.Aux[CRep, N], // the length of the CommonFields generic HList
            trGen: Generic.Aux[T#TheType, ORep], // generic for T#TheType
            iGen: Generic.Aux[In, All], // generic for input tuple
            split: Split.Aux[All, N, CRep, ORep] // the input tuple, split at N, produces HLists for the CommonFields generic rep as well as for the T#TheType generic rep
        ): A.Wrapper[T] = {
            val all = iGen.to(in)
            val (cFields, tFields) = split(all)
            val com = cGen.from(cFields)
            val tr = trGen.from(tFields)
            
            A.Wrapper(com, tr)
        }
    }
    def construct[T <: TheTrait] = new Constructor[T]
    
    println(construct[Obj](5.1, false, "sub", 10))
}

Unfortunately, the correct implicits cannot be found, in particular I see the following error: No implicit arguments of type: hlist.Split.Aux[Double :: Boolean :: String :: Int :: HNil, Succ[Succ[shapeless.nat._0]], Double :: Boolean :: HNil, HNil] It seems like it is finding the right generic representation for CommonFields (by the appearances of Double :: Boolean :: HNil in the error), but cannot tell what the TheType should be. Is this asking too much of shapeless/the scala compiler? Can I give more type hints somewhere? Is there another way to achieve something like this? I can try to expand on why I have created a type structure like this if that would be helpful. Any ideas are appreciated!

EDIT:

Just to experiment, I made a variation using path dependent typing instead of the type projection, but still was unable to get it to work:

object Test {
    import A._
    class Constructor[T <: TheTrait] {
        // take a tuple of all the fields of CommonFields and T#Trait and produce an instance of each in a A.Wrapper
        def apply[In <: Product, All <: HList, ORep <: HList, CRep <: HList, N <: Nat](in: In, t: T /* <=== now takes a `T` instance */)(implicit
            cGen: Generic.Aux[CommonFields, CRep], // generic for CommonFields
            cLen: Length.Aux[CRep, N], // the length of the CommonFields generic HList
            trGen: Generic.Aux[t.TheType, ORep], // generic for T#TheType <==== no more type projection
            iGen: Generic.Aux[In, All], // generic for input tuple
            split: Split.Aux[All, N, CRep, ORep] // the input tuple, split at N, produces HLists for the CommonFields generic rep as well as for the T#TheType generic
        ): Wrapper[T] = {
            val all = iGen.to(in)
            val (cFields, tFields) = split(all)
            val com = cGen.from(cFields)
            val tr = trGen.from(tFields)
            
            Wrapper(com, tr)
        }
    }
    def construct[T <: TheTrait] = new Constructor[T]

    println(
        construct[Obj]((5.1, false, "sub", 10), Obj) // <== passing a `TheTrait` instance
    )
}

But still seeing the error

No implicit arguments of type: hlist.Split.Aux[Double :: Boolean :: String :: Int :: HNil, Succ[Succ[shapeless.nat._0]], Double :: Boolean :: HNil, HNil]

EDIT 2: Rearranging the implicits has helped a tad. Instead of the compiler believing that ORep is HNil, it is now at least looking for it to match String :: Int :: HNil:

    class Constructor[T <: TheTrait] {
        // take a tuple of all the fields of CommonFields and T#Trait and produce an instance of each in a A.Wrapper
        def apply[In <: Product, All <: HList, ORep <: HList, CRep <: HList, N <: Nat](in: In, t: T)(implicit
            cGen: Generic.Aux[CommonFields, CRep], // generic for CommonFields
            cLen: Length.Aux[CRep, N], // the length of the CommonFields generic HList
            iGen: Generic.Aux[In, All], // generic for input tuple
            split: Split.Aux[All, N, CRep, ORep], // the input tuple, split at N, produces HLists for the CommonFields generic rep as well as for the T#TheType generic
            trGen: Generic.Aux[t.TheType, ORep] // generic for T#TheType
        ): Wrapper[T] = {
            val all = iGen.to(in)
            val (cFields, tFields) = split(all)
            val com = cGen.from(cFields)
            val tr = trGen.from(tFields)

            Wrapper(com, tr)
        }
    }
    def construct[T <: TheTrait] = new Constructor[T]

The error now is No implicit arguments of type: Generic.Aux[B.Obj.TheType, String :: Int :: HNil], which feels like progress to me.

I can now actually get the program to compile by explicitly creating an instance of the Generic it is looking for and making it available implicitly:

    implicit val objTypeGen: Generic.Aux[Obj.TheType, String :: Int :: HNil] = Generic.instance[Obj.TheType, String :: Int :: HNil](
        t => t.name :: t.other :: HNil,
        {
            case n :: o :: HNil => Obj.Source(n, o)
        }
    )

But now I have removed all of the ergonomics I had originally set out to build. Hopefully there's enough hints here though for someone to figure out how to not need to explicitly pass a TheTrait instance or define the Generic manually?

Answer 1

Your original code compiles as soon as you move override type TheType = Source from object Obj to trait Obj. Note that construct[Obj](..) refers to the type (trait) Obj, not the singleton type corresponding to the object Obj, therefore in your code Obj#TheType cannot be resolved to a particular type, since it remains abstract.

If you really need override type TheType = Source in object Obj, the invocation construct[Obj.type](..) will also compile.

Edit. Full code:

import shapeless.ops.hlist.{Length, Split}
import shapeless.{Generic, HList, Nat}

object Programme {

  object A {
    trait TheTrait {
      type TheType
    }

    case class CommonFields(height: Double, isTall: Boolean)
    case class Wrapper[T <: TheTrait](commonFields: CommonFields, t: T#TheType)
  }

  import A.TheTrait

  trait Obj extends TheTrait {
    override type TheType = Obj.Source
  }

  object Obj extends Obj {
    case class Source(name: String, other: Int)
  }

  object Test {
    class Constructor[T <: TheTrait] {
      def apply[In <: Product, All <: HList, ORep <: HList, CRep <: HList, N <: Nat](in: In)(implicit
          cGen: Generic.Aux[A.CommonFields, CRep],
          cLen: Length.Aux[CRep, N],
          trGen: Generic.Aux[T#TheType, ORep],
          iGen: Generic.Aux[In, All],
          split: Split.Aux[All, N, CRep, ORep]
      ): A.Wrapper[T] = {
        val all = iGen.to(in)
        val (cFields, tFields) = split(all)
        val com = cGen.from(cFields)
        val tr = trGen.from(tFields)

        A.Wrapper(com, tr)
      }
    }
    def construct[T <: TheTrait] = new Constructor[T]
  }

  import Test.construct

  def main(args: Array[String]): Unit = {
    println(construct[Obj](5.1, false, "sub", 10))
  }
}