Search code examples
f#type-providers

TypeProvider failed to add method from Expression

I created a blank type provider using a .net core template dotnet new typeprovider -n LemonadeProvider -lang F#. The project builds, but when I want to change a provide method to invoke in an expression another method like that:

let allEntities () = "dd"
let meth = ProvidedMethod("AllEntities", [], typeof<string>, invokeCode = (fun _ -> <@@ allEntities () @@>))
myType.AddMember(meth)

which I try to test via following code:

type Connection = LemonadeProvider.GenerativeProvider<"5">


[<Fact>]
let ``AllEntities simply works`` () =
    let obj = Connection()
    Assert.true(obj.AllEntities() = "dd")

I get following error:

LemonadeProvider.Tests.fs(20,19): error FS3033: TypeProvider "LemonadeProviderImplementation+BasicGenerativeProvider" reports an error:: The design-time type 'LemonadeProviderImplementation+allEntities@96' utilized by a type provider was not found in the target reference assembly set '[tgt assembly Microsoft.VisualStudio.CodeCoverage.Shim, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly WitAi.Runtime, Version=1.0.0.0, Culture=neutral; tgt assembly xunit.abstractions, Version=2.0.0.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c; tgt assembly xunit.assert, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c; tgt assembly xunit.core, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c; tgt assembly xunit.execution.desktop, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c; tgt assembly System.ComponentModel.Annotations, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.ComponentModel, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.ComponentModel.EventBasedAsync, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Diagnostics.Contracts, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Dynamic.Runtime, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Linq.Parallel, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Net.NetworkInformation, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Runtime.InteropServices.WindowsRuntime, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Runtime.Serialization.Json, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Runtime.Serialization.Primitives, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Runtime.Serialization.Xml, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.ServiceModel.Duplex, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.ServiceModel.Http, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.ServiceModel.NetTcp, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.ServiceModel.Primitives, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.ServiceModel.Security, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Xml.XmlSerializer, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089; tgt assembly FSharp.Core, Version=4.4.3.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089; tgt assembly System.Data, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089; tgt assembly System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089; tgt assembly System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.IO.Compression.FileSystem, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089; tgt assembly System.Numerics, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089; tgt assembly System.Runtime.Serialization, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089; tgt assembly System.Xml, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089; tgt assembly System.Xml.Linq, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089; tgt assembly System.Collections.Concurrent, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Collections, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Diagnostics.Debug, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Diagnostics.Tools, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Diagnostics.Tracing, Version=4.0.20.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Globalization, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.IO, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Linq, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Linq.Expressions, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Linq.Queryable, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Net.Primitives, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Net.Requests, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Net.WebHeaderCollection, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.ObjectModel, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Reflection, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Reflection.Emit, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Reflection.Emit.ILGeneration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Reflection.Emit.Lightweight, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Reflection.Extensions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Reflection.Primitives, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Resources.ResourceManager, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Runtime, Version=4.0.20.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Runtime.Extensions, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Runtime.Handles, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Runtime.InteropServices, Version=4.0.20.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Runtime.Numerics, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Security.Principal, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Text.Encoding, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Text.Encoding.Extensions, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Text.RegularExpressions, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Threading, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Threading.Tasks, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Threading.Tasks.Parallel, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Threading.Timer, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Xml.ReaderWriter, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tgt assembly System.Xml.XDocument, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; tmp5A6]'. You may be referencing a profile which contains fewer types than those needed by the type provider you are using. [C:\git\LemonadeProvider.Tests.fsproj]

when a ProvideMethod simply return const value everything seems fine and I don't get an error:

let allEntities () = "dd"
let meth = ProvidedMethod("AllEntities", [], typeof<string>, invokeCode = (fun _ -> <@@ ("dd") @@>))
myType.AddMember(meth)

where whole code for provider looks like that:

[<TypeProvider>]
type BasicGenerativeProvider (config : TypeProviderConfig) as this =
    inherit TypeProviderForNamespaces (config, assemblyReplacementMap=[("LemonadeProvider.DesignTime", "LemonadeProvider.Runtime")])

    let ns = "LemonadeProvider"
    let asm = Assembly.GetExecutingAssembly()

    // check we contain a copy of runtime files, and are not referencing the runtime DLL
    do assert (typeof<DataSource>.Assembly.GetName().Name = asm.GetName().Name)  

    let createType typeName (auth: string, version: string) =

        let asm = ProvidedAssembly()
        let myType = ProvidedTypeDefinition(asm, ns, typeName, Some typeof<obj>, isErased=false)

        let ctor = ProvidedConstructor([], invokeCode = fun args -> <@@ "My internal state" :> obj @@>)
        myType.AddMember(ctor)

        let ctor2 = ProvidedConstructor([ProvidedParameter("InnerState", typeof<string>)], invokeCode = fun args -> <@@ (%%(args.[1]):string) :> obj @@>)
        myType.AddMember(ctor2)
        let allEntities () = "dd"
        let meth = ProvidedMethod("AllEntities", [], typeof<string []>, invokeCode = (fun _ -> <@@ allEntities () @@>))
        myType.AddMember(meth)

        asm.AddTypes [ myType ]

        myType

    let myParamType = 
        let t = ProvidedTypeDefinition(asm, ns, "GenerativeProvider", Some typeof<obj>, isErased=false)
        t.DefineStaticParameters( [ProvidedStaticParameter("AuthToken", typeof<string>)], fun typeName args -> createType typeName ("token", "version"))
        t
    do
        this.AddNamespace(ns, [myParamType])
Solution

  • My observations:

    1. When you compile tests, LemonadeProvider.DesignTime dll is participated in fsc pipeline (before actual compillation) to generate types based on createType definition you provided.

    2. Types are generated from LemonadeProvider.DesignTime in separate assembly (that seems to be anonymous), which is included in a process of actual compilation of tests sources along with LemonadeProvider.Runtime.dll. Note, that LemonadeProvider.DesignTime seems to be not included on this step any more. First I thought that LemonadeProvider.Runtime.dll contains generated types, but after check in ilspy I did not find any.

    3. LemonadeProvider.DesignTime is not included for actual source compilation. It means that any type defined here (like definition of module Helpers) will not be available. But when you write: let allEntities() = "dd" - this generates a class inside LemonadeProvider.DesignTime. In my case it was:

    generated func type

    but this class is not reachable

    1. Line:
    let meth = ProvidedMethod("AllEntities", [], typeof<string>, invokeCode = (fun _ -> <@@ allEntities() @@>))

    quotation form AST similar to: enter image description here - note that Type is captured inside AST. So, when deserialization of quotation happens in context where there is no DesignTime.dll, you see a compilation error.

    1. For code:
    let allEntities = "dd"
let allEntitiesQuote = <@@ allEntities @@>

    we have next AST: ValueWithName ("dd", allEntities) - so there is no reference to type.

    1. To include function inside quotation - use DataSource class. It is defined also in Runtime dll, so corresponding generated function type will be found. In Runtime.fsproj Runtime.fs:
    type DataSource(filename:string) = 
    member this.FileName = filename
    static member allEntities() = "dd"

    this worked for me. Also you can have allEntities in Utilities module but remove internal keyword.

    1. If you still want to have function definition in DesignTime dll I assume you need to organize it in a form of quotation and provide it with Provided* methods.