How to create an escape hatch in swift's Encodable

I'm looking for a way to create an escape hatch in Encodable. There are many JSON APIs that are more flexible than Encodable allows for. As an example, take the Anthropic API which allows you to specify a JSON schema as part of the request body. This schema is not known ahead of time to the programmer, as the end user can craft it. See the input_schema field here: https://docs.anthropic.com/en/docs/build-with-claude/tool-use

My goal is to enforce a strict contract where it makes sense (fixed fields) and allow for an escape hatch of [String: Any] when the structure if flexible. These aims are hard to achieve. I have been working on it for two days, writing various encoder hacks trying to get the desired output. What I have found is that it's trivial to stay fully in the untyped world or fully in the strict contract world:

This is very flexible, and works out of the box:

let myTree: [String: Any] = [
    "a": "xyz",
    "b": [
        "c": 2,
        "d": false
    ]
]

do {
    let data = try JSONSerialization.data(withJSONObject: myTree)
    print(String(decoding: data, as: UTF8.self))
} catch {
    print("Could not convert flexible dictionary to JSON: \(error)")
}

This is very rigid, and works out of the box:

struct Root: Encodable {
    let a: String
    let b: Child
}

struct Child: Encodable {
    let c: Int
    let d: Bool
}

let myTree = Root(a: "xyz", b: Child(c: 2, d: false))

do {
    let encoder = JSONEncoder()
    encoder.outputFormatting = .sortedKeys
    let data = try encoder.encode(myTree)
    print(String(decoding: data, as: UTF8.self))
} catch {
    print("Could not convert encodable struct to JSON: \(error)")
}

If you run both of those, you'll see that the two print statements produce the same JSON, great! Now assume that the structure of field b isn't known ahead of time, e.g. in the case of an API where the user specifies the schema. I want to do this:

struct RootWithEscapeHatch: Encodable {
    let a: String
    let b: [String: Any]

    private enum Fields: CodingKey {
        case a
        case b
    }

    func encode(to encoder: any Encoder) throws {
        var container = encoder.container(keyedBy: Fields.self)
        try container.encode(self.a, forKey: .a)
        // try container.encode(self.b, forKey: .b)  <-- What goes here?
    }
}


let myFailingTree = RootWithEscapeHatch(a: "xyz", b: ["c": 2, "d": false])
do {
    let data = try JSONEncoder().encode(myFailingTree)
    print(String(decoding: data, as: UTF8.self))
} catch {
    print("Could not convert encodable with escape hatch to JSON: \(error)")
}

You can see that myFailingTree is isomorphic with myTree examples above. I want the print statements to produce the same JSON. If you have an idea for what goes in the "What goes here?" line, please let me know. I'm looking for a general solution, I.e. I don't want to hard code that the structure will always be ["c": 2, "d": false]. The point is that any [String: Any] field should be serializable just like the first example in this question.

Thanks!

Solution

You can use a custom JSONValue type instead of [String: Any]:

enum JSONValue {
  case none
  case bool(Bool)
  case int(Int)
  case double(Double)
  case string(String)
  indirect case array([JSONValue])
  indirect case object([String: JSONValue])
}

make it Encodable:

extension JSONValue: Encodable {
  private struct CodingKeys: CodingKey {
    var stringValue: String
    init(stringValue: String) {
      self.stringValue = stringValue
    }

    var intValue: Int? = nil
    init?(intValue: Int) { return nil }
  }

  func encode(to encoder: Encoder) throws {
    switch self {
    case .none:
      var container = encoder.singleValueContainer()
      try container.encodeNil()
    case .bool(let value):
      var container = encoder.singleValueContainer()
      try container.encode(value)
    case .int(let value):
      var container = encoder.singleValueContainer()
      try container.encode(value)
    case .double(let value):
      var container = encoder.singleValueContainer()
      try container.encode(value)
    case .string(let value):
      var container = encoder.singleValueContainer()
      try container.encode(value)
    case .array(let values):
      var container = encoder.unkeyedContainer()
      for value in values { try container.encode(value) }
    case .object(let dictionary):
      var container = encoder.container(keyedBy: CodingKeys.self)
      for (key, value) in dictionary {
        try container.encode(value, forKey: .init(stringValue: key))
      }
    }
  }
}

and, optionally, easier to type by conforming it to the various ExpressibleBy…Literal protocols:

extension JSONValue: ExpressibleByNilLiteral {
  init(nilLiteral: ()) {
    self = .none
  }
}

extension JSONValue: ExpressibleByBooleanLiteral {
  init(booleanLiteral value: BooleanLiteralType) {
    self = .bool(value)
  }
}

extension JSONValue: ExpressibleByIntegerLiteral {
  init(integerLiteral value: IntegerLiteralType) {
    self = .int(value)
  }
}

extension JSONValue: ExpressibleByFloatLiteral {
  init(floatLiteral value: FloatLiteralType) {
    self = .double(value)
  }
}

extension JSONValue: ExpressibleByStringLiteral {
  init(stringLiteral value: StringLiteralType) {
    self = .string(value)
  }
}

extension JSONValue: ExpressibleByArrayLiteral {
  init(arrayLiteral elements: JSONValue...) {
    self = .array(elements)
  }
}

extension JSONValue: ExpressibleByDictionaryLiteral {
  init(dictionaryLiteral elements: (String, JSONValue)...) {
    self = .object(.init(uniqueKeysWithValues: elements))
  }
}

With JSONValue you can then get what you're looking for as follows:

struct RootWithEscapeHatch: Encodable {
  let a: String
  let b: JSONValue
}

let tree = RootWithEscapeHatch(a: "xyz", b: ["c": 2, "d": false])
try print(String(data: JSONEncoder().encode(tree), encoding: .utf8)!)

It is more generale than providing [String: Any] since you now can place anything as JSONValue, e.g. nil, scalar types or eterogeneous arrays.