Why is `functionArgs` implemented twice? (i.e, as a primop and in `lib`)

Trying to understand callPackage, so looked up its implementation where it uses lib.functionArgs (source), but there is already a builtins.functionArgs primop, an alias of __functionArgs (implemented in C).

lib.functionArgs is defined as

   /* Extract the expected function arguments from a function.
      This works both with nix-native { a, b ? foo, ... }: style
      functions and functions with args set with 'setFunctionArgs'. It
      has the same return type and semantics as builtins.functionArgs.
      setFunctionArgs : (a → b) → Map String Bool.
   */

    functionArgs = f: f.__functionArgs or (builtins.functionArgs f);

and the __functionArgs attribute above is coming from setFunctionArgs (source):

  /* Add metadata about expected function arguments to a function.
     The metadata should match the format given by
     builtins.functionArgs, i.e. a set from expected argument to a bool
     representing whether that argument has a default or not.
     setFunctionArgs : (a → b) → Map String Bool → (a → b)

     This function is necessary because you can't dynamically create a
     function of the { a, b ? foo, ... }: format, but some facilities
     like callPackage expect to be able to query expected arguments.
  */

  setFunctionArgs = f: args:
    {
      __functor = self: f;
      __functionArgs = args;
    };

I understand what setFunctionArgs does, and the comment above its declaration tells why it is necessary, but I can't understand it; both clauses of that sentence are clear but not sure how the first statement prevents the second one to be achieved (without setFunctionArgs, that is).

danbst also tried to elucidate this further,

lib.nix adds __functionArgs attr to mimic __functionArgs builtin. It used to "pass" actual __functionArgs result down to consumers, because builtin __functionArgs only works on top-most function args

but not sure what the "consumers" are, and couldn't unpack the last clause (i.e., "builtin __functionArgs only works on top-most function args"). Is this a reference to the fact that Nix functions are curried, and

nix-repl> g = a: { b, c }: "lofa"

nix-repl> builtins.functionArgs g
{ }

lib.functionArgs also doesn't solve this problem, but I'm probably off the tracks at this point.

Notes to self

__functor is documented in the Nix manual under Sets.

$ nix repl '<nixpkgs>'
Welcome to Nix version 2.3.6. Type :? for help.

Loading '<nixpkgs>'...
Added 11530 variables.

nix-repl> f = { a ? 7, b }: a + b

nix-repl> set_f = lib.setFunctionArgs f { b = 9; }

nix-repl> set_f
{ __functionArgs = { ... }; __functor = «lambda @ /nix/store/16blhmppp9k6apz41gjlgr0arp88awyb-nixos-20.03.3258.86fa45b0ff1/nixos/lib/trivial.nix:318:19»; }

nix-repl> set_f.__functionArgs
{ b = 9; }

nix-repl> set_f set_f.__functionArgs
16

nix-repl> set_f { a = 27; b = 9; }
36

Solution

lib.functionArgs wraps builtins.functionArgs in order to provide reflective access to generic functions.

This supports reflection with builtins.functionArgs:

f = { a, b, c }: #...

Now consider the eta abstraction of the same function:

f' = attrs: f attrs

This does not support reflection with builtins.functionArgs. With setFunctionArgs, you can restore that information, as long as you also use lib.functionArgs.

I recommend to avoid reflection because everything that I've seen implemented with it can be implemented without it. It expands the definition of a function to include what should normally be considered implementation details. Anyway, the primary motivation seems to be callPackage, which can be implemented with normal attrset operations if you change all packages to add ... as in { lib, stdenv, ... }:. I do have a morbid interest in this misfeature that is function reflection, so if anyone finds another use case, please comment.