Search code examples
deep-learningnlppytorchembedding

What is the Difference between asssigning layer and assigning weight of layer?


Imagine i have two code examples:

model.embedding = new_emb

and

model.shared.weight = new_emb.weight

What would be the difference between them?


Solution

  • If you assign a layer weight, the assigned layer weight has to have the same shape as the current layer weight.

    If you assign a new layer, you can assign any layer (as long as it can handle the input tensor shape and return the correct output tensor shape).