I've recently been interested in GNU Lightning but I was wondering if I have to use an assembler with it.
No. GNU lightning generate (poor, i.e. unoptimized) machine code in memory on the fly without using any external program (such as an assembler).
You should have read the documentation of GNU lightning before asking and look into the examples there.
Notice that libgccjit also generates possibly optimized machine code (reusing most of GCC internal middle end and back end passes) but (in some cases, and some releases at least) do require external resources such as the assembler. However, if you have correctly installed libgccjit you won't see them.