How do I show sprites in the border on C64?

I've seen cool C64 demos showing sprites in the border area of the screen. It shouldn't be possible; I think they managed to fool the graphics chip somehow. How exactly did they do it?

Solution

Yes, you need assembler. It's an interrupt timing trick. The VIC is able to show sprites in the border, but the frame is just hiding them, so the sprites slides behind it. It's connected to scan lines displayed by the VIC.

For lower/upper borders it's quite simple:

Program an interrupt, synced to start at a certain scan line, 7 pixel or something like that before the lower border.
Set the register in VIC to make the border smaller. (There is a register that can do that.)
VIC now believes that the border already started and does not start to paint it.
-> No border at the bottom.
Program another interrupt after the real border to set it back to original.

For sprites in the left/right borders, it's more complicated because the process has to be repeated for every scan line:

Program an interrupt, synced to start at a certain scan line.
Then do some NOPs until you are 7 pixels before the right border.
Set the register in VIC to make the border smaller.
-> No border at the right side.
Do some NOPs until you are after the real border and set the register back to original value.
Again do some NOPs until step 2.

The problem is that all these NOPs are busy waits and steal the cycles you have for your stuff.

I was able to find some code for you, from an sprite scroller in the lower border.
Here is the code. (It was ripped from some demo.)

C198  78        SEI
C199  20 2E C1  JSR C12E     # clear sprite area
C19C  20 48 C1  JSR C148     # init VIC
C19F  A9 BF     LDA #BF      # set up IRQ in C1BF
C1A1  A2 C1     LDX #C1
C1A3  8D 14 03  STA 0314
C1A6  8E 15 03  STX 0315
C1A9  A9 1B     LDA #1B
C1AB  8D 11 D0  STA D011
C1AE  A9 F7     LDA #F7
C1B0  8D 12 D0  STA D012
C1B3  A9 01     LDA #01
C1B5  8D 1A D0  STA D01A
C1B8  A9 7F     LDA #7F
C1BA  8D 0D DC  STA DC0D
C1BD  58        CLI
C1BE  60        RTS

----------------------------------
# init VIC
C148  A2 00     LDX #00
C14A  BD 88 C1  LDA C188,X
C14D  9D 00 D0  STA D000,X   # set first 16 values from table
C150  E8        INX
C151  E0 10     CPX #10
C153  D0 F5     BNE C14A
C155  A9 FF     LDA #FF
C157  8D 15 D0  STA D015
C15A  A9 00     LDA #00
C15C  8D 1C D0  STA D01C
C15F  A9 FF     LDA #FF
C161  8D 17 D0  STA D017
C164  8D 1D D0  STA D01D
C167  A9 C0     LDA #C0
C169  8D 10 D0  STA D010
C16C  A9 F8     LDA #F8
C16E  A2 00     LDX #00
C170  9D F8 07  STA 07F8,X
C173  18        CLC
C174  69 01     ADC #01
C176  E8        INX
C177  E0 08     CPX #08
C179  D0 F5     BNE C170
C17B  A9 0E     LDA #0E
C17D  A2 00     LDX #00
C17F  9D 27 D0  STA D027,X
C182  E8        INX
C183  E0 08     CPX #08
C185  D0 F8     BNE C17F
C187  60        RTS

----------------------------------
# data set into VIC registers
C188  00 F7 30 F7 60 F7 90 F7
C190  C0 F7 F0 F7 20 F7 50 F7

----------------------------------
# main IRQ routine
C1BF  A2 08     LDX #08
C1C1  CA        DEX
C1C2  D0 FD     BNE C1C1
C1C4  A2 28     LDX #28      # 40 or so lines
C1C6  EA        NOP          # "timing"
C1C7  EA        NOP
C1C8  EA        NOP
C1C9  EA        NOP
C1CA  CE 16 D0  DEC D016     # fiddle register
C1CD  EE 16 D0  INC D016
C1D0  AC 12 D0  LDY D012
C1D3  88        DEY
C1D4  EA        NOP
C1D5  98        TYA
C1D6  29 07     AND #07
C1D8  09 18     ORA #18
C1DA  8D 11 D0  STA D011
C1DD  24 EA     BIT   EA
C1DF  EA        NOP
C1E0  EA        NOP
C1E1  CA        DEX
C1E2  10 E4     BPL C1C8     # repeat next line
C1E4  A9 1B     LDA #1B
C1E6  8D 11 D0  STA D011
C1E9  A9 01     LDA #01
C1EB  8D 19 D0  STA D019
C1EE  20 00 C0  JSR C000     # call main code
C1F1  4C 31 EA  JMP EA31     # finish IRQ