TextClip animations (e.g. adding text with moving letters to a video) not working, except for the demo sentence in docs

Question

I took this example of text effects applied to video using moviepy and ran it locally, only changing the text itself, and it doesn't work with any sort of text I entered. For example, if I replace Cool Effect with Quick Effect the video will show Q ff instead.

Likewise, The quick brown fox jumped over the lazy dogs becomes f.

Source of this code example: https://zulko.github.io/moviepy/examples/moving_letters.html is the author of moviepy.

import numpy as np

from moviepy.editor import *
from moviepy.video.tools.segmenting import findObjects

# WE CREATE THE TEXT THAT IS GOING TO MOVE, WE CENTER IT.

screensize = (720,460)
txtClip = TextClip('Cool effect',color='white', font="Amiri-Bold",
                   kerning = 5, fontsize=100)
cvc = CompositeVideoClip( [txtClip.set_pos('center')],
                        size=screensize)

# THE NEXT FOUR FUNCTIONS DEFINE FOUR WAYS OF MOVING THE LETTERS


# helper function
rotMatrix = lambda a: np.array( [[np.cos(a),np.sin(a)], 
                                 [-np.sin(a),np.cos(a)]] )

def vortex(screenpos,i,nletters):
    d = lambda t : 1.0/(0.3+t**8) #damping
    a = i*np.pi/ nletters # angle of the movement
    v = rotMatrix(a).dot([-1,0])
    if i%2 : v[1] = -v[1]
    return lambda t: screenpos+400*d(t)*rotMatrix(0.5*d(t)*a).dot(v)
    
def cascade(screenpos,i,nletters):
    v = np.array([0,-1])
    d = lambda t : 1 if t<0 else abs(np.sinc(t)/(1+t**4))
    return lambda t: screenpos+v*400*d(t-0.15*i)

def arrive(screenpos,i,nletters):
    v = np.array([-1,0])
    d = lambda t : max(0, 3-3*t)
    return lambda t: screenpos-400*v*d(t-0.2*i)
    
def vortexout(screenpos,i,nletters):
    d = lambda t : max(0,t) #damping
    a = i*np.pi/ nletters # angle of the movement
    v = rotMatrix(a).dot([-1,0])
    if i%2 : v[1] = -v[1]
    return lambda t: screenpos+400*d(t-0.1*i)*rotMatrix(-0.2*d(t)*a).dot(v)



# WE USE THE PLUGIN findObjects TO LOCATE AND SEPARATE EACH LETTER

letters = findObjects(cvc) # a list of ImageClips


# WE ANIMATE THE LETTERS

def moveLetters(letters, funcpos):
    return [ letter.set_pos(funcpos(letter.screenpos,i,len(letters)))
              for i,letter in enumerate(letters)]

clips = [ CompositeVideoClip( moveLetters(letters,funcpos),
                              size = screensize).subclip(0,5)
          for funcpos in [vortex, cascade, arrive, vortexout] ]

# WE CONCATENATE EVERYTHING AND WRITE TO A FILE

final_clip = concatenate_videoclips(clips)
final_clip.write_videofile('../../coolTextEffects.avi',fps=25,codec='mpeg4')

I have poured over this snippet and don't see where the "text" itself could be changing. Probably something to do with the letters variable and findObjects, which locates each letter in the TextClip and loops through them.

But I'd like to know more about moviepy and why these 4 effects don't work generally. I have looked through github and youtube, over hundreds of projects, to find someone who demonstrates these, but haven't found any better examples.

If you solve this, I'll probably release a public github package / repo that makes it easier to apply these to video. The docs for moviepy almost entirely omit the TextClip feature.

enter link description here

== UPDATE ==

After a suggestion from @Rotem, I tried messing with the findObjects(clip, rem_thr) rem_thr param. It defaults to 500, meaning that detected objects smaller than 500 are ignored. Changing that to any of these other thresholds (500, 250, 50, 10, 1) doesn't solve the issue:

500 (test text arrive wiggly) findObjects found 18 letters from string of length 23
250 (test text arrive wiggly) findObjects found 20 letters from string of length 23
50 (test text arrive wiggly) findObjects found 22 letters from string of length 23
10 (test text arrive wiggly) findObjects found 22 letters from string of length 23
1 (test text arrive wiggly) findObjects found 22 letters from string of length 23

(I realized after that there are only 20 letters in the string, but I was check for number of characters, including the whitespace. So 22 matches number of letters plus dots over lower case is. And 20 is the desired number. So tweaking with the rem_thr is important, but there's a different optimal detection size for every font size. That's tricky, but demands a better wrapper function to manage.)

What DOES help is maximizing the contrast with letters:

caption = TextClip("some text",
    color='white',
    bg_color="black",
    kerning=5,
    fontsize=33)

With bg_color was not specified, it only found one letter regardless of rem_thr.

Setting bg_color to black helped it find letters. Perhaps I need to use a blocky fat font for this to work, white on black, and then remove the background before applying it to the video.

But this demo is definitely hard to use reliably.

Answer 1

Replace letters = findObjects(cvc) with: letters = findObjects(cvc, 50).

The method findObjects gets a second optional argument: rem_thr=500.
Argument description:

rem_thr : all objects found with size < rem_Thr will be considered false positives and will be removed

When using 'Quick Effect' instead of 'Cool effect', the size of most of the objects is less than 500.
Reducing rem_thr value to 50 solves the issue.

---