We have had a cloudfront rtmp streaming service set up, which seems to work. Upon requesting a streaming url, we get a signed url, with which I've been able to view the stream in JWPlayer and the Wowza Testplayer - after waiting for 20-50 seconds.
Before that, the wowza testplayer says "Failed to play (stream ID: 1)" and JWPlayer says "Error loading stream: ID not found on server". After waiting some seconds (usually something around 20-50 seconds), the stream can be viewed.
The Amazon services were set up for us and I wasn't able to reach these people yet. I haven't found any other cases describing this problem and I would be happy for any pointers on where the problem could be (or what exactly the problem is).
Okay, the problem was that the server generating the signed urls and the streaming server had a slight time difference of ~40 seconds, which resulted in urls valid in +40 seconds on the streaming server. Including time synchronisation between the servers fixed this.