Search code examples
clojure

Why does my code generate a list of empty lists?

I have following data structure: There are scenes which can be parts of sequences.

For example, let's say we have scene sc026:

{:sceneId "sc026"}

It is part of seq07:

(def seq07
  {
   :SeqId    "seq07"
   :Desc     "Sequence name"
   :Scenes   [sc026]
   :Comments []

   }
  )

Given a list of scenes, I want to create a list which for every scene contains a list of IDs of sequences that a particular scene is part of.

Example

Let's assume there is a list of two scenes sc026 and sc027. sc026 is part of seq07, sc027 is not part of any sequence.

The result I want to achieve is this: [["seq07"], []].

What I tried to implement

I have a function generate-scene-overview which, among others, needs to create that list. It has following signature:

(defn- generate-scene-overview
  [scene-list time-info seqs]

scene-list is the collection of scenes (result of (filter some? my-scene-list) where my-scene-list is a list of scenes that contains nil elements).

seqs is a list of sequences.

Sequences in seqs can be structured and unstructured. The unstructured ones have a non-empty list in :Scenes field.

Therefore, in generate-scene-overview I first extract the unstructured scenes from seqs:

unstructured-seqs (filter
                    (fn [cur-seq]
                      (let
                        [scenes (get cur-seq :Scenes)]
                        (not (empty? scenes))
                        )
                      )
                    seqs)

Next I need to convert the unstructured sequences into a collection of scene-sequence tuples:

unstructured-seq-tuples (compose-unstructured-tuple-list unstructured-seqs)

compose-unstructured-tuple-list is defined as follows.

(defn- compose-unstructured-tuple-list
  [unstructured-seqs]
  (into []
        (map
          (fn [cur-seq]
            (let
              [
               scenes (get cur-seq :Scenes)
               seqId (get cur-seq :SeqId)
               scene-seq-tuples (into []
                                      (map
                                        (fn [cur-scene]
                                          (let [scene-id (get cur-scene :sceneId)]

                                            {
                                             :SceneId scene-id
                                             :SeqId   seqId
                                             }

                                            )
                                          )
                                        scenes
                                        )
                                      )

               ]
              scene-seq-tuples
              )
            )
          )
        unstructured-seqs
        )
  )

Next, I need to combine the tuples for structured sequences with those from unstructured ones:

seq-tuples (set/union unstructured-seq-tuples structured-seq-tuples)

Finally, seq-tuples are converted into a list of sequence IDs for each scene:

scene-seqs (compose-scene-seqs scene-list seq-tuples)

compose-scene-seqs is defined as follows:

(defn compose-scene-seqs
  [scene-list seq-tuples]
  (into [] (map (fn [cur-scene]
                  (let
                    [scene-id (get cur-scene :sceneId)]
                    (findSeqIdsBySceneId scene-id seq-tuples)
                    )
                  )
                scene-list
                )
        )
  )

findSeqIdsBySceneId looks like this:

(defn findSeqIdsBySceneId
  [scene-id seq-tuples]
  (let
    [
     scene-tuples (filter (fn [cur-tuple]
                            (let [cur-tuple-scene-id (get cur-tuple :SceneId)]
                              (= scene-id cur-tuple-scene-id))
                            )
                          seq-tuples
                          )
     seqs (map (fn [cur-tuple]
                 (get cur-tuple :SeqId)
                 )
               scene-tuples
               )
     ]
    seqs
    )
  )

My problem

When I run the above code in debugger, scene-seqs only contains empty collections.

It should contain exactly one non-empty collection for scene sc026 (with string seq07 inside it).

Screenshot 1

How I tried to diagnose the problem

I tried to reproduce the problem in automated tests.

First attempt -- findSeqIdsBySceneId:

(deftest findSeqIdsBySceneId-test
  (is (= ["seq07"]
         (findSeqIdsBySceneId "sc026" [{

                                        :SceneId "sc026"
                                        :SeqId   "seq07"
                                        }])
         )
      )
  (is (= ["seq07", "seq06"]
         (findSeqIdsBySceneId "sc026" [{

                                        :SceneId "sc026"
                                        :SeqId   "seq07"
                                        }
                                       {

                                        :SceneId "sc026"
                                        :SeqId   "seq06"
                                        }
                                       ])
         )
      )

  )

Those tests run, so I wrote a couple of tests for compose-scene-seqs:

(deftest compose-scene-seqs-test
  (is (= [["seq07"]]
         (let
           [
            scene-list [{:sceneId "sc026"}]
            seq-tuples [
                        {

                         :SceneId "sc026"
                         :SeqId   "seq07"
                         }
                        ]
            ]
           (compose-scene-seqs scene-list seq-tuples)
           )

         ))

  )

(deftest compose-scene-seqs-test2
  (is (= [["seq07"] []]
         (let
           [
            scene-list [
                        {:sceneId "sc026"}
                        {:sceneId "sc027"}
                        ]
            seq-tuples [
                        {

                         :SceneId "sc026"
                         :SeqId   "seq07"
                         }
                        ]
            ]
           (compose-scene-seqs scene-list seq-tuples)
           )

         ))

  )

(deftest compose-scene-seqs-test3
  (is (= [[] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] ["seq07"] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] []]
         (let
           [
            scene-list   my-scene-list
            seq-tuples [
                        {

                         :SceneId "sc026"
                         :SeqId   "seq07"
                         }
                        ]
            ]
           (compose-scene-seqs scene-list seq-tuples)
           )

         ))

  )

All of them run.

If I replace

scene-list   my-scene-list

with

scene-list   (filter some? перечень-сцен2)

I get the following assertion error, but even then there is one non-empty collection:

Screenshot 2

Question

What else can I do to diagnose and fix the error?

Update 1:

Full code is available in this GitHub gist.

I managed to reproduce the error in the following test:

(deftest compose-scene-seqs-test4
  (is (= [[] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] ["seq07"] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] []]
         (let
           [
            scene-list   (filter some? перечень-сцен2)
            unstructured-seqs [seq07]
            unstructured-seq-tuples (compose-unstructured-tuple-list unstructured-seqs)
            seq-tuples (set/union unstructured-seq-tuples [])
            ]
           (compose-scene-seqs scene-list seq-tuples)
           )
         )
      )
  )
Solution

  • Here is a complete working solution for the stated task:

    (defn- contains-scene? [seq scene-id]
  (some #(= scene-id (:sceneId %)) (:Scenes seq)))

(defn- seq-ids-containing-scene [seqs scene-id]
  (keep #(and (contains-scene? % scene-id) (:SeqId %)) seqs))

(defn seq-ids-containing-scenes [seqs scenes]
  (map #(seq-ids-containing-scene seqs (:sceneId %)) scenes))

    Test case:

    (def sc026 {:sceneId "sc026"})

(def sc027 {:sceneId "sc027"})

(def seq07 {:SeqId    "seq07"
            :Desc     "Sequence name"
            :Scenes   [sc026]
            :Comments []})

(seq-ids-containing-scenes [seq07] [sc026 sc027]) ;; => (("seq07") ())

    I couldn't follow the logic of the attempted solution. It introduces a concept "unstructured" (not the same as Clojure's destructuring) which doesn't seem to add value. I tried re-creating the issue but found the presented code was incomplete, so I can't offer any help on why it fails.

    Here is a second alternative solution which builds a map scene-map in a single pass over the collection of sequences. scene-map has scene id key and a collection of sequence ids as the corresponding value:

    (defn seq-ids-containing-scenes* [seqs scenes]
  (let [maps (for [seq seqs
                   scene (:Scenes seq)]
               {(:sceneId scene) [(:SeqId seq)]})
        scene-map (apply merge-with into maps)]
    (map #(get scene-map (:sceneId %) []) scenes)))

    Update:

    I found the bug in the original code presented in the question. In function compose-unstructured-tuple-list, replace the first map with mapcat.