Split list of characters into list of words

Question

I am currently learning SML functional language and I am trying to make a function that takes a list of characters then it searches for any white spaces in the list when it finds a white space it concatinates the characters before the white space to make a string after that it returns a list of strings that are made of characters split by white spaces.

Here is my code but there is something wrong with it the compiler says that there is an error at eof !

fun sepWords ([]) = []
  | sepWords (x :: xs) =
    let 
      val word = "" 
      val list
      sepWords (xs)
    in 
      if (Char.isSpace (x)) then (word = "")
      else (word ^ x)
      word :: list
    end;

Answer 1

1. You have a syntax error around the lines val list and sepWords (xs). Perhaps you meant to write val list = sepWords (xs)? You also have a syntax error around the lines if ... and word :: list. I'm unsure what the intent here is, but maybe you think that word :: list will have the side-effect of adding 'word' to 'list'?

2. You have a type error in your if ... since the 'then' branch has the expression word = "" that has type bool and the 'else' branch has the expression word ^ x that has the type string. An if-then-else must have the same type on each branch for the type-checker to accept the program.

3. Instead of making a function with type char list -> string list, why not make a function with type string -> string list? If you do that, you can even avoid the intermediate step of converting your string into a char list by keeping track of character indexes into the original string (e.g. by using the substring type).

   A good name for that function could be 'words'.

4. You have not defined behavior for when multiple whitespace occurs consecutively. Should words "hello world" produce ["hello", "world"] or ["hello", "", "world"]?

5. There are actually built-in library functions that do this:

   - String.tokens Char.isSpace "hello  world";
   > val it = ["hello", "world"] : string list
   
   - String.fields Char.isSpace "hello  world";
   > val it = ["hello", "", "world"] : string list
   

6. The alternative of first converting your string to a char list is a fine exercise in list recursion even if the strategy isn't very efficient. You could go about this by solving the problem of extracting a single word from your input along with the remainder of the string:

   fun firstWord [] = ([], [])
     | firstWord (c::cs) =
       if Char.isSpace c
       then ([], cs)  (* throw away the space c *)
       else case firstWord cs of
              (fw, remainder) => (c::fw, remainder)
   

   You can call this like:

   - firstWord (explode "hello  world");
   > val it =
       ([#"h", #"e", #"l", #"l", #"o"], [#" ", #"w", #"o", #"r", #"l", #"d"])
           : char list * char list
   

   And you can call it recursively as long as the remainder is not empty:

   fun words [] = []
     | words cs =
       let val (fw, remainder) = firstWord cs
       in implode fw :: words remainder end
   

   And using this:

   - allWords (explode "hello  world");
   > val it = ["hello", "", "world"] : string list