Trying to get first word from character list

Question

I have a character list [#"h", #"i", #" ", #"h", #"i"] which I want to get the first word from this (the first character sequence before each space).

I've written a function which gives me this warning:

stdIn:13.1-13.42 Warning: type vars not generalized because of value restriction are instantiated to dummy types (X1,X2,...)

Here is my code:

fun next [] = ([], [])
   | next (hd::tl) = if(not(ord(hd) >= 97 andalso ord(hd) <= 122)) then ([], (hd::tl))
       else
         let
           fun getword [] = [] | getword (hd::tl) = if(ord(hd) >= 97 andalso ord(hd) <= 122) then [hd]@getword tl else [];
         in
           next (getword (hd::tl))
         end;

EDIT:

Expected input and output

next [#"h", #"i", #" ", #"h", #"i"] => ([#"h", #"i"], [#" ", #"h", #"i"]) 

Can anybody help me with this solution? Thanks!

Answer 1

This functionality already exists within the standard library:

val nexts = String.tokens Char.isSpace
val nexts_test = nexts "hi hi   hi" = ["hi", "hi", "hi"]

But if you were to build such a function anyway, it seems that you return ([], []) sometimes and a single list at other times. Normally in a recursive function, you can build the result by doing e.g. c :: recursive_f cs, but this is assuming your function returns a single list. If, instead, it returns a tuple, you suddenly have to unpack this tuple using e.g. pattern matching in a let-expression:

let val (x, y) = recursive_f cs
in (c :: x, y + ...) end

Or you could use an extra argument inside a helper function (since the extra argument would change the type of the function) to store the word you're extracting, instead. A consequence of doing that is that you end up with the word in reverse and have to reverse it back when you're done recursing.

fun isLegal c = ord c >= 97 andalso ord c <= 122  (* Only lowercase ASCII letters *)
(* But why not use one of the following:
   fun isLegal c = Char.isAlpha c
   fun isLegal c = not (Char.isSpace c)  *)

fun next input =
    let fun extract (c::cs) word =
              if isLegal c
              then extract cs (c::word)
              else (rev word, c::cs)
          | extract [] word = (rev word, [])
    in extract input [] end

val next_test_1 =
    let val (w, r) = next (explode "hello world")
    in (implode w, implode r) = ("hello", " world")
    end

val next_test_2 = next [] = ([], [])