Search code examples
bashstanford-nlp

"f was unexpected at this time" - bash script


I am currently trying to use the Stanford CoreNLP pipeline to run sentiment analysis. I need it to iterate through a folder of individual text files (which contain movie reviews) in order to establish the sentiment of each review. I have tried to create a batch script in order to iterate through the 3 different folders containing the reviews. when running the script through the shell runner program i receive the following error:

f was unexpected at this time

the script is as follows:

dir=C:\stanford-corenlp-full-2016-10-31
for f in "$dir\train\neg" && 
    for f in "$dir\train\pos" && 
        for f in "$dir\train\unsup" ; do
    echo $f >> filelist.txt
    java -mx8g edu.stanford.nlp.sentiment.SentimentTraining -numHid 25 -trainPath filelist.txt -devPath dev.txt -train -model model.ser.gz 
done

this is the first bash script i have ever written so I am assuming my syntax is possibly incorrect somewhere?

Also I am using Windows 10.

Any advice would be amazing,

many thanks

hey guys your advice has been extremely useful. to try and make my life a bit easier I have tried to convert my script to a batch script so that it shouldnt have any issues with being run on windows. my new script looks as follows:

        @echo off
dir=C:\stanford-corenlp-full-2016-10-31
for %%f in "%dir\train\neg" & "%dir\train\pos" & "%dir\train\unsup" do
    ECHO %%f >> filelist.txt
    java -mx8g edu.stanford.nlp.sentiment.SentimentTraining -numHid 25 -trainPath filelist.txt -devPath dev.txt -train -model model.ser.gz 
done
pause

which results in the following error: "%dir\train\pos" was unexpected at this time

anyone understand what i've done wrong? I'm assuming it's some sort of a syntax issue I just can't see it.


Solution

  • You don't need multiple for's you can put all the items after the first like this:

     for f in "$dir\train\neg" "$dir\train\pos" "$dir\train\unsup" ; do
    

    Or even:

    for f in "$dir\train\"{neg,pos,unsup}
    do
        ....
    done
    

    Also, I would think you would need to replace all single slashes "\" with two "\\". but can't say for sure as I don't have Windows.

    So it might be

     dir=C:\\stanford-corenlp-full-2016-10-31
    
     for f in "$dir\\train\\"{neg,pos,unsup}
     do
         ...
     done
    

    Its unclear whether:

    • you need to call Java 3 times ( once for each dir)

    Or:

    • you need to put all 3 dirs in filelist.txt and then call Java at the end

    Example to call Java 3 times with only one dir in filelist.txt each time:

     for f in "$dir\\train\\"{neg,pos,unsup}
     do 
         #note this ">" will over-write the file each time
         echo "$f" > filelist.txt  
         java    .... filelist.txt
     done
    

    Creating a list file with 3 directories then call Java once:

     # note this ">" will overwrite the file with nothing
     # so it will then be empty
     # so that it doesn't have what's left over from last time
     # that you ran the script
     > filelist.txt  
    
     for f in "$dir\\train\\"{neg,pos,unsup}
     do
          #note this ">>" will add a line to the file
          echo "$f" >> filelist.txt  
     done
    
     # call Java after you've finished creating the file
     java ..... filelist.txt
    

    that you need

    But, you don't even need the loop (assuming you have the 'ls' command available to you in that environment)

    dir=C:\\stanford-corenlp-full-2016-10-31
    
    ls -1 -d "$dir\\train\\"{neg,pos,unsup} > filelist.txt
    
    java -mx8g edu.stanford.nlp.sentiment.SentimentTraining -numHid 25 -trainPath filelist.txt -devPath dev.txt -train -model model.ser.gz 
    

    NOTE The {neg,pos,unsup} syntax will work in bash but not in a windows bat file