Search code examples
web-scrapingnested-loopsimacros

iOpus iMacro webscraping loop

I am using the Opus iMacros add-on with Internet Explorer. The following Macro takes a variable from the first line of my excel CSV File and performs a search on a website. The search results from the website may be anywhere from 10 to 200 records, which are grouped 20 per page (1-10 pages). The macro then extracts each page to a text file for future reference.

SET !ERRORIGNORE YES
SET !EXTRACT_TEST_POPUP NO
TAB T=1
TAB CLOSEALLOTHERS
SET !DATASOURCE 7Digits.csv
SET !DATASOURCE_Columns 1
SET !DATASOURCE_LINE {{!LOOP}}
'Login
URL GOTO=https://SomeWebsite.com/login
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:form1 ATTR=NAME:USER_NAME CONTENT=ABC123
TAG POS=1 TYPE=INPUT:PASSWORD FORM=NAME:form1 ATTR=NAME:PASSWORD CONTENT=XXX

'set search criteria on multiple search page
TAG POS=1 TYPE=TEXTAREA FORM=NAME:form1 ATTR=NAME: NUMBER CONTENT={{!COL1}}
TAG POS=1 TYPE=INPUT:IMAGE FORM=NAME:form1 ATTR=ID:SEARCH
‘Extract results and SAVE
TAG POS=1 TYPE=A ATTR=TXT:1
TAG POS=2 TYPE=TABLE ATTR=TXT:*location* EXTRACT=TXT
SAVEAS TYPE=txt FOLDER=* FILE=mytable_{{!NOW:yymmdd_hhnnss}}
TAG POS=1 TYPE=A ATTR=TXT:2
TAG POS=2 TYPE=TABLE ATTR=TXT:*location* EXTRACT=TXT
SAVEAS TYPE=txt FOLDER=* FILE=mytable_{{!NOW:yymmdd_hhnnss}}
TAG POS=1 TYPE=A ATTR=TXT:3
TAG POS=2 TYPE=TABLE ATTR=TXT:*location* EXTRACT=TXT
SAVEAS TYPE=txt FOLDER=* FILE=mytable_{{!NOW:yymmdd_hhnnss}}
TAG POS=1 TYPE=A ATTR=TXT:4
TAG POS=2 TYPE=TABLE ATTR=TXT:*location* EXTRACT=TXT
SAVEAS TYPE=txt FOLDER=* FILE=mytable_{{!NOW:yymmdd_hhnnss}}
TAG POS=1 TYPE=A ATTR=TXT:5
TAG POS=2 TYPE=TABLE ATTR=TXT:*location* EXTRACT=TXT
SAVEAS TYPE=txt FOLDER=* FILE=mytable_{{!NOW:yymmdd_hhnnss}}
TAG POS=1 TYPE=A ATTR=TXT:6
TAG POS=2 TYPE=TABLE ATTR=TXT:*location* EXTRACT=TXT
SAVEAS TYPE=txt FOLDER=* FILE=mytable_{{!NOW:yymmdd_hhnnss}}
TAG POS=1 TYPE=A ATTR=TXT:7
TAG POS=2 TYPE=TABLE ATTR=TXT:*location* EXTRACT=TXT
SAVEAS TYPE=txt FOLDER=* FILE=mytable_{{!NOW:yymmdd_hhnnss}}
TAG POS=1 TYPE=A ATTR=TXT:8
TAG POS=2 TYPE=TABLE ATTR=TXT:*location* EXTRACT=TXT
SAVEAS TYPE=txt FOLDER=* FILE=mytable_{{!NOW:yymmdd_hhnnss}}
TAG POS=1 TYPE=A ATTR=TXT:9
TAG POS=2 TYPE=TABLE ATTR=TXT:*location* EXTRACT=TXT
SAVEAS TYPE=txt FOLDER=* FILE=mytable_{{!NOW:yymmdd_hhnnss}}
TAG POS=1 TYPE=A ATTR=TXT:10
TAG POS=2 TYPE=TABLE ATTR=TXT:*location* EXTRACT=TXT
SAVEAS TYPE=txt FOLDER=* FILE=mytable_{{!NOW:yymmdd_hhnnss}}
'END

My challenge is to figure out a way to cancel the “SAVEAS” if there are less than 10 pages of search results. For instance, if the search only returns 60 results, the Macro only needs to perform the SAVEAS command 3 times (20 x 3). How can I move to the next {{!LOOP}} without saving the same text file 7 more times?

I have tried the “!FAIL_ON_ALL_NAVIGATEERRORS” command, but the macro stops entirely. I would like to move to the next record instead of stopping the macro.

Solution

  • This is how you convert iMacros code to JavaScript.

    How to invoke an iMacro from JavaScript?

    Also your question holds an answer because you asked " if there are less then ". So you have to convert to code to JS and use if clauses.