I have created a java application which basically scraps data from a website and create a .csv file. It is running fine when I execute it in NetBeans and creates file of 34kb. But, when I build it and run the .JAR executable file it creates the same file but size of the file is 4kb and not bringing all the data. I build the project on java 1.8. I don't understand this issue actually what the reason behind it. Is it the problem with the java version? Thanks in advance for your help. sorry for not adding the code here earlier. here's the code of my project.
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Date;
import java.util.Iterator;
import java.util.Locale;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
//this program is for open-end-summary
/**
*
* @author talha
*/
public class ComMufapScraping {
boolean writeCSVToConsole = true;
boolean writeCSVToFile = true;
boolean sortTheList = true;
boolean writeToConsole;
boolean writeToFile;
public static Document doc = null;
public static Elements tbodyElements = null;
public static Elements elements = null;
public static Elements tdElements = null;
public static Elements trElement2 = null;
public static String Dcomma = ",";
public static String line = "";
public static ArrayList<Elements> sampleList = new ArrayList<Elements>();
public static void createConnection() throws IOException {
System.setProperty("http.proxyHost", "191.1.1.202");
System.setProperty("http.proxyPort", "8080");
String tempUrl = "http://www.mufap.com.pk/nav_returns_performance.php?tab=01";
doc = Jsoup.connect(tempUrl).get();
}
public static void parsingHTML() throws Exception {
for (Element table : doc.getElementsByTag("table")) {
for (Element trElement : table.getElementsByTag("tr")) {
trElement2 = trElement.getElementsByTag("tr");
tdElements = trElement.getElementsByTag("td");
File fold = new File("C:\\open-end-smry.csv");
fold.delete();
File fnew = new File("C:\\open-end-smry.csv");
FileWriter sb = new FileWriter(fnew, true);
if (trElement.hasClass("tab-data1")) {
for (Iterator<Element> it = tdElements.iterator(); it.hasNext();) {
if (it.hasNext()) {
sb.append("\r\n");
}
for (Iterator<Element> it2 = trElement2.iterator(); it.hasNext();) {
Element tdElement2 = it.next();
final String content = tdElement2.text();
if (it2.hasNext()) {
sb.append(formatData(content));
sb.append(" , ");
}
if (!it.hasNext()) {
String content1 = content.replaceAll(",$", " ");
sb.append(formatData(content1));
//it2.next();
}
}
System.out.println(sb.toString());
sb.flush();
sb.close();
}
}
System.out.println(sampleList.add(tdElements));
}
}
}
private static final SimpleDateFormat FORMATTER_MMM_d_yyyy = new SimpleDateFormat("MMM d, yyyy", Locale.US);
private static final SimpleDateFormat FORMATTER_dd_MMM_yyyy = new SimpleDateFormat("dd-MMM-YYYY", Locale.US);
public static String formatData(String text) {
String tmp = null;
try {
Date d = FORMATTER_MMM_d_yyyy.parse(text);
tmp = FORMATTER_dd_MMM_yyyy.format(d);
} catch (ParseException pe) {
tmp = text;
}
return tmp;
}
public static void main(String[] args) throws IOException, Exception {
createConnection();
parsingHTML();
}
}
here is the log cat file after building the project.
ant -f C:\Users\talha\Documents\NetBeansProjects\com.open_end_smry -Dnb.internal.action.name=rebuild clean jar init: deps-clean: Updating property file: C:\Users\talha\Documents\NetBeansProjects\com.open_end_smry\build\built-clean.properties Deleting directory C:\Users\talha\Documents\NetBeansProjects\com.open_end_smry\build clean: init: deps-jar: Created dir: C:\Users\talha\Documents\NetBeansProjects\com.open_end_smry\build Updating property file: C:\Users\talha\Documents\NetBeansProjects\com.open_end_smry\build\built-jar.properties Created dir: C:\Users\talha\Documents\NetBeansProjects\com.open_end_smry\build\classes Created dir: C:\Users\talha\Documents\NetBeansProjects\com.open_end_smry\build\empty Created dir: C:\Users\talha\Documents\NetBeansProjects\com.open_end_smry\build\generated-sources\ap-source-output Compiling 1 source file to C:\Users\talha\Documents\NetBeansProjects\com.open_end_smry\build\classes warning: [options] bootstrap class path not set in conjunction with -source 1.7 1 warning compile: Created dir: C:\Users\talha\Documents\NetBeansProjects\com.open_end_smry\dist Copying 1 file to C:\Users\talha\Documents\NetBeansProjects\com.open_end_smry\build Copy libraries to C:\Users\talha\Documents\NetBeansProjects\com.open_end_smry\dist\lib. Building jar: C:\Users\talha\Documents\NetBeansProjects\com.open_end_smry\dist\com.open_end_smry.jar To run this application from the command line without Ant, try: java -jar "C:\Users\talha\Documents\NetBeansProjects\com.open_end_smry\dist\com.open_end_smry.jar" jar: BUILD SUCCESSFUL (total time: 0 seconds)
You have the delete function of the file inside your loop, so the file will deleted in every Loop. Put These part outside of your Loop:
public static void parsingHTML() throws Exception {
File fold = new File("C:\\open-end-smry.csv");
fold.delete();
for (Element table : doc.getElementsByTag("table")) {
for (Element trElement : table.getElementsByTag("tr")) {
trElement2 = trElement.getElementsByTag("tr");
tdElements = trElement.getElementsByTag("td");
File fnew = new File("C:\\open-end-smry.csv");
FileWriter sb = new FileWriter(fnew, true);
if (trElement.hasClass("tab-data1")) {
for (Iterator<Element> it = tdElements.iterator(); it.hasNext();) {
if (it.hasNext()) {
sb.append("\r\n");
}
for (Iterator<Element> it2 = trElement2.iterator(); it.hasNext();) {
Element tdElement2 = it.next();
final String content = tdElement2.text();
if (it2.hasNext()) {
sb.append(formatData(content));
sb.append(" , ");
}
if (!it.hasNext()) {
String content1 = content.replaceAll(",$", " ");
sb.append(formatData(content1));
//it2.next();
}
}
System.out.println(sb.toString());
sb.flush();
sb.close();
}
}
System.out.println(sampleList.add(tdElements));
}
}
}