Java How to format URL as a String to connect with JSoup Malformed URL error

Question

I have a program that connects to a user defined URL from a TextField and scrapes the images on that web page. The user defined URL is gotten from the textfield via .getText() and assigned to a String. The String is then used to connect to the Web page with JSoup and puts the webpage into a document.

String address = labelforAddress.getText();
            try {
                document = Jsoup.connect(address).get();

            }catch(IOException ex){
                ex.printStackTrace();
            }

I've tried differently formatted URLS: "https://www.", "www.", "https://" but everything I use throws the malformed URL error. Someone please show me how to get the text from the TextField the correct way. Below is the entire code. package sample;

import javafx.application.Application;
import javafx.fxml.FXMLLoader;
import javafx.scene.Parent;
import javafx.scene.Scene;
import javafx.scene.control.Button;
import javafx.scene.control.Label;
import javafx.scene.control.TextField;
import javafx.scene.layout.GridPane;
import javafx.stage.FileChooser;
import javafx.stage.Stage;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;

import java.io.*;
import java.net.URL;
import java.util.ArrayList;
import java.util.LinkedList;
import java.util.List;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;

public class Main extends Application {

    Document document;
    LinkedList<String> imageURLList = new LinkedList<String>();
    ArrayList<File> fileList = new ArrayList<File>();
    int fileCount = 1;
    @Override
    public void start(Stage primaryStage) throws Exception{
        Parent root = FXMLLoader.load(getClass().getResource("sample.fxml"));
        primaryStage.setTitle("Webpage Photo Scraper");
        GridPane gp = new GridPane();

        Label labelforAddress = new Label("URL");
        GridPane.setConstraints(labelforAddress, 0,0);

        TextField URLAddress = new TextField();
        GridPane.setConstraints(URLAddress, 1,0);

        Button scrape = new Button("Scrape for Photos");
        GridPane.setConstraints(scrape, 0,1);
        scrape.setOnAction(event->{
            String address = labelforAddress.getText();
            try {
                document = Jsoup.connect(address).get();

            }catch(IOException ex){
                ex.printStackTrace();
            }
            Elements imgTags = document.getElementsByAttributeValueContaining("src", "/CharacterImages");
            for(Element imgTag: imgTags){
                imageURLList.add(imgTag.absUrl("src"));
            }
            for(String url: imageURLList){
                File file = new File("C:\\Users\\Andrei\\Documents\\file" + fileCount + ".txt");
                downloadFromURL(url, file);
                fileList.add(file);
                fileCount++;
            }

        });

        Button exportToZipFile = new Button("Export to Zip File");
        GridPane.setConstraints(exportToZipFile, 0,2);
        exportToZipFile.setOnAction(event -> {
            FileChooser fileChooser = new FileChooser();
            FileChooser.ExtensionFilter exfilt = new FileChooser.ExtensionFilter("Zip Files", ".zip");
            fileChooser.getExtensionFilters().add(exfilt);
            try{

                FileOutputStream fos = new FileOutputStream(fileChooser.showSaveDialog(primaryStage));
                ZipOutputStream zipOut = new ZipOutputStream(fos);
                for(int count = 0; count<=fileList.size()-1; count++){
                    File fileToZip = new File(String.valueOf(fileList.get(count)));
                    FileInputStream fis = new FileInputStream(fileToZip);
                    ZipEntry zipEntry = new ZipEntry(fileToZip.getName());
                    zipOut.putNextEntry(zipEntry);

                    byte[] bytes = new byte[1024];
                    int length;
                    while((length = fis.read(bytes)) >= 0) {
                        zipOut.write(bytes, 0, length);
                    }
                    fis.close();
                }
                zipOut.close();
                fos.close();
            }catch(IOException e1){
                e1.printStackTrace();
            }

        });
        primaryStage.setScene(new Scene(gp, 300, 275));
        primaryStage.show();
        gp.getChildren().addAll(exportToZipFile, labelforAddress, scrape, URLAddress);
    }
    public static void downloadFromURL(String url, File file){
        try {
            URL Url = new URL(url);
            BufferedInputStream bis = new BufferedInputStream(Url.openStream());
            FileOutputStream fis = new FileOutputStream(file);
            byte[] buffer = new byte[1024];
            int count = 0;
            while((count = bis.read(buffer, 0,1024)) !=-1){
                fis.write(buffer, 0, count);
            }
            fis.close();
            bis.close();
        }catch(IOException e){
            e.printStackTrace();
        }
    }

    public static void main(String[] args) {
        launch(args);
    }
}

Answer 1

Your text field containing the value entered by user is stored in URLAddress object but you always try to get the url from labelforAddress object which is a label always containing "URL" text.
So the solution is to use:

  String address = URLAddress.getText();

If you read carefully error message it would help you to find the cause, because it always displays the value it considers wrong. In this case I see:

  Caused by: java.net.MalformedURLException: no protocol: URL

and it shows the unrecognized address is: URL.

If you encounter this kind of error next time try:

• debugging the aplication in runtime to see values of each variable
• logging variable values in the console to see if variables contain values you expect