Java Fork/Join Framework File Downloading Examples

by

Overview of the Fork/Join Framework in Java

The Fork/Join framework in Java is an example of the divide and conquer strategy. It is best used for scenarios where you have multiple tasks that can be executed individually.

There are two important classes: RecursiveTask<T> and RecursiveAction.

While RecursiveTask returns a value, RecursiveAction doesn’t (Similar to Callable and Runnable).

Using Fork/Join to Download Many Files

Imagine a scenario where you need to download many files from the internet. The nature of file downloading is the tasks can be done in parallel. The order of downloaded files is not important.

Consider this example:

public class ForkJoinDownload {
    public static void main(String[] args) {
        var urlsToDownload = new ArrayList<String>();

        for (int i = 0; i < 100; i++) {
            urlsToDownload.add("http://example.com/download/" + i);
        }

        var forkJoinPool = new ForkJoinPool();

        var result = forkJoinPool.invoke(new DownloadMultipleFiles(urlsToDownload));

        System.out.println("Downloaded " + result + " files");
    }
}

class DownloadMultipleFiles extends RecursiveTask<Integer> {
    private List<String> urlsToDownload;

    public DownloadMultipleFiles(List<String> urlsToDownload) {
        this.urlsToDownload = urlsToDownload;
    }

    @Override
    protected Integer compute() {

        //Divide the task if the list size is > 1

        if (urlsToDownload.size() > 1) {

            var leftList = this.urlsToDownload.subList(0, this.urlsToDownload.size() / 2);
            var rightList = this.urlsToDownload.subList(leftList.size(), this.urlsToDownload.size());

            var leftTask = new DownloadMultipleFiles(leftList);
            var rightTask = new DownloadMultipleFiles(rightList);
            leftTask.fork();

            var rightResult = rightTask.compute();
            var leftResult = leftTask.join();
            return rightResult + leftResult;

        } else {
            //Simulate a long running task
            try {
                if (this.urlsToDownload.get(0).endsWith("60")) {
                    throw new RuntimeException("Invalid file");
                }
                Thread.sleep((int) (Math.random() * 100));
            } catch (InterruptedException | RuntimeException e) {
                //
                return 0;
            }

            System.out.println(Thread.currentThread().getName() + ": " + urlsToDownload.get(0) + " downloaded");
            return urlsToDownload.size();
        }
    }
}

In this implementation, I started with a list of 100 files. In the DownloadMultipleFiles class, there is a check if the list size is greater than 1, I will split the task. Otherwise, the code will initiate the download procedure.

Since you can find download file code easily from the internet, I did a Thread.sleep here to simulate the wait.

If there are exceptions when downloading a file, the code will return 0, which signifies that there is no file downloaded.

As expected, running the code above provides the following output:

Download files using the Fork/Join framework

As you can see, the fork join pool spawn worker threads to handle the tasks. The download tasks run in parallel, which makes the downloading much faster.

Conclusion

In this post, I’ve shown you how to use the Fork/Join framework in java to run tasks in parallel. Remember that not all tasks can be run in parallel (think about the tasks that need to be executed in order).

Leave a Comment