Spark Program to load a CSV file into a Dataset using Java 8

Hey there!

Welcome to ClearUrDoubt.com.

In this post, we will look at a Spark(2.3.0) Program to load a CSV file into a Dataset using Java 8.

Please go through the below post before going through this post.

Program to load a text file into a Dataset in Spark using Java 8

Consider a scenario where clients have provided feedback about the employees working under them. We need to find the top employee under each client based on the feedback.

Let’s create a sample program to create sample data:

Sample file:

 

Let’s look at the pom.xml and Spark programs:

 

Spark program:

 

Spark submit command:

Output:

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The output file will be saved in the file specified in the spark-submit command.

Happy Learning :).

Please leave a reply in case of any queries.

One Reply to “Spark Program to load a CSV file into a Dataset using Java 8”

Leave a Reply

Your email address will not be published. Required fields are marked *