Hey there! Welcome to ClearUrDoubt.com. In this post, we will look at a program to load a text file into a Dataset in Spark(2.3.0) using Java 8. Check out the below classes/concepts before going through the program: static org.apache.spark.sql.functions.col org.apache.spark.sql.Dataset org.apache.spark.sql.Row Read more
Hey there! Welcome to ClearUrDoubt.com. In this post, we will look at the WordCount program in Spark using Java 8. Before going through the below program, it’s better to look into below concepts for a better understanding of the program: org.apache.spark.api.java.JavaRDD org.apache.spark.api.java.JavaPairRDD Read more
Hey there! Welcome to ClearUrDoubt.com. In this post, we will go through the steps to read a CSV file in Spark SQL using spark-shell. Step 1: In Spark 1.6.0, to read a CSV file, we need to use a Read more