Apache Spark – Clear ur Doubt

Program to load a text file into a Dataset in Spark using Java 8

Hey there! Welcome to ClearUrDoubt.com. In this post, we will look at a program to load a text file into a Dataset in Spark(2.3.0) using Java 8. Check out the below classes/concepts before going through the program: static org.apache.spark.sql.functions.col org.apache.spark.sql.Dataset org.apache.spark.sql.Row Read more

Sep 23

How to read a CSV file in spark-shell using Spark SQL

Hey there! Welcome to ClearUrDoubt.com. In this post, we will go through the steps to read a CSV file in Spark SQL using spark-shell. Step 1: In Spark 1.6.0, to read a CSV file, we need to use a Read more

Aug 02

Apache Spark Overview

Apache Spark is a fast and general-purpose cluster computing system. Spark supports in-memory processing for parallel computation which is why it is pretty much faster than a Map-Reduce process. It supports a rich set of tools like Spark SQL Read more

S	M	T	W	T	F	S
		1	2	3	4	5
6	7	8	9	10	11	12
13	14	15	16	17	18	19
20	21	22	23	24	25	26
27	28	29	30	31