Finding word counts in a file using Spark

Hey there! Welcome to ClearUrDoubt.com. In this post, we will look at the steps to find the word counts in a file using Spark 2.4.3. Input file:

Spark shell statements:

Output:

Happy Learning! Please leave a reply Read more

How to add a new column and update its value based on the other column in the Dataframe in Spark

Hey there! Welcome to ClearUrDoubt.com. In this post, we will look at updating a column value based on another column value in a dataframe using when() utility function in Spark.

  WHEN() function documentation: static Column when(Column condition, Object value) Evaluates a list Read more

Spark program to load a table data from Cassandra to Hive using Java

Hey there! Welcome to ClearUrDoubt.com. In this post, we will look at a Spark program to load a table data from Cassandra to Hive using Java. Step 1: Create a table in Cassandra and insert records into it.

  Step 2: Read more

Word Count program using Spark and Scala

Hey there! Welcome to ClearUrDoubt.com. In this post, we will look at the WordCount program in Spark using Scala. Here is the Scala program:

  Input(HDFS) file:

  Spark-submit command:

  Output:

  Happy Learning! 🙂 Please Read more