Hey there!
Welcome to ClearUrDoubt.com.
In this post, we will look at withColumnRenamed() function in Apache Spark SQL API.
withColumnRenamed(String columnName, String newColumnName) is used to rename a column in a Dataframe.
Let’s look at the below code snippet in spark-shell for renaming a column:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 |
scala> val df = sc.range(1,10).toDF() df: org.apache.spark.sql.DataFrame = [_1: bigint] scala> df.show +---+ | _1| +---+ | 1| | 2| | 3| | 4| | 5| | 6| | 7| | 8| | 9| +---+ scala> val renamed_df = df.withColumnRenamed("_1", "range") renamed_df: org.apache.spark.sql.DataFrame = [range: bigint] scala> renamed_df.show +-----+ |range| +-----+ | 1| | 2| | 3| | 4| | 5| | 6| | 7| | 8| | 9| +-----+ scala> |
sc.range(1,10).toDF() – creates an RDD with numbers from 1 to 10 and converts the RDD into a Dataframe.
Initially, the column name is shown as _1 and it is renamed to range using withColumnRenamed(“_1”, “range”).
Happy Learning :).
Please let us know in case of any queries.