pyspark.sql.functions.
array_remove
Collection function: Remove all elements that equal to element from the given array.
col – name of column containing array
element – element to be removed from the array
>>> df = spark.createDataFrame([([1, 2, 3, 1, 1],), ([],)], ['data']) >>> df.select(array_remove(df.data, 1)).collect() [Row(array_remove(data, 1)=[2, 3]), Row(array_remove(data, 1)=[])]
New in version 2.4.