pyspark.sql.functions.collect_list

pyspark.sql.functions.collect_list(col)

Aggregate function: returns a list of objects with duplicates.

Note

The function is non-deterministic because the order of collected results depends on the order of the rows which may be non-deterministic after a shuffle.

>>> df2 = spark.createDataFrame([(2,), (5,), (5,)], ('age',))
>>> df2.agg(collect_list('age')).collect()
[Row(collect_list(age)=[2, 5, 5])]

New in version 1.6.

pyspark.sql.functions.col pyspark.sql.functions.collect_set