pyspark.sql.functions.exists

pyspark.sql.functions.exists(col, f)[source]

Returns whether a predicate holds for one or more elements in the array.

Parameters

  • col – name of column or expression

  • f – an function (x: Column) -> Column: ... returning the Boolean expression. Can use methods of pyspark.sql.Column, functions defined in pyspark.sql.functions and Scala UserDefinedFunctions. Python UserDefinedFunctions are not supported (SPARK-27052).

Returns

a pyspark.sql.Column

>>> df = spark.createDataFrame([(1, [1, 2, 3, 4]), (2, [3, -1, 0])],("key", "values"))
>>> df.select(exists("values", lambda x: x < 0).alias("any_negative")).show()
+------------+
|any_negative|
+------------+
|       false|
|        true|
+------------+

New in version 3.1.

pyspark.sql.functions.encode pyspark.sql.functions.exp