pyspark.sql.streaming.DataStreamWriter.outputMode

DataStreamWriter.outputMode(outputMode)

Specifies how data of a streaming DataFrame/Dataset is written to a streaming sink.

Options include:

• append: Only the new rows in the streaming DataFrame/Dataset will be written to

  the sink

• complete: All the rows in the streaming DataFrame/Dataset will be written to the sink

  every time these is some updates

• update: only the rows that were updated in the streaming DataFrame/Dataset will be

  written to the sink every time there are some updates. If the query doesn’t contain aggregations, it will be equivalent to append mode.

Note

Evolving.

>>> writer = sdf.writeStream.outputMode('append')

New in version 2.0.

pyspark.sql.streaming.DataStreamWriter.options pyspark.sql.streaming.DataStreamWriter.partitionBy