pyspark.sql.SparkSession.builder.config

builder.config(key=None, value=None, conf=None)

Sets a config option. Options set using this method are automatically propagated to both SparkConf and SparkSession’s own configuration.

For an existing SparkConf, use conf parameter.

>>> from pyspark.conf import SparkConf
>>> SparkSession.builder.config(conf=SparkConf())
<pyspark.sql.session...

For a (key, value) pair, you can omit parameter names.

>>> SparkSession.builder.config("spark.some.config.option", "some-value")
<pyspark.sql.session...
Parameters

  • key – a key name string for configuration property

  • value – a value for configuration property

  • conf – an instance of SparkConf

New in version 2.0.

pyspark.sql.SparkSession.builder.appName pyspark.sql.SparkSession.builder.enableHiveSupport