Spark SQL and DataFrames: Spark 2.2 Documentation

Guide for migration. It’s time for the latest and greatest Spark!

 

Highlight:

  • SparkSession is unified.
  • DataFrame and DataSet API’s are unified too. A DataFrame is an alias of DataSet[Row].
  • DDL SQL statements have changes in behavior.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s