Search code examples
azure-storagepolybasesql-server-2022

The option 'hadoop connectivity' cannot be enabled in this edition of SQL Server

I'm trying to run polybase in SQL Server 2022.

When running EXEC sp_configure @configname = 'hadoop connectivity', @configvalue = 7;

I get the error

Msg 46934, Level 16, State 1, Procedure sp_configure, Line 177 [Batch Start Line 4] The option 'hadoop connectivity' cannot be enabled in this edition of SQL Server.

I'm running this command, as I am trying to enable Azure storage for Polybase. This worked in 2019.

I don't understand why I'm getting this error.

Both Polybase services are running and I can query external SQL Server tables.

Solution

  • Hadoop is no longer support in Polybase in SQL Server 2022 onwards, this is documented in several locations, such as PolyBase Connectivity Configuration (Transact-SQL) - General Remarks:

    Starting in SQL Server 2022 (16.x), Hadoop is no longer supported in PolyBase.

    And more verbosely Introducing data virtualization with PolyBase - PolyBase Connectors:

    PolyBase connectors

    The PolyBase feature provides connectivity to the following external data sources:

    External data sources SQL Server 2016-2019 with PolyBase SQL Server 2022 (16.x) with PolyBase APS PDW Azure Synapse Analytics
    Oracle, MongoDB, Teradata Read Read No No
    Generic ODBC Read (Windows Only) Read (Windows Only) No No
    Azure Storage Read/Write Read/Write Read/Write Read/Write
    Hadoop Read/Write No Read/Write No
    SQL Server Read Read No No
    S3-compatible object storage No Read/Write No No
    • SQL Server 2022 (16.x) does not support Hadoop.
    • SQL Server 2016 (13.x) introduced PolyBase with support for connections to Hadoop and Azure Blob Storage.
    • SQL Server 2019 (15.x) introduced additional connectors, including SQL Server, Oracle, Teradata, and MongoDB.
    • SQL Server 2022 (16.x) introduced the S3-compatible storage connector.

    (Both Emphasis mine)

    As such, if you are migrating from SQL Server 2019 to SQL Server 2022 and need similar functionality you will need to use a different solution. If this is deal breaking (at least for now) then stay on 2019, as Microsoft "will continue to support the functionality to the end of life of those products" for Hadoop.

    Unfortunately, I will admit, I don't have familiarity with Hadoop, or PolyBase, however, Microsoft also offer extensive documentation on alternatives in Big data options on the Microsoft SQL Server platform - Functionality replacement options for Big Data and SQL Server.