Azure Databricks JpaRepository - [Databricks][JDBC](10220) Driver does not support this optional feature

Question

In my java 17 spring boot application I am trying to connect to an azure databrick and use a JpaRepository.

I get the following error message when using the repository:

CannotCreateTransactionException: Could not open JPA EntityManager for transaction
...
org.hibernate.TransactionException: JDBC begin transaction failed
...
java.sql.SQLFeatureNotSupportedException: [Databricks][JDBC](10220) Driver does not support this optional feature.

I will answer this question myself to share knowledge because I was not ablet to easily find a solution.

Answer 1

This has to do with the databricks driver not supporting autocommit. This can be fixed by modifying the @EnableJpaRepositories annotaton in your confuguration.

@EnableJpaRepositories(
        ...
        enableDefaultTransactions = false
)

as suggested here https://stackoverflow.com/a/72170546/6128747

My whole setup with java 17 and databricks is the following:

build.gradle

implementation 'org.springframework.boot:spring-boot-starter-data-jpa'
implementation 'com.databricks:databricks-jdbc:2.6.32'

applicaton.properties

spring.datasource.databricks.driver=com.databricks.client.jdbc.Driver
spring.datasource.databricks.jdbc-url=jdbc:databricks:yourDatabricksUrl=http;ssl=1;EnableArrow=0;httpPath=yourHttpPath;AuthMech=3;UID=token;PWD=secretToken;

DataBricksDataSourceConfiguration.java

Note that databricks is configures as a secondary datasource that is why I have to provide the package names.

package com.vgp.controlcenter.configuration.datasource;

import java.sql.SQLException;
import java.util.Objects;

import javax.sql.DataSource;

import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.jdbc.DataSourceBuilder;
import org.springframework.boot.orm.jpa.EntityManagerFactoryBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.env.Environment;
import org.springframework.data.annotation.ReadOnlyProperty;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
import org.springframework.orm.jpa.JpaTransactionManager;
import org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean;
import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.annotation.EnableTransactionManagement;

@Configuration
@EnableTransactionManagement
@EnableJpaRepositories(
        basePackages = {my.project.repository.databricks"},
        entityManagerFactoryRef = "databricksEntityManagerFactory",
        transactionManagerRef = "databricksTransactionManager",
        enableDefaultTransactions = false
)
public class DataBricksDataSourceConfiguration {

    @Bean("databricks-datasource")
    @ReadOnlyProperty
    @ConfigurationProperties("spring.datasource.databricks")
    public DataSource databricksDatasource()  {
        return DataSourceBuilder.create().build();
    }

    @Bean
    public LocalContainerEntityManagerFactoryBean databricksEntityManagerFactory(@Qualifier("databricks-datasource") DataSource dataSource, EntityManagerFactoryBuilder builder) {
        return builder
                .dataSource(dataSource)
                .packages("my.project.model.db.databricks")
                .persistenceUnit("databricks")
                .build();
    }

    @Bean
    public PlatformTransactionManager databricksTransactionManager(
            @Qualifier("databricksEntityManagerFactory") LocalContainerEntityManagerFactoryBean databricksEntityManagerFactory) {
        return new JpaTransactionManager(Objects.requireNonNull(databricksEntityManagerFactory.getObject()));
    }
}

The repository and Entities are just normal as you would expect. But maybe as a note I had problems with double quotes because we used spring.jpa.properties.hibernate.globally_quoted_identifiers=true so I got [PARSE_SYNTAX_ERROR] Syntax error at or near '"field"'. We just disabled / removed globally_quoted_identifiers, the only other solution I had was using a native queries in the repository.

Also

do NOT try to use dataSource.getConnection().setAutoCommit(false); to fix this error this will actually lead to the same error message.

Note

Also I'm only using databricks to read data. So I don't know what effects disabling auto commit has on writing data.