I've come here from questions like: java: cannot access scala.Serializable class file for scala.Serializable not found
And other similar questions.
Recently, I've taken the initiative to move my from java 11 to java 17, including this EMR spark app I have. I'm not very familiar with scala but the move seemed simple enough, moving everything from scala 2.12 to 2.13.
However, I've now run into this error:
SparkJobTest.java:[28,13] cannot find symbol
[ERROR] symbol: class Serializable
[ERROR] location: package scala
Early on, my inclination was I forgot a dependency somewhere, but looking at my tree, nothing is obvious:
[INFO] --- maven-dependency-plugin:3.5.0:tree (default-cli) @ spark-ingester-spring-boot-starter ---
[INFO] com.mycompany.mcp:spark-ingester-spring-boot-starter:jar:7.0.0-SNAPSHOT
[INFO] +- org.springframework.boot:spring-boot-autoconfigure:jar:2.5.2:compile
[INFO] | \- org.springframework.boot:spring-boot:jar:2.5.2:compile
[INFO] | +- org.springframework:spring-core:jar:5.3.8:compile
[INFO] | | \- org.springframework:spring-jcl:jar:5.3.8:compile
[INFO] | \- org.springframework:spring-context:jar:5.3.8:compile
[INFO] | +- org.springframework:spring-aop:jar:5.3.8:compile
[INFO] | \- org.springframework:spring-expression:jar:5.3.8:compile
[INFO] +- org.apache.spark:spark-core_2.13:jar:3.4.1:provided
[INFO] | +- org.scala-lang.modules:scala-parallel-collections_2.13:jar:1.0.4:provided
[INFO] | +- org.apache.avro:avro:jar:1.11.1:provided
[INFO] | +- org.apache.avro:avro-mapred:jar:1.11.1:provided
[INFO] | | \- org.apache.avro:avro-ipc:jar:1.11.1:provided
[INFO] | | \- org.tukaani:xz:jar:1.9:provided
[INFO] | +- com.twitter:chill_2.13:jar:0.10.0:provided
[INFO] | | \- com.esotericsoftware:kryo-shaded:jar:4.0.2:provided
[INFO] | | \- com.esotericsoftware:minlog:jar:1.3.0:provided
[INFO] | +- com.twitter:chill-java:jar:0.10.0:provided
[INFO] | +- org.apache.xbean:xbean-asm9-shaded:jar:4.22:provided
[INFO] | +- org.apache.hadoop:hadoop-client-api:jar:3.3.4:provided
[INFO] | +- org.apache.hadoop:hadoop-client-runtime:jar:3.3.4:provided
[INFO] | +- org.apache.spark:spark-launcher_2.13:jar:3.4.1:provided
[INFO] | +- org.apache.spark:spark-kvstore_2.13:jar:3.4.1:provided
[INFO] | | +- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:provided
[INFO] | | \- com.fasterxml.jackson.core:jackson-annotations:jar:2.14.2:compile
[INFO] | +- org.apache.spark:spark-network-common_2.13:jar:3.4.1:provided
[INFO] | | \- com.google.crypto.tink:tink:jar:1.7.0:provided
[INFO] | +- org.apache.spark:spark-network-shuffle_2.13:jar:3.4.1:provided
[INFO] | +- org.apache.spark:spark-unsafe_2.13:jar:3.4.1:provided
[INFO] | +- javax.activation:activation:jar:1.1.1:provided
[INFO] | +- org.apache.curator:curator-recipes:jar:2.13.0:provided
[INFO] | | \- org.apache.curator:curator-framework:jar:2.13.0:provided
[INFO] | +- org.apache.zookeeper:zookeeper:jar:3.6.3:provided
[INFO] | | +- org.apache.zookeeper:zookeeper-jute:jar:3.6.3:provided
[INFO] | | \- org.apache.yetus:audience-annotations:jar:0.5.0:provided
[INFO] | +- jakarta.servlet:jakarta.servlet-api:jar:4.0.3:provided
[INFO] | +- commons-codec:commons-codec:jar:1.15:compile
[INFO] | +- org.apache.commons:commons-compress:jar:1.22:compile
[INFO] | +- org.apache.commons:commons-lang3:jar:3.12.0:compile
[INFO] | +- org.apache.commons:commons-math3:jar:3.6.1:compile
[INFO] | +- org.apache.commons:commons-text:jar:1.10.0:compile
[INFO] | +- commons-io:commons-io:jar:2.11.0:compile
[INFO] | +- commons-collections:commons-collections:jar:3.2.2:provided
[INFO] | +- org.apache.commons:commons-collections4:jar:4.4:compile
[INFO] | +- com.google.code.findbugs:jsr305:jar:3.0.0:compile
[INFO] | +- org.slf4j:slf4j-api:jar:2.0.6:compile
[INFO] | +- org.slf4j:jul-to-slf4j:jar:2.0.6:compile
[INFO] | +- org.slf4j:jcl-over-slf4j:jar:2.0.6:compile
[INFO] | +- org.apache.logging.log4j:log4j-slf4j2-impl:jar:2.19.0:provided
[INFO] | +- org.apache.logging.log4j:log4j-api:jar:2.19.0:compile
[INFO] | +- org.apache.logging.log4j:log4j-core:jar:2.19.0:compile
[INFO] | +- org.apache.logging.log4j:log4j-1.2-api:jar:2.19.0:provided
[INFO] | +- com.ning:compress-lzf:jar:1.1.2:provided
[INFO] | +- org.xerial.snappy:snappy-java:jar:1.1.10.1:provided
[INFO] | +- org.lz4:lz4-java:jar:1.8.0:provided
[INFO] | +- com.github.luben:zstd-jni:jar:1.5.2-5:provided
[INFO] | +- org.roaringbitmap:RoaringBitmap:jar:0.9.38:provided
[INFO] | | \- org.roaringbitmap:shims:jar:0.9.38:provided
[INFO] | +- org.scala-lang.modules:scala-xml_2.13:jar:2.1.0:provided
[INFO] | +- org.scala-lang:scala-reflect:jar:2.13.8:provided
[INFO] | +- org.json4s:json4s-jackson_2.13:jar:3.7.0-M11:provided
[INFO] | | \- org.json4s:json4s-core_2.13:jar:3.7.0-M11:provided
[INFO] | | +- org.json4s:json4s-ast_2.13:jar:3.7.0-M11:provided
[INFO] | | \- org.json4s:json4s-scalap_2.13:jar:3.7.0-M11:provided
[INFO] | +- org.glassfish.jersey.core:jersey-client:jar:2.36:provided
[INFO] | | +- jakarta.ws.rs:jakarta.ws.rs-api:jar:2.1.6:provided
[INFO] | | \- org.glassfish.hk2.external:jakarta.inject:jar:2.6.1:provided
[INFO] | +- org.glassfish.jersey.core:jersey-common:jar:2.36:provided
[INFO] | | +- jakarta.annotation:jakarta.annotation-api:jar:1.3.5:compile
[INFO] | | \- org.glassfish.hk2:osgi-resource-locator:jar:1.0.3:provided
[INFO] | +- org.glassfish.jersey.core:jersey-server:jar:2.36:provided
[INFO] | | \- jakarta.validation:jakarta.validation-api:jar:2.0.2:provided
[INFO] | +- org.glassfish.jersey.containers:jersey-container-servlet:jar:2.36:provided
[INFO] | +- org.glassfish.jersey.containers:jersey-container-servlet-core:jar:2.36:provided
[INFO] | +- org.glassfish.jersey.inject:jersey-hk2:jar:2.36:provided
[INFO] | | +- org.glassfish.hk2:hk2-locator:jar:2.6.1:provided
[INFO] | | | +- org.glassfish.hk2.external:aopalliance-repackaged:jar:2.6.1:provided
[INFO] | | | +- org.glassfish.hk2:hk2-api:jar:2.6.1:provided
[INFO] | | | \- org.glassfish.hk2:hk2-utils:jar:2.6.1:provided
[INFO] | | \- org.javassist:javassist:jar:3.25.0-GA:provided
[INFO] | +- io.netty:netty-all:jar:4.1.87.Final:provided
[INFO] | | +- io.netty:netty-buffer:jar:4.1.87.Final:runtime
[INFO] | | +- io.netty:netty-codec:jar:4.1.87.Final:runtime
[INFO] | | +- io.netty:netty-codec-http:jar:4.1.87.Final:runtime
[INFO] | | +- io.netty:netty-codec-http2:jar:4.1.87.Final:runtime
[INFO] | | +- io.netty:netty-codec-socks:jar:4.1.87.Final:provided
[INFO] | | +- io.netty:netty-common:jar:4.1.87.Final:runtime
[INFO] | | +- io.netty:netty-handler:jar:4.1.87.Final:runtime
[INFO] | | +- io.netty:netty-transport-native-unix-common:jar:4.1.87.Final:provided
[INFO] | | +- io.netty:netty-handler-proxy:jar:4.1.87.Final:provided
[INFO] | | +- io.netty:netty-resolver:jar:4.1.87.Final:runtime
[INFO] | | +- io.netty:netty-transport:jar:4.1.87.Final:runtime
[INFO] | | +- io.netty:netty-transport-classes-epoll:jar:4.1.87.Final:runtime
[INFO] | | \- io.netty:netty-transport-classes-kqueue:jar:4.1.87.Final:provided
[INFO] | +- io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.87.Final:provided
[INFO] | +- io.netty:netty-transport-native-epoll:jar:linux-aarch_64:4.1.87.Final:provided
[INFO] | +- io.netty:netty-transport-native-kqueue:jar:osx-aarch_64:4.1.87.Final:provided
[INFO] | +- io.netty:netty-transport-native-kqueue:jar:osx-x86_64:4.1.87.Final:provided
[INFO] | +- com.clearspring.analytics:stream:jar:2.9.6:provided
[INFO] | +- io.dropwizard.metrics:metrics-core:jar:4.2.15:provided
[INFO] | +- io.dropwizard.metrics:metrics-jvm:jar:4.2.15:provided
[INFO] | +- io.dropwizard.metrics:metrics-json:jar:4.2.15:provided
[INFO] | +- io.dropwizard.metrics:metrics-graphite:jar:4.2.15:provided
[INFO] | +- io.dropwizard.metrics:metrics-jmx:jar:4.2.15:provided
[INFO] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.14.2:compile
[INFO] | +- com.fasterxml.jackson.module:jackson-module-scala_2.13:jar:2.14.2:provided
[INFO] | | \- com.thoughtworks.paranamer:paranamer:jar:2.8:provided
[INFO] | +- org.apache.ivy:ivy:jar:2.5.1:provided
[INFO] | +- oro:oro:jar:2.0.8:provided
[INFO] | +- net.razorvine:pickle:jar:1.3:provided
[INFO] | +- net.sf.py4j:py4j:jar:0.10.9.7:provided
[INFO] | +- org.apache.spark:spark-tags_2.13:jar:3.4.1:provided
[INFO] | \- org.apache.commons:commons-crypto:jar:1.1.0:provided
[INFO] +- org.apache.spark:spark-sql_2.13:jar:3.4.1:provided
[INFO] | +- org.rocksdb:rocksdbjni:jar:7.9.2:provided
[INFO] | +- com.univocity:univocity-parsers:jar:2.9.1:provided
[INFO] | +- org.apache.spark:spark-sketch_2.13:jar:3.4.1:provided
[INFO] | +- org.apache.spark:spark-catalyst_2.13:jar:3.4.1:provided
[INFO] | | +- org.scala-lang.modules:scala-parser-combinators_2.13:jar:2.1.1:provided
[INFO] | | +- org.codehaus.janino:janino:jar:3.1.9:provided
[INFO] | | +- org.codehaus.janino:commons-compiler:jar:3.1.9:provided
[INFO] | | +- org.antlr:antlr4-runtime:jar:4.9.3:provided
[INFO] | | +- org.apache.arrow:arrow-vector:jar:11.0.0:provided
[INFO] | | | +- org.apache.arrow:arrow-format:jar:11.0.0:provided
[INFO] | | | +- org.apache.arrow:arrow-memory-core:jar:11.0.0:provided
[INFO] | | | \- com.google.flatbuffers:flatbuffers-java:jar:1.12.0:provided
[INFO] | | \- org.apache.arrow:arrow-memory-netty:jar:11.0.0:provided
[INFO] | +- org.apache.orc:orc-core:jar:shaded-protobuf:1.8.4:provided
[INFO] | | +- org.apache.orc:orc-shims:jar:1.8.4:provided
[INFO] | | +- io.airlift:aircompressor:jar:0.21:provided
[INFO] | | +- org.jetbrains:annotations:jar:17.0.0:provided
[INFO] | | \- org.threeten:threeten-extra:jar:1.7.1:provided
[INFO] | +- org.apache.orc:orc-mapreduce:jar:shaded-protobuf:1.8.4:provided
[INFO] | +- org.apache.hive:hive-storage-api:jar:2.8.1:provided
[INFO] | +- org.apache.parquet:parquet-column:jar:1.12.3:provided
[INFO] | | +- org.apache.parquet:parquet-common:jar:1.12.3:provided
[INFO] | | \- org.apache.parquet:parquet-encoding:jar:1.12.3:provided
[INFO] | +- org.apache.parquet:parquet-hadoop:jar:1.12.3:provided
[INFO] | | +- org.apache.parquet:parquet-format-structures:jar:1.12.3:provided
[INFO] | | \- org.apache.parquet:parquet-jackson:jar:1.12.3:provided
[INFO] | \- com.google.protobuf:protobuf-java:jar:3.21.12:provided
[INFO] +- org.scala-lang:scala-library:jar:2.13.12:provided
[INFO] +- com.crealytics:spark-excel_2.13:jar:3.4.1_0.19.0:compile
[INFO] | +- org.apache.poi:poi:jar:5.2.3:compile
[INFO] | +- org.apache.poi:poi-ooxml:jar:5.2.3:compile
[INFO] | +- org.apache.poi:poi-ooxml-lite:jar:5.2.3:compile
[INFO] | +- org.apache.xmlbeans:xmlbeans:jar:5.1.1:compile
[INFO] | +- com.norbitltd:spoiwo_2.13:jar:2.2.1:compile
[INFO] | | \- com.github.tototoshi:scala-csv_2.13:jar:1.3.10:compile
[INFO] | +- com.github.pjfanning:excel-streaming-reader:jar:4.0.5:compile
[INFO] | +- com.github.pjfanning:poi-shared-strings:jar:2.5.6:compile
[INFO] | | \- com.h2database:h2:jar:2.1.214:runtime
[INFO] | +- com.zaxxer:SparseBitSet:jar:1.2:compile
[INFO] | +- com.github.virtuald:curvesapi:jar:1.07:compile
[INFO] | \- org.scala-lang.modules:scala-collection-compat_2.13:jar:2.10.0:compile
[INFO] +- com.amazonaws:aws-java-sdk-sqs:jar:1.12.508:compile
[INFO] | +- com.amazonaws:aws-java-sdk-core:jar:1.12.508:compile
[INFO] | | +- software.amazon.ion:ion-java:jar:1.0.2:compile
[INFO] | | +- com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:jar:2.12.6:compile
[INFO] | | \- joda-time:joda-time:jar:2.8.1:compile
[INFO] | \- com.amazonaws:jmespath-java:jar:1.12.508:compile
[INFO] +- com.mycompany.mcp.fhir:common-fhir:jar:9.0.0-SNAPSHOT:provided
[INFO] | +- io.swagger.core.v3:swagger-annotations:jar:2.2.16:provided
[INFO] | +- io.swagger:swagger-annotations:jar:1.6.11:provided
[INFO] | +- org.reflections:reflections:jar:0.10.2:provided
[INFO] | +- javax.xml.bind:jaxb-api:jar:2.3.1:provided
[INFO] | | \- javax.activation:javax.activation-api:jar:1.2.0:provided
[INFO] | \- commons-beanutils:commons-beanutils:jar:1.9.4:provided
[INFO] +- com.mycompany.mcp.fhir:common-utilities:jar:9.0.0-SNAPSHOT:provided
[INFO] | \- org.springframework:spring-web:jar:5.3.22:compile
[INFO] | \- org.springframework:spring-beans:jar:5.3.22:compile
[INFO] +- com.mycompany.mcp.utilities:common-clients:jar:8.0.0-SNAPSHOT:compile
[INFO] | +- com.mycompany.mcp.resources:singularity-models:jar:7.15.1-SNAPSHOT:compile
[INFO] | | +- com.amazonaws:aws-java-sdk-dynamodb:jar:1.12.398:compile
[INFO] | | \- com.amazonaws:aws-java-sdk-sts:jar:1.12.398:compile
[INFO] | +- com.mycompany.mcp.utilities:model-utilities:jar:8.0.0-SNAPSHOT:compile
[INFO] | | +- com.github.java-json-tools:msg-simple:jar:1.2:compile
[INFO] | | | \- com.github.java-json-tools:btf:jar:1.3:compile
[INFO] | | +- com.github.java-json-tools:jackson-coreutils:jar:2.0:compile
[INFO] | | \- com.contentful.java:java-sdk:jar:10.5.16:compile
[INFO] | | +- com.squareup.retrofit2:retrofit:jar:2.9.0:compile
[INFO] | | +- com.squareup.retrofit2:adapter-rxjava3:jar:2.9.0:compile
[INFO] | | +- com.squareup.retrofit2:converter-gson:jar:2.9.0:compile
[INFO] | | +- io.reactivex.rxjava3:rxjava:jar:3.1.3:compile
[INFO] | | +- com.squareup.okhttp3:okhttp:jar:3.12.12:compile
[INFO] | | | \- com.squareup.okio:okio:jar:1.15.0:compile
[INFO] | | \- com.github.tony19:named-regexp:jar:0.2.5:compile
[INFO] | +- org.elasticsearch:elasticsearch:jar:6.8.21:compile
[INFO] | | +- org.elasticsearch:elasticsearch-core:jar:6.8.21:compile
[INFO] | | +- org.elasticsearch:elasticsearch-secure-sm:jar:6.8.21:compile
[INFO] | | +- org.elasticsearch:elasticsearch-x-content:jar:6.8.21:compile
[INFO] | | | +- com.fasterxml.jackson.dataformat:jackson-dataformat-smile:jar:2.8.11:compile
[INFO] | | | \- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar:2.8.11:compile
[INFO] | | +- org.apache.lucene:lucene-core:jar:7.7.3:compile
[INFO] | | +- org.apache.lucene:lucene-analyzers-common:jar:7.7.3:compile
[INFO] | | +- org.apache.lucene:lucene-backward-codecs:jar:7.7.3:compile
[INFO] | | +- org.apache.lucene:lucene-grouping:jar:7.7.3:compile
[INFO] | | +- org.apache.lucene:lucene-highlighter:jar:7.7.3:compile
[INFO] | | +- org.apache.lucene:lucene-join:jar:7.7.3:compile
[INFO] | | +- org.apache.lucene:lucene-memory:jar:7.7.3:compile
[INFO] | | +- org.apache.lucene:lucene-misc:jar:7.7.3:compile
[INFO] | | +- org.apache.lucene:lucene-queries:jar:7.7.3:compile
[INFO] | | +- org.apache.lucene:lucene-queryparser:jar:7.7.3:compile
[INFO] | | +- org.apache.lucene:lucene-sandbox:jar:7.7.3:compile
[INFO] | | +- org.apache.lucene:lucene-spatial:jar:7.7.3:compile
[INFO] | | +- org.apache.lucene:lucene-spatial-extras:jar:7.7.3:compile
[INFO] | | +- org.apache.lucene:lucene-spatial3d:jar:7.7.3:compile
[INFO] | | +- org.apache.lucene:lucene-suggest:jar:7.7.3:compile
[INFO] | | +- org.elasticsearch:elasticsearch-cli:jar:6.8.21:compile
[INFO] | | | \- net.sf.jopt-simple:jopt-simple:jar:5.0.2:compile
[INFO] | | +- com.carrotsearch:hppc:jar:0.7.1:compile
[INFO] | | +- com.tdunning:t-digest:jar:3.2:compile
[INFO] | | +- org.hdrhistogram:HdrHistogram:jar:2.1.9:compile
[INFO] | | \- org.elasticsearch:jna:jar:5.5.0:compile
[INFO] | +- com.google.guava:guava:jar:32.1.2-jre:compile
[INFO] | | +- com.google.guava:failureaccess:jar:1.0.1:compile
[INFO] | | +- com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava:compile
[INFO] | | +- org.checkerframework:checker-qual:jar:3.33.0:compile
[INFO] | | +- com.google.errorprone:error_prone_annotations:jar:2.18.0:compile
[INFO] | | \- com.google.j2objc:j2objc-annotations:jar:2.8:compile
[INFO] | +- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:2.15.2:compile
[INFO] | +- com.github.java-json-tools:json-patch:jar:1.13:compile
[INFO] | \- com.github.ben-manes.caffeine:caffeine:jar:3.1.8:compile
[INFO] +- software.amazon.awssdk:sqs:jar:2.20.101:compile
[INFO] | +- software.amazon.awssdk:aws-query-protocol:jar:2.20.101:compile
[INFO] | +- software.amazon.awssdk:protocol-core:jar:2.20.101:compile
[INFO] | +- software.amazon.awssdk:sdk-core:jar:2.20.101:compile
[INFO] | | +- software.amazon.awssdk:profiles:jar:2.20.101:compile
[INFO] | | \- org.reactivestreams:reactive-streams:jar:1.0.3:compile
[INFO] | +- software.amazon.awssdk:auth:jar:2.20.101:compile
[INFO] | | \- software.amazon.eventstream:eventstream:jar:1.0.1:compile
[INFO] | +- software.amazon.awssdk:http-client-spi:jar:2.20.101:compile
[INFO] | +- software.amazon.awssdk:regions:jar:2.20.101:compile
[INFO] | +- software.amazon.awssdk:annotations:jar:2.20.101:compile
[INFO] | +- software.amazon.awssdk:utils:jar:2.20.101:compile
[INFO] | +- software.amazon.awssdk:aws-core:jar:2.20.101:compile
[INFO] | +- software.amazon.awssdk:metrics-spi:jar:2.20.101:compile
[INFO] | +- software.amazon.awssdk:json-utils:jar:2.20.101:compile
[INFO] | | \- software.amazon.awssdk:third-party-jackson-core:jar:2.20.101:compile
[INFO] | +- software.amazon.awssdk:endpoints-spi:jar:2.20.101:compile
[INFO] | +- software.amazon.awssdk:apache-client:jar:2.20.101:runtime
[INFO] | | \- org.apache.httpcomponents:httpcore:jar:4.4.13:compile
[INFO] | \- software.amazon.awssdk:netty-nio-client:jar:2.20.101:runtime
[INFO] +- software.amazon.awssdk:sns:jar:2.20.101:compile
[INFO] +- com.amazonaws:aws-java-sdk-sns:jar:1.12.508:compile
[INFO] +- com.amazonaws:aws-java-sdk-s3:jar:1.12.508:compile
[INFO] | \- com.amazonaws:aws-java-sdk-kms:jar:1.12.508:compile
[INFO] +- com.fasterxml.jackson.core:jackson-core:jar:2.12.1:compile
[INFO] +- com.mycompany.mcp-common-eventbus:models:jar:5.0.2:compile
[INFO] +- com.mycompany.mcp.metamorphosis:metamorphosis-models:jar:2.9.0:compile
[INFO] +- com.mycompany:mcp-boot-starter-monitoring:pom:5.4.0:compile
[INFO] | +- com.mycompany:mcp-common-monitoring:jar:2.9.0:compile
[INFO] | | +- ch.qos.logback:logback-access:jar:1.2.3:compile
[INFO] | | | \- ch.qos.logback:logback-core:jar:1.2.3:compile
[INFO] | | +- ch.qos.logback:logback-classic:jar:1.2.3:compile
[INFO] | | +- net.logstash.logback:logstash-logback-encoder:jar:6.2:compile
[INFO] | | \- org.aspectj:aspectjrt:jar:1.9.4:compile
[INFO] | +- com.mycompany:mcp-boot-autoconfigure:jar:5.4.0:compile
[INFO] | \- org.springframework.boot:spring-boot-starter:jar:2.5.2:compile
[INFO] | +- org.springframework.boot:spring-boot-starter-logging:jar:2.5.2:compile
[INFO] | | \- org.apache.logging.log4j:log4j-to-slf4j:jar:2.14.1:compile
[INFO] | \- org.yaml:snakeyaml:jar:1.28:compile
[INFO] +- junit:junit:jar:4.13.2:test
[INFO] | \- org.hamcrest:hamcrest-core:jar:1.3:test
[INFO] +- org.projectlombok:lombok:jar:1.18.28:compile
[INFO] +- org.apache.hadoop:hadoop-aws:jar:3.3.6:provided
[INFO] | +- com.amazonaws:aws-java-sdk-bundle:jar:1.12.367:provided
[INFO] | \- org.wildfly.openssl:wildfly-openssl:jar:1.1.3.Final:provided
[INFO] +- org.apache.hadoop:hadoop-common:jar:3.3.6:provided
[INFO] | +- org.apache.hadoop.thirdparty:hadoop-shaded-protobuf_3_7:jar:1.1.1:provided
[INFO] | +- org.apache.hadoop:hadoop-annotations:jar:3.3.6:provided
[INFO] | +- org.apache.hadoop.thirdparty:hadoop-shaded-guava:jar:1.1.1:provided
[INFO] | +- commons-cli:commons-cli:jar:1.2:provided
[INFO] | +- org.apache.httpcomponents:httpclient:jar:4.5.13:compile
[INFO] | +- commons-net:commons-net:jar:3.9.0:provided
[INFO] | +- javax.servlet:javax.servlet-api:jar:3.1.0:compile
[INFO] | +- jakarta.activation:jakarta.activation-api:jar:1.2.1:provided
[INFO] | +- org.eclipse.jetty:jetty-server:jar:9.4.51.v20230217:provided
[INFO] | | +- org.eclipse.jetty:jetty-http:jar:9.4.51.v20230217:provided
[INFO] | | \- org.eclipse.jetty:jetty-io:jar:9.4.51.v20230217:provided
[INFO] | +- org.eclipse.jetty:jetty-util:jar:9.4.51.v20230217:provided
[INFO] | +- org.eclipse.jetty:jetty-servlet:jar:9.4.51.v20230217:provided
[INFO] | | +- org.eclipse.jetty:jetty-security:jar:9.4.51.v20230217:provided
[INFO] | | \- org.eclipse.jetty:jetty-util-ajax:jar:9.4.51.v20230217:provided
[INFO] | +- org.eclipse.jetty:jetty-webapp:jar:9.4.51.v20230217:provided
[INFO] | | \- org.eclipse.jetty:jetty-xml:jar:9.4.51.v20230217:provided
[INFO] | +- javax.servlet.jsp:jsp-api:jar:2.1:provided
[INFO] | +- com.sun.jersey:jersey-core:jar:1.19.4:provided
[INFO] | | \- javax.ws.rs:jsr311-api:jar:1.1.1:provided
[INFO] | +- com.sun.jersey:jersey-servlet:jar:1.19.4:provided
[INFO] | +- com.github.pjfanning:jersey-json:jar:1.20:provided
[INFO] | | +- org.codehaus.jettison:jettison:jar:1.1:provided
[INFO] | | \- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:provided
[INFO] | +- com.sun.jersey:jersey-server:jar:1.19.4:provided
[INFO] | +- commons-logging:commons-logging:jar:1.1.3:compile
[INFO] | +- ch.qos.reload4j:reload4j:jar:1.2.22:provided
[INFO] | +- org.apache.commons:commons-configuration2:jar:2.8.0:provided
[INFO] | +- org.slf4j:slf4j-reload4j:jar:1.7.36:provided
[INFO] | +- com.google.re2j:re2j:jar:1.1:provided
[INFO] | +- com.google.code.gson:gson:jar:2.9.0:compile
[INFO] | +- org.apache.hadoop:hadoop-auth:jar:3.3.6:provided
[INFO] | | +- com.nimbusds:nimbus-jose-jwt:jar:9.8.1:provided
[INFO] | | | \- com.github.stephenc.jcip:jcip-annotations:jar:1.0-1:provided
[INFO] | | \- org.apache.kerby:kerb-simplekdc:jar:1.0.1:provided
[INFO] | | +- org.apache.kerby:kerb-client:jar:1.0.1:provided
[INFO] | | | +- org.apache.kerby:kerby-config:jar:1.0.1:provided
[INFO] | | | +- org.apache.kerby:kerb-common:jar:1.0.1:provided
[INFO] | | | | \- org.apache.kerby:kerb-crypto:jar:1.0.1:provided
[INFO] | | | +- org.apache.kerby:kerb-util:jar:1.0.1:provided
[INFO] | | | \- org.apache.kerby:token-provider:jar:1.0.1:provided
[INFO] | | \- org.apache.kerby:kerb-admin:jar:1.0.1:provided
[INFO] | | +- org.apache.kerby:kerb-server:jar:1.0.1:provided
[INFO] | | | \- org.apache.kerby:kerb-identity:jar:1.0.1:provided
[INFO] | | \- org.apache.kerby:kerby-xdr:jar:1.0.1:provided
[INFO] | +- com.jcraft:jsch:jar:0.1.55:provided
[INFO] | +- org.apache.curator:curator-client:jar:5.2.0:provided
[INFO] | +- org.apache.kerby:kerb-core:jar:1.0.1:provided
[INFO] | | \- org.apache.kerby:kerby-pkix:jar:1.0.1:provided
[INFO] | | +- org.apache.kerby:kerby-asn1:jar:1.0.1:provided
[INFO] | | \- org.apache.kerby:kerby-util:jar:1.0.1:provided
[INFO] | +- org.codehaus.woodstox:stax2-api:jar:4.2.1:provided
[INFO] | +- com.fasterxml.woodstox:woodstox-core:jar:5.4.0:provided
[INFO] | \- dnsjava:dnsjava:jar:2.1.7:provided
[INFO] +- org.mockito:mockito-core:jar:3.3.3:test
[INFO] | +- net.bytebuddy:byte-buddy:jar:1.10.5:test
[INFO] | +- net.bytebuddy:byte-buddy-agent:jar:1.10.5:test
[INFO] | \- org.objenesis:objenesis:jar:2.6:provided
[INFO] +- com.holdenkarau:spark-testing-base_2.13:jar:3.4.1_1.4.4:test
[INFO] | +- org.apache.spark:spark-streaming_2.13:jar:3.4.1:test
[INFO] | +- org.apache.spark:spark-hive_2.13:jar:3.4.1:test
[INFO] | | +- org.apache.hive:hive-common:jar:2.3.9:test
[INFO] | | | +- commons-lang:commons-lang:jar:2.6:test
[INFO] | | | +- jline:jline:jar:2.12:test
[INFO] | | | +- com.tdunning:json:jar:1.8:test
[INFO] | | | \- com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter:jar:0.1.2:test
[INFO] | | +- org.apache.hive:hive-exec:jar:core:2.3.9:test
[INFO] | | | +- org.antlr:antlr-runtime:jar:3.5.2:test
[INFO] | | | +- org.antlr:ST4:jar:4.0.4:test
[INFO] | | | \- stax:stax-api:jar:1.0.1:test
[INFO] | | +- org.apache.hive:hive-metastore:jar:2.3.9:test
[INFO] | | | +- javolution:javolution:jar:5.5.1:test
[INFO] | | | +- com.jolbox:bonecp:jar:0.8.0.RELEASE:test
[INFO] | | | +- com.zaxxer:HikariCP:jar:2.5.1:test
[INFO] | | | +- org.datanucleus:datanucleus-api-jdo:jar:4.2.4:test
[INFO] | | | +- org.datanucleus:datanucleus-rdbms:jar:4.1.19:test
[INFO] | | | +- commons-pool:commons-pool:jar:1.5.4:test
[INFO] | | | +- commons-dbcp:commons-dbcp:jar:1.4:test
[INFO] | | | +- javax.jdo:jdo-api:jar:3.0.1:test
[INFO] | | | | \- javax.transaction:jta:jar:1.1:test
[INFO] | | | \- org.datanucleus:javax.jdo:jar:3.2.0-m3:test
[INFO] | | | \- javax.transaction:transaction-api:jar:1.1:test
[INFO] | | +- org.apache.hive:hive-serde:jar:2.3.9:test
[INFO] | | | \- net.sf.opencsv:opencsv:jar:2.3:test
[INFO] | | +- org.apache.hive:hive-shims:jar:2.3.9:test
[INFO] | | | +- org.apache.hive.shims:hive-shims-common:jar:2.3.9:test
[INFO] | | | +- org.apache.hive.shims:hive-shims-0.23:jar:2.3.9:test
[INFO] | | | \- org.apache.hive.shims:hive-shims-scheduler:jar:2.3.9:test
[INFO] | | +- org.apache.hive:hive-llap-common:jar:2.3.9:test
[INFO] | | +- org.apache.hive:hive-llap-client:jar:2.3.9:test
[INFO] | | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:test
[INFO] | | | \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:test
[INFO] | | +- org.jodd:jodd-core:jar:3.5.2:test
[INFO] | | +- org.datanucleus:datanucleus-core:jar:4.1.17:test
[INFO] | | +- org.apache.thrift:libthrift:jar:0.12.0:test
[INFO] | | +- org.apache.thrift:libfb303:jar:0.9.3:test
[INFO] | | \- org.apache.derby:derby:jar:10.14.2.0:test
[INFO] | +- org.apache.spark:spark-yarn_2.13:jar:3.4.1:test
[INFO] | +- org.apache.spark:spark-mllib_2.13:jar:3.4.1:test
[INFO] | | +- org.apache.spark:spark-graphx_2.13:jar:3.4.1:test
[INFO] | | | \- net.sourceforge.f2j:arpack_combined_all:jar:0.1:test
[INFO] | | +- org.apache.spark:spark-mllib-local_2.13:jar:3.4.1:test
[INFO] | | +- org.scalanlp:breeze_2.13:jar:2.1.0:test
[INFO] | | | +- org.scalanlp:breeze-macros_2.13:jar:2.1.0:test
[INFO] | | | +- com.github.wendykierp:JTransforms:jar:3.1:test
[INFO] | | | | \- pl.edu.icm:JLargeArrays:jar:1.5:test
[INFO] | | | \- org.typelevel:spire_2.13:jar:0.18.0:test
[INFO] | | | +- org.typelevel:spire-macros_2.13:jar:0.18.0:test
[INFO] | | | +- org.typelevel:spire-platform_2.13:jar:0.18.0:test
[INFO] | | | +- org.typelevel:spire-util_2.13:jar:0.18.0:test
[INFO] | | | \- org.typelevel:algebra_2.13:jar:2.8.0:test
[INFO] | | | \- org.typelevel:cats-kernel_2.13:jar:2.8.0:test
[INFO] | | +- org.glassfish.jaxb:jaxb-runtime:jar:2.3.2:test
[INFO] | | | +- jakarta.xml.bind:jakarta.xml.bind-api:jar:2.3.2:test
[INFO] | | | \- com.sun.istack:istack-commons-runtime:jar:3.0.8:test
[INFO] | | +- dev.ludovic.netlib:blas:jar:3.0.3:test
[INFO] | | +- dev.ludovic.netlib:lapack:jar:3.0.3:test
[INFO] | | \- dev.ludovic.netlib:arpack:jar:3.0.3:test
[INFO] | +- org.scalatest:scalatest_2.13:jar:3.2.15:test
[INFO] | | +- org.scalatest:scalatest-core_2.13:jar:3.2.15:test
[INFO] | | | +- org.scalatest:scalatest-compatible:jar:3.2.15:test
[INFO] | | | \- org.scalactic:scalactic_2.13:jar:3.2.15:test
[INFO] | | +- org.scalatest:scalatest-featurespec_2.13:jar:3.2.15:test
[INFO] | | +- org.scalatest:scalatest-flatspec_2.13:jar:3.2.15:test
[INFO] | | +- org.scalatest:scalatest-freespec_2.13:jar:3.2.15:test
[INFO] | | +- org.scalatest:scalatest-funsuite_2.13:jar:3.2.15:test
[INFO] | | +- org.scalatest:scalatest-funspec_2.13:jar:3.2.15:test
[INFO] | | +- org.scalatest:scalatest-propspec_2.13:jar:3.2.15:test
[INFO] | | +- org.scalatest:scalatest-refspec_2.13:jar:3.2.15:test
[INFO] | | +- org.scalatest:scalatest-wordspec_2.13:jar:3.2.15:test
[INFO] | | +- org.scalatest:scalatest-diagrams_2.13:jar:3.2.15:test
[INFO] | | +- org.scalatest:scalatest-matchers-core_2.13:jar:3.2.15:test
[INFO] | | +- org.scalatest:scalatest-shouldmatchers_2.13:jar:3.2.15:test
[INFO] | | \- org.scalatest:scalatest-mustmatchers_2.13:jar:3.2.15:test
[INFO] | +- org.scalatestplus:scalacheck-1-15_2.13:jar:3.2.3.0:test
[INFO] | +- org.scalatestplus:junit-4-13_2.13:jar:3.2.15.0:test
[INFO] | +- org.scalacheck:scalacheck_2.13:jar:1.15.2:test
[INFO] | | \- org.scala-sbt:test-interface:jar:1.0:test
[INFO] | \- io.netty:netty-tcnative-classes:jar:2.0.52.Final:test
[INFO] +- org.powermock:powermock-module-junit4:jar:2.0.9:test
[INFO] | \- org.powermock:powermock-module-junit4-common:jar:2.0.9:test
[INFO] | +- org.powermock:powermock-reflect:jar:2.0.9:test
[INFO] | \- org.powermock:powermock-core:jar:2.0.9:test
[INFO] \- org.powermock:powermock-api-mockito2:jar:2.0.9:test
[INFO] \- org.powermock:powermock-api-support:jar:2.0.9:test
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7.507 s
[INFO] Finished at: 2023-09-22T18:34:11-07:00
[INFO] ------------------------------------------------------------------------
From the other questions like the one linked above, it sounds like my inclination was correct and I missed a package somewhere, but I don't see any packages that provide scala 2.12 anywhere. Is there any other way to locate this nested scala package?
Scala has binary compatility, as you seem to already know.
Part of your code is going to run on spark context.
So, upgrading all your dependencies to be binary compatible with Scala 2.13 won't be enough.
Your spark cluster must also be built with Scala 2.13.
In spark official site you can to download a 2.13 version of it.
In this link choose a package type with Scala 2.13 and try run this version in your local.
This answer is more of an educated guess.
(Something so fundamental like the JVM not finding a class that is part of the Scala library, might have happened because of scala versions incompatibility)
However, If your spark installation in your local is already built with Scala 2.13 you might disregard my answer.
P.S. that would be a comment but I do not have enough reputation for that.
Hope that helps!