javahadoopapache-sparkhdfsapache-flink
HDFS Path for Spark Submit and Flink on YARN
i work with cloudera live vm, there i have a hadoop and spral standalone cluster.
now i want submit my jobs with spark submit and flink run scripts. this works, too. but my apps can find the path to input and outputs files in the hdfs.
i set the path like: hdfs://127.0.0.1:50010/user/cloudera/outputs
the information from the port i get from this:
how to set the path for the hdfs in java??
best regards, paul
Solution
You don't have to set the path to the DataNode host. On the "Overview" page, you'll see the connection information for the NameNode, and that is also where you have to connect to.
- Java can't find symbol
- Java application not recording from microphone when packaged using install4j
- Using kafka-schema-registry-maven-plugin with parent and child schemas
- How to add a "driver" to javax.comm? Serial port programming in Java
- Java 6: Unsupported @SuppressWarnings("rawtypes") warning
- java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to com.testing.models.Account
- Connecting 2 java files in visual studio
- Unnecessary warnings for Freemarker classes on native image execution
- How do I increase the default timeout in the Cassandra Java driver using the DriverConfigLoader?
- NetBeans IDE Java 1.4 compatibility: compiler not warning on JDK 5+ classes/methods
- what is the need for importing libraries multiple times
- Converting number to word
- Spring Boot - Return an empty array of objects
- Map implementation with duplicate keys
- Do we have XSDs particularly for Spring 5.x?
- Spring + Ehcache : How to cache find all result
- Java 17+ Native-Image Logback
- Kafka Best Practices + how to set recommended setting for JVM
- Encode a codepoint
- What causes "Unable to access jarfile" error?
- Lombok getter/setter vs Java 14 record
- Equivalent to DataJpaTest for Spring Data's ReactiveCrudRepository and R2DBC
- "FATAL: remaining connection slots are reserved for non-replication superuser connections" triggers despite closing connections
- How to load properties from application.properties with @ExtendWith(SpringExtension.class)
- How to return N consecutive elements from a Collection?
- %Like% Query in spring JpaRepository
- Paging implimentation using Hibernate with Struts2
- How to return values of only one parameter in GetMapping?
- Page through an array
- How can I create a memory leak in Java?