scalaapache-sparkanti-join
Left Anti join in Spark dataframes
I have two dataframes, and I would like to retrieve only the information of one of the dataframes, which is not found in the inner join, see the picture:
I have tried several ways: Inner join and filtering the rows that return at least one null, all the types of joins described on the docs for Spark 1.6 but failed to do obtain the results from just one join.
Can anybody help?
Solution
This is called right excluding join and you can do like below
df1.join(df2,df1("column1")===df2("column2"),"right_outer").filter("column1 is null").show
- Math.Sin() gives incorrect value
- How to run my python script when the sunOS is start booting
- Express-session: not resetting cookie expiration on each request
- Getting a stack overflow exception when normalizing a vector
- Edit default summary function in R gives error for multiple variables
- What was a For loop? Why isn't it needed in R?
- How to use download button in shiny and save results in various formats (csv, texte, pdf, spss...)?
- Why are there two assignment operators, `<-` and `->` in R?
- lm()$assign: what is it?
- How to get the value of list(...) in R and S functions
- Design matrix for MLM from library(lme4) with fixed and random effects
- how to generate elements not included in my sample
- Create a matrix with gradually changing values without a for loop
- Emacs ESS and S-plus ( S+ ) 8.1 compatability
- How to lag date-index in a time-series in R?
- Nonlinear regression in R / S
- Calling R from S-Plus?