I'm trying to connect to Apache Solr datasource from Superset. As per my understanding Solr is in Java and Superset is developed in python and there is no dialect for Solr in SqlAlchemy.
You can't create a Superset datasource for Solr out of the box as (to your point) there is no SQLAlchemy dialect for Solr.
Note that SQLAlchemy (or Superset) wouldn't care whether that datasource is written in java (or Fortran for that matter) - as long as there is a functional SqlAlchemy dialect and Python driver.
That being said, the reason why a SQLAlchemy dialect doesn't exist for Solr is that they're built on different purposes and based off different structures.
Your best bet is probably to implement some type of data extraction process, to get the data you need out of Solr, and put it into a supported database.