Is there an easy way to stream driver logs from Databricks cluster into CLI? I tried Databricks CLI, but found only Event logs. I was thinking about databricks-connect, but never found any details if it is possible to got logs from there. If there are other ways to have logs streaming would love to see. I need it for demo to show that my databricks cluster actually receives the data. If you have other ideas how to show it please share. Thank you!
It's not possible to stream logs live, but you can configure the cluster to push driver & executor logs to the DBFS, and then you can copy that logs from DBFS to the local machine using the Databricks CLI's fs command (doc).