Databricks in Power Platform dataflows

Not all Power Query / Dataflow engines are the same... this includes connectors. Issue: I was recently challenged to connect Microsoft Sustainability Manager, which uses Dataverse underneath, to a Databricks SQL warehouse. I thought it would be easy, as I know that in Power BI, there are two separate Databricks connectors... so the same should apply to the Power Platform? It’s different. Solution: If you need to overcome this issue, there is a legacy trick from the time when the Databricks connector did not exist, even for Power BI: using Spark with HTTPS SQL endpoint configured. This way, you can pull any data from Databricks into your Dataverse solution, including Microsoft Sustainability Manager! Other less sufficient methods would be to use Power Automate, Rest API, expose this data somewhere else as files, or first ingest to it Power BI and later into Dataverse. Each of those would be unnecessarily difficult and limited. Additional observations: I always knew Power Platform handles dataflows a bit differently but I never knew about connector limitations. Dataflows in the Power Platform have one of the most disconnected ALM capabilities. There is only one owner for each dataflow and to edit a component owned by somebody else, first you need to steal it! Power Platform Dataflows are not a high priority in Microsoft's Development roadmap, so unfortunately there are no hopes it will be improved soon. Have you ever faced any other discrepancies? Please comment below! #PowerApps #PowerPlatform #Dataverse #PowerQuery #Dataflow #Fabric #Databricks #SQL #MSM #MicrosoftSustainabilityManager