Does Datalore Enterprise supports Spark (PySpark)? Are there any documentation? or any suggestions or pointers will be helpful.
Unfortunately, it seems like they currently only have support for Spark SQL through the thrift-server (hive2). Trying to connect using traditional spark / pyspark is not possible without open ports on the agent docker container. I have an open post here: Issues Running PySpark on Remote Hadoop/Yarn Cluster where I’m asking for JetBrains direction on this.