In order to be able to make use of Datalore, I’d need to be able to access a number of SQL servers.
In our case, the SQL servers are in Google Cloud SQL.
To connect to Google Cloud SQL, we need to either run a proxy in datalore, or whitelist ip addresses and set up a public ip address on Cloud SQL.
Using the proxy makes a lot more sense, and there seems to be some ability to run the proxy from python (https://github.com/GoogleCloudPlatform/cloud-sql-python-connector)
However, we need to be able to use either an ssh key or a google cloud credentials file in order to set up any proxy, and Datalore secrets seem to only support a limited number of characters.
We don’t want to store secrets in a git repository, and we don’t want to have to upload a secret file for each notebook.
Is there any way we could use Cloud SQL or add secret files?
Hi @dane.r.lowe,
Thanks for report, as a first solution we’ll increase a size of a secret. I’ll post here when it is implemented.
We are also going to release directories that can be attached to multiple notebooks soon, so it will be a solution for the credentials file.
Please let us know if there are other issues or suggestions!
Thanks.
I think what I will do is create a python Context Manager that creates client cert/key temp files if they dont exist (by reading contents from env vars), then creates a psycopg2/pymysql connection with password from env vars.
Is there an IP address range that Datalore uses?
We’ve increased the secret length limit, so the first approach should work now.
Datalore uses AWS for agents, there are no other restrictions, so the IP range is quire broad: https://ip-ranges.amazonaws.com/ip-ranges.json. Currently only eu-west-1 is used but it may change in future