Is there a way to install Cuda and switch agent to python 3.5?
Or access agent’s console at least? 
On a GPU agent CUDA should be installed. I see no point of installing it on a non GPU agent. Concerning the Python 3.5, we support only the latest Python 3.6. We do not recommend doing anything with the agent though theoretically it’s possible.
Could you describe why you need Python 3.5?
For some reason tensorflow-gpu doesn’t recognize installed CUDA on agent, thought it might be problem with python version. I know that you don’t support tf yet, but I’ve heard from Denis Stepanov, that you can use Tensorflow if all code is written within one block.
Installing tf with cuda support is quite an elaborate process requiring not only installing CUDA and cuDNN, but also setting right environment variables. We don’t support tensorflow, so, please, do not use it.