MSYS Git and Cygwin and PuTTY do play well together, but there are a couple gotchas.
When using plink.exe for Git’s SSH client, you must make sure you both add your key to your PuTTY Agent and also you need to connect once first manually to make sure you add the host’s key to the known hosts cache, otherwise MSYS Git will abandon the connection because apparently it can’t handle asking you whether you want to save the host key – you just need to have it saved already.
I’ve found that my ideal middle ground is to choose to add only Git to the system path. Now, normally I install git from cygwin, but if you add only MSYS Git to your PATH then you shouldn’t install the git from Cygwin because the two will conflict and Cygwin’s Git will be confused by the GIT_SSH variable since Cygwin’s git can’t use PuTTY for SSH.
