But you could replace sudo with something that can login locally via other means like SSH. You need to use SSH(-agent) forwarding to pass down your identity so you won't have to type in any password. You then can decide with -A or -a option for ssh whether to enable or disable SSH forwarding (and thus root access).
It's not very fine grained though and it won't ask for a password each time you 'sshdo' but you could probably set something up with PAM settings (multi-factor). I'll look into that later.
Anyway, for root:
/root/.ssh/authorized_keys should contain the allowed key. You'll want root access only locally and only via passwordless-authentication. For that, add to
PermitRootLogin no Match Address 127.0.0.1 PermitRootLogin without-password
In your normal user directory, you can add an easy-to-use alias similar to sudo to
alias sshdo='ssh -q -t root@localhost -- cd $PWD \&\& sudo'
Test it and then disable sudo.
Hopefully this gives some good ideas.