How to filter 99.99% of ssh brute force attacks

Recently I've decided to experiment with ssh ciphers / key exchange algorithms to raise the security of my servers. This is the /etc/ssh/sshd_config I've got:

HostKey /etc/ssh/ssh_host_ed25519_key
Ciphers chacha20-poly1305@openssh.com
KexAlgorithms curve25519-sha256@libssh.org

If you don't have HostKey for Ed25519 generate it:

# ssh-keygen -t ed25519

You need to use recent version of ssh / pyTTY to be able to login to this server.
It seems that using only this Cipher/Kex filters all brute force scanners probably because they do not support it. I see only this kind of messages:

Feb 13 14:41:39 horizon9 sshd[22849]: SSH: Server;Ltype: Version;Remote: xxx.x.xx.xxx-53810;Protocol: 2.0;Client: libssh2_1.7.0
Feb 13 14:41:39 horizon9 sshd[22849]: fatal: ssh_dispatch_run_fatal: no matching cipher found [preauth]

More information about ciphers/algorithms read here:

https://cr.yp.to/ecdh.html#curve25519-paper
https://en.wikipedia.org/wiki/Salsa20
https://en.wikipedia.org/wiki/Poly1305