I've scanned through all resources, still cannot find a way to change extraPortMappings in Kind cluster without deleting and creating again.

Is it possible and how?

It's not said explicitly in the official docs, but I found some references that confirm: your thoughts are correct and changing extraPortMappings (as well as other cluster settings) is only possible with recreation of the kind cluster.

if you use extraPortMappings in your config, they are “fixed” and cannot be modified, unless you recreate the cluster.

Source - Issues I Came Across

Note that the cluster configuration cannot be changed. The only workaround is to delete the cluster (see below) and create another one with the new configuration.

Source - Kind Installation

However, there are obvious disadvantages in the configuration and update of the cluster, and the cluster can only be configured and updated by recreating the cluster. So you need to consider configuration options when you initialize the cluster.

Source - Restrictions