I want to setup a k8s cluster, but I despair with the nginx-ingress controller and some special settings I need to set: especially proxy_pass.

I tried to achieve that already with the "server-snippet"-snippet, but it didn't work.

apiVersion: networking.k8s.io/v1beta1 # for versions before 1.14 use extensions/v1beta1
kind: Ingress
metadata:
    name: ingress
    annotations:
        nginx.ingress.kubernetes.io/rewrite-target: /$1
        nginx.ingress.kubernetes.io/affinity: "cookie"
        nginx.ingress.kubernetes.io/session-cookie-name: "route"
        nginx.ingress.kubernetes.io/session-cookie-expires: "172800"
        nginx.ingress.kubernetes.io/session-cookie-max-age: "172800"
        nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
        nginx.ingress.kubernetes.io/ssl-redirect: "false"
        nginx.ingress.kubernetes.io/hsts: "false"
        nginx.ingress.kubernetes.io/server-snippet: |
            location / {
                internal;
                proxy_set_header X-Forwarded-Host $host;
                proxy_set_header X-Forwarded-Server $host;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Proto $scheme;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_buffering off;

                proxy_pass http://localhost:30022/site/;
                proxy_redirect default;
                proxy_cookie_path /site/ /;
            }
spec:
    rules:
        - host: preview.test.de
          http:
              paths:
                  - path: /
                    backend:
                        serviceName: backend-service
                        servicePort: 8080

What I want to achieve is this nginx config:

location / {
                internal;
                proxy_set_header X-Forwarded-Host $host;
                proxy_set_header X-Forwarded-Server $host;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Proto $scheme;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_buffering off;

                proxy_pass http://localhost:30022/site/;
                proxy_redirect default;
                proxy_cookie_path /site/ /;
            }

In an optimal world I would like to achieve that the host and the port in the proxy_pass directive would be depend on the backend pod I want to connect to, so that there is no hard coded port.

Can anyone help me out with this problem?

I believe what you're looking for is:

nginx.ingress.kubernetes.io/rewrite-target: /site

However this will effectively mean that you can only use this particular ingress instance for this one app, or others like it, since this rewrite would apply to all the rules for that ingress instance.

You might be able to accomplish the same thing using a regex based rule, but this approach is definitely simpler.

Regarding your question about how to handle the two different paths which require rewrites, this should be possible with the following ingress config:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
    name: ingress
    annotations:
        nginx.ingress.kubernetes.io/rewrite-target: /$1
        nginx.ingress.kubernetes.io/use-regex: true
spec:
    rules:
        - host: preview.test.de
          http:
              paths:
                  - path: /(site/.*)
                    backend:
                        serviceName: backend-service
                        servicePort: 8080
                  - path: /(cms/.*)
                    backend:
                        serviceName: cms-service
                        servicePort: 8080

However as you might be able to guess, this might become more difficult if you end up having many paths with lots of rewrites later on.

As for best practices for configuring this type of set up, I would generally recommend to adopt sub-domains. Set up one for each site/path you want to be reachable, and let them have their own distinct nginx/ingress/etc to handle their rewrites as needed. Then if you want to tie these together later into some other top level domain/site, it's easily done and not having to manage many rewrite rules in the one location, which can get quite messy.