EDIT:

I deleted minikube, enabled kubernetes in Docker desktop for Windows and installed ingress-nginx manually.

$helm upgrade --install ingress-nginx ingress-nginx --repo https://kubernetes.github.io/ingress-nginx --namespace ingress-nginx --create-namespace
Release "ingress-nginx" does not exist. Installing it now.
Error: rendered manifests contain a resource that already exists. Unable to continue with install: ServiceAccount "ingress-nginx" in namespace "ingress-nginx" exists and cannot be imported into the current release: invalid ownership metadata; annotation validation error: missing key "meta.helm.sh/release-name": must be set to "ingress-nginx"; annotation validation error: missing key "meta.helm.sh/release-namespace": must be set to "ingress-nginx"

It gave me an error but I think it's because I did it already before because:

$kubectl get svc -n ingress-nginx
NAME                                 TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             LoadBalancer   10.106.222.233   localhost     80:30199/TCP,443:31093/TCP   11m
ingress-nginx-controller-admission   ClusterIP      10.106.52.106    <none>        443/TCP                      11m

Then applied all my yaml files again but this time ingress is not getting any address:

$kubectl get ing
NAME                CLASS    HOSTS       ADDRESS   PORTS   AGE
myapp-ingress       <none>   myapp.com             80      10m

---

I am using docker desktop (windows) and installed nginx-ingress controller via minikube addons enable command:

$kubectl get pods -n ingress-nginx
NAME                                        READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create--1-lp4md     0/1     Completed   0          67m
ingress-nginx-admission-patch--1-jdkn7      0/1     Completed   1          67m
ingress-nginx-controller-5f66978484-6mpfh   1/1     Running     0          67m

And applied all my yaml files:

$kubectl get svc --all-namespaces -o wide
NAMESPACE       NAME                                 TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE   SELECTOR
default         event-service-svc                    ClusterIP      10.108.251.79    <none>        80/TCP                       16m   app=event-service-app
default         kubernetes                           ClusterIP      10.96.0.1        <none>        443/TCP                      16m   <none>
default         mssql-clusterip-srv                  ClusterIP      10.98.10.22      <none>        1433/TCP                     16m   app=mssql
default         mssql-loadbalancer                   LoadBalancer   10.109.106.174   <pending>     1433:31430/TCP               16m   app=mssql
default         user-service-svc                     ClusterIP      10.111.128.73    <none>        80/TCP                       16m   app=user-service-app
ingress-nginx   ingress-nginx-controller             NodePort       10.101.112.245   <none>        80:31583/TCP,443:30735/TCP   68m   app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
ingress-nginx   ingress-nginx-controller-admission   ClusterIP      10.105.169.167   <none>        443/TCP                      68m   app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
kube-system     kube-dns                             ClusterIP      10.96.0.10       <none>        53/UDP,53/TCP,9153/TCP       72m   k8s-app=kube-dns

All pods and services seems to be running properly. Checked the pod logs, all migrations etc. has worked and app is up and running. But when I try to send an HTTP request, I get a socket hang up error. I've checked all the logs for all pods, couldn't find anything useful.

$kubectl get ingress
NAME                CLASS   HOSTS           ADDRESS     PORTS   AGE
myapp-ingress   nginx   myapp.com       localhost   80      74s

This one is also a bit weird, I was expecting ADRESS to be set to an IP not to localhost. So adding 127.0.0.1 entry for myapp.com in /etc/hosts also didn't seem so right.

My question here is what I might be doing wrong? Or how can I even trace where are my requests are being forwarded to?

ingress-svc.yaml

  apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: myapp-ingress
    spec:
      rules:
      - host: myapp.com
        http:
          paths:
          - path: /api/Users
            pathType: Prefix
            backend:
              service:
                name: user-service-svc
                port:
                  number: 80
          - path: /api/Events
            pathType: Prefix
            backend:
              service:
                name: event-service-svc
                port:
                  number: 80

events-depl.yaml:

  apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: event-service-app
      labels:
        app: event-service-app
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: event-service-app
      template:
        metadata:
          labels:
            app: event-service-app
        spec:
          containers:
            - name: event-service-app
              image: ghcr.io/myapp/event-service:master
              imagePullPolicy: Always
              ports:
                - containerPort: 80
          imagePullSecrets:
            - name: myapp
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: event-service-svc
    spec:
      selector:
        app: event-service-app
      ports:
        - protocol: TCP
          port: 80
          targetPort: 80

Reproduction

I reproduced the case using minikube v1.24.0, Docker desktop 4.2.0, engine 20.10.10

First, localhost in ingress appears due to logic, it doesn't really matter what IP address is behind the domain in /etc/hosts, I added a different one for testing and still it showed localhost. Only metallb will provide an IP address from set up network.

What happens

When minikube driver is docker, minikube creates a big container (VM) where kubernetes components are run. This can be checked by running docker ps command in host system:

$ docker ps

CONTAINER ID   IMAGE                                 COMMAND                  CREATED          STATUS          PORTS                                                                                                                                  NAMES
f087dc669944   gcr.io/k8s-minikube/kicbase:v0.0.28   "/usr/local/bin/entr…"   16 minutes ago   Up 16 minutes   127.0.0.1:59762->22/tcp, 127.0.0.1:59758->2376/tcp, 127.0.0.1:59760->5000/tcp, 127.0.0.1:59761->8443/tcp, 127.0.0.1:59759->32443/tcp   minikube

And then minikube ssh to get inside this container and run docker ps to see all kubernetes containers.

Moving forward. Before introducing ingress, it's already clear that even NodePort doesn't work as intended. Let's check it.

There are two ways to get minikube VM IP:

1. run minikube IP
2. kubectl get nodes -o wide and find the node's IP

What should happen next with NodePort is requests should go to minikube_IP:Nodeport while it doesn't work. It happens because docker containers inside the minikube VM are not exposed outside of the cluster which is another docker container.

On minikube to access services within cluster there is a special command - minikube service %service_name% which will create a direct tunnel to the service inside the minikube VM (you can see that it contains service URL with NodePort which is supposed to be working):

$ minikube service echo

|-----------|------|-------------|---------------------------|
| NAMESPACE | NAME | TARGET PORT |            URL            |
|-----------|------|-------------|---------------------------|
| default   | echo |        8080 | http://192.168.49.2:32034 |
|-----------|------|-------------|---------------------------|
* Starting tunnel for service echo.
|-----------|------|-------------|------------------------|
| NAMESPACE | NAME | TARGET PORT |          URL           |
|-----------|------|-------------|------------------------|
| default   | echo |             | http://127.0.0.1:61991 |
|-----------|------|-------------|------------------------|
* Opening service default/echo in default browser...
! Because you are using a Docker driver on windows, the terminal needs to be open to run it

And now it's available on host machine:

$ curl http://127.0.0.1:61991/

StatusCode        : 200
StatusDescription : OK

Adding ingress

Moving forward and adding ingress.

$ minikube addons enable ingress
$ kubectl get svc -A

NAMESPACE       NAME                                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
default         echo                                 NodePort    10.111.57.237   <none>        8080:32034/TCP               25m
ingress-nginx   ingress-nginx-controller             NodePort    10.104.52.175   <none>        80:31041/TCP,443:31275/TCP   2m12s

Trying to get any response from ingress by hitting minikube_IP:NodePort with no luck:

$ curl 192.168.49.2:31041

curl : Unable to connect to the remote server
At line:1 char:1
+ curl 192.168.49.2:31041

Trying to create a tunnel with minikube service command:

$ minikube service ingress-nginx-controller -n ingress-nginx

|---------------|--------------------------|-------------|---------------------------|
|   NAMESPACE   |           NAME           | TARGET PORT |            URL            |
|---------------|--------------------------|-------------|---------------------------|
| ingress-nginx | ingress-nginx-controller | http/80     | http://192.168.49.2:31041 |
|               |                          | https/443   | http://192.168.49.2:31275 |
|---------------|--------------------------|-------------|---------------------------|
* Starting tunnel for service ingress-nginx-controller.
|---------------|--------------------------|-------------|------------------------|
|   NAMESPACE   |           NAME           | TARGET PORT |          URL           |
|---------------|--------------------------|-------------|------------------------|
| ingress-nginx | ingress-nginx-controller |             | http://127.0.0.1:62234 |
|               |                          |             | http://127.0.0.1:62235 |
|---------------|--------------------------|-------------|------------------------|
* Opening service ingress-nginx/ingress-nginx-controller in default browser...
* Opening service ingress-nginx/ingress-nginx-controller in default browser...
! Because you are using a Docker driver on windows, the terminal needs to be open to run it.

And getting 404 from ingress-nginx which means we can send requests to ingress:

$ curl http://127.0.0.1:62234

curl : 404 Not Found
nginx
At line:1 char:1
+ curl http://127.0.0.1:62234

Solutions

Above I explained what happens. Here are three solutions how to get it work:

1. Use another minikube driver (e.g. virtualbox. I used hyperv since my laptop has windows 10 pro)

minikube ip will return "normal" IP address of virtual machine and all network functionality will work just fine. You will need to add this IP address into /etc/hosts for domain used in ingress rule

Note! Even though localhost was shown in kubectl get ing ingress output in ADDRESS.

2. Use built-in kubernetes feature in Docker desktop for Windows.

You will need to manually install ingress-nginx and change ingress-nginx-controller service type from NodePort to LoadBalancer so it will be available on localhost and will be working. Please find my another answer about Docker desktop for Windows

3. (testing only) - use port-forward

It's almost exactly the same idea as minikube service command. But with more control. You will open a tunnel from host VM port 80 to ingress-nginx-controller service (eventually pod) on port 80 as well. /etc/hosts should contain 127.0.0.1 test.domain entity.

$ kubectl port-forward service/ingress-nginx-controller -n ingress-nginx 80:80

Forwarding from 127.0.0.1:80 -> 80
Forwarding from [::1]:80 -> 80

And testing it works:

$ curl test.domain

StatusCode        : 200
StatusDescription : OK

Update for kubernetes in docker desktop on windows and ingress:

On modern ingress-nginx versions .spec.ingressClassName should be added to ingress rules. See last updates, so ingress rule should look like:

apiVersion: networking.k8s.io/v1
kind: Ingress
...
spec:
  ingressClassName: nginx # can be checked by kubectl get ingressclass
  rules:
  - host: myapp.com
    http:
      ...