I can not seem to find any documentation on how to make Selenium open the browser in incognito mode.

Do I have to setup a custom profile in the browser or?

First of all, since selenium by default starts up a browser with a clean, brand-new profile, you are actually already browsing privately. Referring to:

• Python - Start firefox with Selenium in private mode
• How might I simulate a private browsing experience in Watir? (Selenium)

But you can strictly enforce/turn on incognito/private mode anyway.

For chrome pass --incognito command-line argument:

--incognito Causes the browser to launch directly in incognito mode.

from selenium import webdriver

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--incognito")

driver = webdriver.Chrome(chrome_options=chrome_options)
driver.get('https://google.com')

FYI, here is what it would open up:

For firefox, set browser.privatebrowsing.autostart to True:

from selenium import webdriver

firefox_profile = webdriver.FirefoxProfile()
firefox_profile.set_preference("browser.privatebrowsing.autostart", True)

driver = webdriver.Firefox(firefox_profile=firefox_profile)

FYI, this corresponds to the following checkbox in settings: