We have a bit of an interesting situation here.

We are using an API with OAuth enabled via Code Authorization with PKCE standard flows on Keycloak to get access_token values for passing in Authorization: Bearer ... headers.

We can easily create users, require them to provide OTP, change password, etc. And if we don't have OTP required, then the standard mechanisms of authenticating programmatically and parsing the callback data works fine.

However, when we try and do authentication with PKCE and TOTP, we get an extra issue - a 200 response with a different page with the OTP prompt on it - which breaks the standard flow. Attempting to parse it's action= URL and then pass in the OTP on the form encoded data returns a "Invalid Password" authentication error message, which means that something is weird.

We're using a basic default browser flow for Keycloak, but are attempting to programmatically in Python to do everything without python-keycloak (which cannot do PKCE).

Effectively, this is the process we're trying to use (derived from here):

import base64
import hashlib
import html
import json
import os
import re
import urllib.parse
import requests

from string import ascii_letters, digits
import secrets

def generate_pkce():
    verifier = "".join(secrets.choice(ascii_letters+digits) for i in range(64))
    challenge = base64.urlsafe_b64encode(
        hashlib.sha256(verifier.encode('utf-8')).digest()
    ).decode('utf-8').replace('=', '')
    return {"verifier": verifier, "challenge": challenge}

session = requests.Session()

server_host = "kc.example"
realm = "test"
client_id = "api"
redirect_uri = "http://localhost/callback"
https = True
server_port = 443

base_uri = (f"{'https' if https else 'http'}://{server_host}"
            f"{':' + str(server_port) if server_port not in [80, 443] else ''}")

pkce = generate_pkce()

username = "[email protected]"
password = "NotARealPassword"

req = session.get(f"{base_uri}/realms/{realm}/.well-known/openid-configuration")

assert req.status_code == 200
openid_config = req.json()

state = "".join(secrets.choice(ascii_letters + digits) for i in range(16))

auth_url = openid_config['authorization_endpoint']
params = {
    "response_type": "code",
    "client_id": client_id,
    "redirect_uri": redirect_uri,
    "scope": "openid",
    "state": state,
    "code_challenge_method": "S256",
    "code_challenge": pkce['challenge']
}
resp = session.get(url=auth_url,params=params,allow_redirects=False)

page = resp.text
form_action = html.unescape(
    re.search(r'<form\s+.*?\s+action="(.*?)"', page, re.DOTALL).group(1)
)

## AT THIS POINT: We will have the URL that will be used to authenticate to with the form
## Call this "Relevant Code Point 1"

resp = session.post(
    url=form_action,
    data={
        "username": username,
        "password": password
    },
    allow_redirects=False
)

assert resp.status_code in [301, 302]

# Parse callback data to extract code
redirect = resp.headers["Location"]

query = urllib.parse.urlparse(redirect).query
redirect_params = urllib.parse.parse_qs(query)

auth_code = redirect_params['code'][0]

...

When we get to "Relevant Code Point 1" (note it in the comments), when we have no OTP needed, this will return an actual 301 or 302 redirect that goes back to our redirect_uri - however as there's no server listening, we simply parse the data from the parameters and extract what we need from it, namely the authorization code.

Now, when we have OTP enabled, however, the assertion line fails because the response is 200, with a form requesting the email address and the OTP code. This poses an extra problem - because we now have an extra step we have to do and the standard flow doesn't work.

I looked at various documentation and it SUGGESTS that adding the key-value"totp": "123456" (where 123456 is the actual 2FA code) to the payload being sent to the form action would pass it, however it doesn't seem to work.

Can anyone provide some guidance as to what we need to do here to make OTP work in the flow?