I have the following structure of Kubernetes cluster.
There is Gateway: Spring app with net.bretti.openapi-route-definition-locator to dispatch requests to several microservices.
There are business applications (microservices in Spring and Quarkus). One of Spring apps uses Oidc Keycloak authentication. Keycloak server is running in the same cluster.
Each of apps (and the Gateway) have replica count 2 or 3.
The problem is in Spring OAuth2 app (let's name it aaa). The gateway has ingress rule to redirect requests with /aaa prefix to the aaa application. The Service object of this application has property: sessionAffinity: ClientIP
in its spec.
As I understand, after a user request from the browser goes to the gateway, it redirects to the Service, it then redirects to one of working pods. The pod checks user - if not authenticated, Spring redirects to aaa/oauth2/authorization, then to keycloak (keycloak/realms/my-realm) with redirect-uri query param (aaa/login/oauth2/code).
The callback from Keycloak through the Gateway returns back to Service and should go to the same Pod. But sometimes this does not occur. The redirect arrives to the second pod. And I got "The page isn't redirecting properly" and see keycloak?error page.
Here are some related configs:
application.yaml in Spring aaa application
spring:
security:
oauth2:
client:
provider:
keycloak:
user-name-attribute: preferred_username
issuer-uri: https://my-keycloak/realms/my-realm
registration:
keycloak:
client-id: ${OIDC_CLIENT_ID}
client-secret: ${OIDC_CLIENT_SECRET}
scope: openid
authorization-grant-type: authorization_code
redirect-uri: "{baseUrl}/aaa/login/oauth2/code/keycloak"
client-authentication-method: client_secret_post
SecurityConfig.java
@Bean
public SecurityFilterChain filterChain(HttpSecurity http) throws Exception {
http
.authorizeHttpRequests(authorize -> authorize
.requestMatchers("/actuator/*").permitAll()
.requestMatchers("/openapi.yaml").permitAll()
.anyRequest().authenticated()
)
.oauth2Login(oauth2 -> oauth2
.authorizationEndpoint(authorization -> authorization
.baseUri("/aaa/oauth2/authorization")
)
.loginPage("/aaa/oauth2/authorization/keycloak")
.redirectionEndpoint(redirection -> redirection
.baseUri("/aaa/login/oauth2/code/*")
)
);
return http.build();
}
The flow works correctly if I set replicas: 1.
What may be the problem and how to diagnose and fix it?