Setting up my development environment with Docker compose is really simple. However I am trying to setup the same project with Kubernetes. To make things simple I have broke down the setup to a single microservice with RabbitMQ. Below is deployment yaml file where I have attempted to declare all the things I believe I need and it does not work.
Deployment file:
apiVersion: v1
kind: Service
metadata:
name: rabbitmq
spec:
ports:
- port: 5672
targetPort: 5672
selector:
app: rabbitmq
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: rabbitmq
spec:
replicas: 1
selector:
matchLabels:
app: rabbitmq
template:
metadata:
labels:
app: rabbitmq
spec:
containers:
- name: rabbitmq
image: rabbitmq:3-management
ports:
- containerPort: 5672
- containerPort: 15672
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: notification
name: notification
spec:
replicas: 1
selector:
matchLabels:
app: notification
template:
metadata:
labels:
app: notification
spec:
containers:
- image: aaronbalthaser/notification:v0.5
name: notification
env:
- name: RABBITMQ_URI
value: amqp://rabbitmq:5672
- name: MAILGUN_API_KEY
value: "ea0d74d8da769f9c405b02c85fb3dd37-7113c52e-98a7f729"
- name: MAILGUN_DOMAIN
value: "sandbox37f3b689954a497e8c19af2282730630.mailgun"
- name: DOMAIN
value: "http://localhost:4200"
Main TS file:
async function bootstrap() {
const app = await NestFactory.create(NotificationModule);
const configService = app.get(ConfigService);
app.connectMicroservice({
transport: Transport.RMQ,
options: {
urls: [configService.getOrThrow("RABBITMQ_URI")],
queue: NOTIFICATION_SERVICE,
},
});
await app.startAllMicroservices();
}
bootstrap();
Dockerfile:
###################
# DEVELOPMENT
###################
FROM node:alpine As development
WORKDIR /usr/src/app
COPY package.json ./
COPY pnpm-lock.yaml ./
COPY tsconfig.json ./
COPY nest-cli.json ./
COPY ca.pem ./
RUN npm install -g pnpm
COPY apps/notification apps/notification
COPY libs libs
RUN pnpm install
RUN pnpm run build notification
###################
# PRODUCTION
###################
FROM node:alpine as production
ARG NODE_ENV=production
ENV NODE_ENV=${NODE_ENV}
WORKDIR /usr/src/app
COPY package.json ./
COPY pnpm-lock.yaml ./
COPY ca.pem ./
RUN npm install -g pnpm
RUN pnpm install --prod
COPY --from=development /usr/src/app/dist ./dist
CMD ["node", "dist/apps/notification/main"]
When attempting to run the file I get the following output. The console log is me outputting those host address I am passing in from the Config service.
For context below is the output I see when I use Docker compose:
Setting up my development environment with Docker compose is really simple. However I am trying to setup the same project with Kubernetes. To make things simple I have broke down the setup to a single microservice with RabbitMQ. Below is deployment yaml file where I have attempted to declare all the things I believe I need and it does not work.
Deployment file:
apiVersion: v1
kind: Service
metadata:
name: rabbitmq
spec:
ports:
- port: 5672
targetPort: 5672
selector:
app: rabbitmq
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: rabbitmq
spec:
replicas: 1
selector:
matchLabels:
app: rabbitmq
template:
metadata:
labels:
app: rabbitmq
spec:
containers:
- name: rabbitmq
image: rabbitmq:3-management
ports:
- containerPort: 5672
- containerPort: 15672
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: notification
name: notification
spec:
replicas: 1
selector:
matchLabels:
app: notification
template:
metadata:
labels:
app: notification
spec:
containers:
- image: aaronbalthaser/notification:v0.5
name: notification
env:
- name: RABBITMQ_URI
value: amqp://rabbitmq:5672
- name: MAILGUN_API_KEY
value: "ea0d74d8da769f9c405b02c85fb3dd37-7113c52e-98a7f729"
- name: MAILGUN_DOMAIN
value: "sandbox37f3b689954a497e8c19af2282730630.mailgun."
- name: DOMAIN
value: "http://localhost:4200"
Main TS file:
async function bootstrap() {
const app = await NestFactory.create(NotificationModule);
const configService = app.get(ConfigService);
app.connectMicroservice({
transport: Transport.RMQ,
options: {
urls: [configService.getOrThrow("RABBITMQ_URI")],
queue: NOTIFICATION_SERVICE,
},
});
await app.startAllMicroservices();
}
bootstrap();
Dockerfile:
###################
# DEVELOPMENT
###################
FROM node:alpine As development
WORKDIR /usr/src/app
COPY package.json ./
COPY pnpm-lock.yaml ./
COPY tsconfig.json ./
COPY nest-cli.json ./
COPY ca.pem ./
RUN npm install -g pnpm
COPY apps/notification apps/notification
COPY libs libs
RUN pnpm install
RUN pnpm run build notification
###################
# PRODUCTION
###################
FROM node:alpine as production
ARG NODE_ENV=production
ENV NODE_ENV=${NODE_ENV}
WORKDIR /usr/src/app
COPY package.json ./
COPY pnpm-lock.yaml ./
COPY ca.pem ./
RUN npm install -g pnpm
RUN pnpm install --prod
COPY --from=development /usr/src/app/dist ./dist
CMD ["node", "dist/apps/notification/main"]
When attempting to run the file I get the following output. The console log is me outputting those host address I am passing in from the Config service.
For context below is the output I see when I use Docker compose:
Share Improve this question asked Feb 17 at 21:30 Aaron BalthaserAaron Balthaser 2,6443 gold badges38 silver badges62 bronze badges1 Answer
Reset to default 0Place both the RabbitMQ service/deployment and the notification service/deployment in the same namespace. This is the standard and simplest approach for services that need to communicate directly.
(If Namespaces Must Differ): Use the Fully Qualified Domain Name (FQDN) of the RabbitMQ service in your notification service's connection string. The FQDN follows the pattern: <service-name>.<namespace>.svc.cluster.local. This tells Kubernetes exactly where to find the service.
Debugging
Determine which namespace your resources (Service, Deployments) are in. The simplest approach is to keep everything in the default namespace unless you have a specific reason to use separate namespaces.
Check the status of your Pods, Services, and Deployments. Look for any errors or warnings.
Check the logs of both the notification service container and the RabbitMQ container. Look for error messages related to connection failures or DNS resolution.
Use the describe pod command to get detailed information about a specific Pod, including events that might indicate problems.
Examine Kubernetes events for any cluster-level issues that might be affecting your deployment.
Enter the notification service container's shell (using exec). From within the container, use network tools (like nslookup or ping) to try to resolve the "rabbitmq" hostname. This directly tests DNS resolution from the Pod's perspective.
You should be able to pinpoint the cause of the DNS resolution failure and get your Kubernetes deployment running. The namespace check and readiness probe are the most likely solutions.