最新消息:雨落星辰是一个专注网站SEO优化、网站SEO诊断、搜索引擎研究、网络营销推广、网站策划运营及站长类的自媒体原创博客

google cloud platform - java expansion service (grpc server) is UNAVAILABLE when ReadFromPubSub in beam application tries to acc

programmeradmin3浏览0评论

My beam python on flinkrunner(PortableRunner) has the following code.

from apache_beam.io.external.gcp.pubsub import ReadFromPubSub
from dependency_injector import providers
...

    pubsub_reader = providers.Singleton(
        ReadFromPubSub,
        subscription=config.gcp_pubsub_subscription_info,
        expansion_service="beam-expansion-service.flink:8097",
    )

And I ran the beam application on K8S with the java expansion service like the below.

(in Dockerfile)
FROM --platform=linux/amd64 openjdk:11
WORKDIR /app
RUN curl -o beam-sdks-java-io-google-cloud-platform-expansion-service.jar .61.0/beam-sdks-java-io-google-cloud-platform-expansion-service-2.61.0.jar
(in Deployment/Service yaml)
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: beam-expansion-service
spec:
  replicas: 1
  selector:
    matchLabels:
      app: beam-expansion-service
  template:
    metadata:
      labels:
        app: beam-expansion-service
    spec:
      containers:
      - name: expansion-service
        image: beam-expansion-service:1
        command: ["java", "-jar", "/app/beam-sdks-java-io-google-cloud-platform-expansion-service.jar", "8097", "--javaClassLookupAllowlistFile=*"]
        ports:
        - containerPort: 8097
---
apiVersion: v1
kind: Service
metadata:
  name: beam-expansion-service
spec:
  clusterIP: None
  selector:
    app: beam-expansion-service
  ports:
    - protocol: TCP
      port: 8097
      targetPort: 8097
---

I use apache_beam.io.external.gcp.pubsub.ReadFromPubSub so that I make my beam application get messages from GCP PubSub on FlinkRunner, not Dataflow runner. And they ("java expansion service pod" and "beam on flinkrunner" pod) are running on the same namespace "flink". And I got an error like the below.

grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.UNAVAILABLE
        details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:10.232.23.14:8097: Endpoint is neither UDS or TCP loopback address."
        debug_error_string = "UNKNOWN:Error received from peer  {created_time:"2025-03-17T16:02:35.189471737+00:00", grpc_status:14, grpc_message:"failed to connect to all addresses; last error: UNKNOWN: ipv4:10.232.23.14:8097: Endpoint is neither UDS or TCP loopback address."}"

Why can't my beam connect service to java expansion service(grpc server)? When I executed "nc -v 10.232.23.14 8097" or "nc -v beam-expansion-service.flink 8097", the address was opened.

Thanks in advance.

与本文相关的文章

发布评论

评论列表(0)

  1. 暂无评论