DolphinScheduler 启动flink任务报错 Could not start rest endpoint on any port in port range 8081
·
写了个flink消费kafka的例子,想扔到海豚启动的,但是启动的时候报错Could not start rest endpoint on any port in port range 8081(在flinkUI提交任务没问题,直接消费到消息了),完整的报错信息是这样的:
The program finished with the following exception:
org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Could not deploy Yarn job cluster.
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372)
at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:105)
at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:851)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:245)
at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1095)
at org.apache.flink.client.cli.CliFrontend.lambda$mainInternal$9(CliFrontend.java:1189)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:688)
at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at org.apache.flink.client.cli.CliFrontend.mainInternal(CliFrontend.java:1189)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1157)
Caused by: org.apache.flink.client.deployment.ClusterDeploymentException: Could not deploy Yarn job cluster.
at org.apache.flink.yarn.YarnClusterDescriptor.deployJobCluster(YarnClusterDescriptor.java:507)
at org.apache.flink.client.deployment.executors.AbstractJobClusterExecutor.execute(AbstractJobClusterExecutor.java:83)
at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:2238)
at org.apache.flink.client.program.StreamContextEnvironment.executeAsync(StreamContextEnvironment.java:189)
at org.apache.flink.client.program.StreamContextEnvironment.execute(StreamContextEnvironment.java:118)
at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:2099)
at com.example.FlinkKafkaConsumerExample.main(FlinkKafkaConsumerExample.java:65)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
... 12 more
Caused by: org.apache.flink.yarn.YarnClusterDescriptor$YarnDeploymentException: The YARN application unexpectedly switched to state FAILED during deployment.
Diagnostics from YARN: Application application_1768270035395_0001 failed 1 times (global limit =2; local limit is =1) due to AM Container for appattempt_1768270035395_0001_000001 exited with exitCode: 1
Failing this attempt.Diagnostics: [2026-01-13 10:13:42.136]Exception from container-launch.
Container id: container_1768270035395_0001_01_000001
Exit code: 1
一开始我以为是端口问题,然后在flink的配置文件flink-conf.yaml修改了端口信息:rest.port: 8082,后面继续在海豚启动任务,发现还是报端口连不上的错,报错端口从8081变成了8082;
然后我继续百度,终于发现了解决方法:就是在 Dolphin Scheduler 中指定 Flink 作业的 REST 端口
-Drest.port=8082

最后成功在flinkUI看到任务启动成功,并消费成功了:

更多推荐
所有评论(0)