Flink 2.x 源码编译指南

编译 Apache Flink 2.x 版本需要准备合适的开发环境,并遵循官方推荐的构建流程。以下是详细步骤:


环境准备

  1. JDK 版本
    JDK 11,确保环境变量 JAVA_HOME 配置正确。

  2. Maven 工具
     Maven 3.8.2


源码获取

从官方仓库克隆源码并切换到目标分支:

git clone https://github.com/apache/flink.git
cd flink
git checkout master


编译命令

  1. 基础编译
    执行以下命令编译全部模块并跳过测试:

    ./mvnw clean package -DskipTests -Djdk11 -Pjava11-target



编译产物

编译成功后,主要产出位于:

  • 二进制包flink-dist/target/flink-2.x-bin.tar.gz

编译结果

[INFO] Reactor Summary for Flink : 2.2-SNAPSHOT:
[INFO]
[INFO] Flink : ............................................ SUCCESS [ 12.027 s]
[INFO] Flink : Annotations ................................ SUCCESS [  3.288 s]
[INFO] Flink : Architecture Tests ......................... SUCCESS [  2.535 s]
[INFO] Flink : Architecture Tests : Base .................. SUCCESS [  6.734 s]
[INFO] Flink : Test utils : ............................... SUCCESS [  2.234 s]
[INFO] Flink : Test utils : Junit ......................... SUCCESS [ 12.391 s]
[INFO] Flink : Metrics : .................................. SUCCESS [  1.900 s]
[INFO] Flink : Metrics : Core ............................. SUCCESS [  4.881 s]
[INFO] Flink : Core API ................................... SUCCESS [ 11.581 s]
[INFO] Flink : Architecture Tests : Test .................. SUCCESS [  2.681 s]
[INFO] Flink : Test utils : MigrationTestUtils ............ SUCCESS [  2.618 s]
[INFO] Flink : Core ....................................... SUCCESS [ 56.611 s]
[INFO] Flink : DataStream : API ........................... SUCCESS [  1.861 s]
[INFO] Flink : RPC : ...................................... SUCCESS [  0.714 s]
[INFO] Flink : RPC : Core ................................. SUCCESS [  1.701 s]
[INFO] Flink : RPC : Akka ................................. SUCCESS [  7.268 s]
[INFO] Flink : RPC : Akka-Loader .......................... SUCCESS [  3.083 s]
[INFO] Flink : Test utils : Connector ..................... SUCCESS [  1.400 s]
[INFO] Flink : Queryable state : .......................... SUCCESS [  0.655 s]
[INFO] Flink : Queryable state : Client Java .............. SUCCESS [  2.365 s]
[INFO] Flink : FileSystems : .............................. SUCCESS [  0.742 s]
[INFO] Flink : FileSystems : Hadoop FS .................... SUCCESS [  3.967 s]
[INFO] Flink : Connectors : ............................... SUCCESS [  0.943 s]
[INFO] Flink : Connectors : Datagen ....................... SUCCESS [  1.455 s]
[INFO] Flink : Runtime .................................... SUCCESS [02:07 min]
[INFO] Flink : Table : .................................... SUCCESS [  0.783 s]
[INFO] Flink : Table : Common ............................. SUCCESS [ 15.889 s]
[INFO] Flink : Table : API Java ........................... SUCCESS [  7.880 s]
[INFO] Flink : Connectors : File Sink Common .............. SUCCESS [  1.773 s]
[INFO] Flink : Streaming Java ............................. SUCCESS [ 50.201 s]
[INFO] Flink : Table : API bridge base .................... SUCCESS [  6.748 s]
[INFO] Flink : Table : API Java bridge .................... SUCCESS [ 10.047 s]
[INFO] Flink : Table : Code Splitter ...................... SUCCESS [ 10.446 s]
[INFO] Flink : DataStream : Impl .......................... SUCCESS [  3.886 s]
[INFO] Flink : Clients : Test Utils ....................... SUCCESS [  1.690 s]
[INFO] Flink : Clients .................................... SUCCESS [ 10.317 s]
[INFO] Flink : State backends : ........................... SUCCESS [  0.660 s]
[INFO] Flink : State backends : RocksDB ................... SUCCESS [  5.193 s]
[INFO] Flink : State backends : Common .................... SUCCESS [  1.852 s]
[INFO] Flink : DSTL ....................................... SUCCESS [  0.567 s]
[INFO] Flink : DSTL : DFS ................................. SUCCESS [  2.810 s]
[INFO] Flink : State backends : Changelog ................. SUCCESS [  3.431 s]
[INFO] Flink : Test utils : Utils ......................... SUCCESS [  4.216 s]
[INFO] Flink : Libraries : ................................ SUCCESS [  0.727 s]
[INFO] Flink : Libraries : CEP ............................ SUCCESS [  6.733 s]
[INFO] Flink : Table : Runtime ............................ SUCCESS [ 18.600 s]
[INFO] Flink : Table : API Scala .......................... SUCCESS [ 35.060 s]
[INFO] Flink : Table : SQL Parser ......................... SUCCESS [  7.613 s]
[INFO] Flink : Table : Calcite Bridge ..................... SUCCESS [  1.484 s]
[INFO] Flink : Table : API Scala bridge ................... SUCCESS [ 12.278 s]
[INFO] Flink : Test utils : Connectors .................... SUCCESS [  4.004 s]
[INFO] Flink : Connectors : Base .......................... SUCCESS [  5.084 s]
[INFO] Flink : Connectors : Files ......................... SUCCESS [  7.214 s]
[INFO] Flink : Table : Planner ............................ SUCCESS [02:48 min]
[INFO] Flink : Formats : .................................. SUCCESS [  0.561 s]
[INFO] Flink : Format : Common ............................ SUCCESS [  0.882 s]
[INFO] Flink : Formats : Json ............................. SUCCESS [  5.428 s]
[INFO] Flink : Table : SQL Gateway : API .................. SUCCESS [  2.396 s]
[INFO] Flink : Formats : Csv .............................. SUCCESS [  4.655 s]
[INFO] Flink : Test utils : Table Filesystem .............. SUCCESS [  4.365 s]
[INFO] Flink : Table : SQL Gateway ........................ SUCCESS [  7.405 s]
[INFO] Flink : Formats : Avro ............................. SUCCESS [  7.911 s]
[INFO] Flink : Formats : SQL Avro ......................... SUCCESS [  3.675 s]
[INFO] Flink : Formats : Parquet .......................... SUCCESS [ 19.653 s]
[INFO] Flink : Formats : SQL Parquet ...................... SUCCESS [  1.957 s]
[INFO] Flink : Formats : Orc .............................. SUCCESS [  6.253 s]
[INFO] Flink : Formats : SQL Orc .......................... SUCCESS [  0.908 s]
[INFO] Flink : Python ..................................... SUCCESS [ 21.152 s]
[INFO] Flink : Table : SQL Client ......................... SUCCESS [  5.685 s]
[INFO] Flink : Models : ................................... SUCCESS [  0.578 s]
[INFO] Flink : Models : OpenAI ............................ SUCCESS [  6.635 s]
[INFO] Flink : Architecture Tests : Production ............ SUCCESS [  3.053 s]
[INFO] Flink : FileSystems : Hadoop FS shaded ............. SUCCESS [  3.018 s]
[INFO] Flink : FileSystems : S3 FS Base ................... SUCCESS [  2.164 s]
[INFO] Flink : FileSystems : S3 FS Hadoop ................. SUCCESS [ 16.757 s]
[INFO] Flink : FileSystems : S3 FS Presto ................. SUCCESS [ 40.789 s]
[INFO] Flink : FileSystems : OSS FS ....................... SUCCESS [  5.267 s]
[INFO] Flink : FileSystems : Azure FS Hadoop .............. SUCCESS [  6.641 s]
[INFO] Flink : FileSystems : Google Storage FS Hadoop ..... SUCCESS [ 10.977 s]
[INFO] Flink : Runtime web ................................ SUCCESS [02:49 min]
[INFO] Flink : Connectors : Hadoop compatibility .......... SUCCESS [  5.272 s]
[INFO] Flink : Connectors : Datagen Tests ................. SUCCESS [  3.367 s]
[INFO] Flink : Formats : Avro confluent registry .......... SUCCESS [  2.773 s]
[INFO] Flink : Formats : Sequence file .................... SUCCESS [  4.170 s]
[INFO] Flink : Formats : Compress ......................... SUCCESS [  4.161 s]
[INFO] Flink : Formats : Protobuf ......................... SUCCESS [ 12.152 s]
[INFO] Flink : Formats : Orc nohive ....................... SUCCESS [  4.315 s]
[INFO] Flink : Formats : Hadoop bulk ...................... SUCCESS [  5.025 s]
[INFO] Flink : Formats : SQL Csv .......................... SUCCESS [  0.643 s]
[INFO] Flink : Formats : SQL Json ......................... SUCCESS [  0.583 s]
[INFO] Flink : Formats : SQL Avro Confluent Registry ...... SUCCESS [  4.510 s]
[INFO] Flink : Formats : SQL Protobuf ..................... SUCCESS [  1.053 s]
[INFO] Flink : State backends : ForSt ..................... SUCCESS [  6.694 s]
[INFO] Flink : Examples : ................................. SUCCESS [  0.872 s]
[INFO] Flink : Examples : Streaming ....................... SUCCESS [  5.079 s]
[INFO] Flink : Examples : Table ........................... SUCCESS [  8.738 s]
[INFO] Flink : Examples : Build Helper : .................. SUCCESS [  0.721 s]
[INFO] Flink : Examples : Build Helper : Streaming State machine SUCCESS [  0.573 s]
[INFO] Flink : Container .................................. SUCCESS [  1.842 s]
[INFO] Flink : Queryable state : Runtime .................. SUCCESS [  3.648 s]
[INFO] Flink : Table : Planner Loader Bundle .............. SUCCESS [  3.179 s]
[INFO] Flink : Table : Planner Loader ..................... SUCCESS [  5.570 s]
[INFO] Flink : Table : Test Utils ......................... SUCCESS [  3.618 s]
[INFO] Flink : Tests ...................................... SUCCESS [ 30.245 s]
[INFO] Flink : Dist-Scala ................................. SUCCESS [  1.401 s]
[INFO] Flink : Kubernetes ................................. SUCCESS [ 13.690 s]
[INFO] Flink : Yarn ....................................... SUCCESS [  7.017 s]
[INFO] Flink : Table : API Java Uber ...................... SUCCESS [  4.078 s]
[INFO] Flink : External resources : ....................... SUCCESS [  0.671 s]
[INFO] Flink : External resources : GPU ................... SUCCESS [  1.253 s]
[INFO] Flink : Metrics : Dropwizard ....................... SUCCESS [  1.413 s]
[INFO] Flink : Metrics : Graphite ......................... SUCCESS [  0.966 s]
[INFO] Flink : Metrics : InfluxDB ......................... SUCCESS [  2.022 s]
[INFO] Flink : Metrics : Prometheus ....................... SUCCESS [  2.067 s]
[INFO] Flink : Metrics : JMX .............................. SUCCESS [  2.932 s]
[INFO] Flink : Metrics : StatsD ........................... SUCCESS [  1.179 s]
[INFO] Flink : Metrics : Datadog .......................... SUCCESS [  1.176 s]
[INFO] Flink : Metrics : Slf4j ............................ SUCCESS [  1.113 s]
[INFO] Flink : Metrics : OpenTelemetry .................... SUCCESS [  3.570 s]
[INFO] Flink : Libraries : State processor API ............ SUCCESS [  6.553 s]
[INFO] Flink : Dist ....................................... SUCCESS [ 21.863 s]
[INFO] Flink : Yarn Tests ................................. SUCCESS [  1.832 s]
[INFO] Flink : E2E Tests : ................................ SUCCESS [  0.138 s]
[INFO] Flink : E2E Tests : CLI ............................ SUCCESS [  0.336 s]
[INFO] Flink : E2E Tests : Parent Child classloading program SUCCESS [  0.307 s]
[INFO] Flink : E2E Tests : Parent Child classloading lib-package SUCCESS [  0.152 s]
[INFO] Flink : E2E Tests : Datastream allround ............ SUCCESS [  1.677 s]
[INFO] Flink : E2E Tests : Batch SQL ...................... SUCCESS [  0.661 s]
[INFO] Flink : E2E Tests : Stream SQL ..................... SUCCESS [  0.418 s]
[INFO] Flink : E2E Tests : Distributed cache via blob ..... SUCCESS [  0.374 s]
[INFO] Flink : E2E Tests : Stream stateful job upgrade .... SUCCESS [  0.929 s]
[INFO] Flink : E2E Tests : Queryable state ................ SUCCESS [  1.838 s]
[INFO] Flink : E2E Tests : Local recovery and allocation .. SUCCESS [  0.369 s]
[INFO] Flink : Quickstart : ............................... SUCCESS [  1.816 s]
[INFO] Flink : Quickstart : Java .......................... SUCCESS [  1.042 s]
[INFO] Flink : E2E Tests : Quickstart : Dummy-Dependency .. SUCCESS [  0.153 s]
[INFO] Flink : E2E Tests : Quickstart ..................... SUCCESS [  0.307 s]
[INFO] Flink : E2E Tests : Confluent schema registry ...... SUCCESS [  2.820 s]
[INFO] Flink : E2E Tests : Stream state TTL ............... SUCCESS [  7.336 s]
[INFO] ------------------------------------------------------------------------

访问 http://localhost:8081 确认 Web UI 正常运行。


注意事项

  • 首次编译可能耗时较长(约 30 分钟至数小时)。
  • 具体版本可能有差异,请参考官方文档中的 README.md

更多推荐