Apache Kafka is today’s topic. I am getting started developing realtime data processing platform which uses Apache Storm. Storm receives data from Kafka as messages. So I’d like to learn the architecture of Kafka and how to build it.

You can learn here very detail. So I am going to introduce how to build Kafka project on MacOSX and IntelliJ IDEA.

Gradle

Kafka is the project made by Gradle. Of course you can download and compile gradle itself here. But if your machine is MacOSX, you can install with Homebrew more easily.

$ brew install gradle

That’s all!

Build

You can checkin Kafka source code repository.

$ git clone http://git-wip-us.apache.org/repos/asf/kafka.git kafka

And build whole project with gradle.

$ cd kafka
$ gradle

OK. You can get all built files under gradle/wrapper/*. In this directory, there is a class which generate IntelliJ setting. You can use it with this command.

$ ./gradlew idea

If there are no problem through this process, all you have to do is finished except one thing. Just open it with IntelliJ IDEA.

It very easy, isn’t it? All process is written in here. Of course you can also use Eclipse.

Thank you!