Tuesday, March 10, 2015

Custom Java UDA implementation in Apache Drill 0.7

     28 comments


CUSTOM JAVA UDA IMPLEMENTATION IN APACHE DRILL


PREREQUISITE
The following software is to be installed.
  • Ubuntu 13.10
  • Java 1.7
  • Maven 3.0.4
  • Git
  • Eclipse Kepler with Maven Plugins
  • Apache Drill 0.7.0 (Installed in an embedded mode)
DOWNLOAD AND COMPILE SOURCE CODE OF DRILL
Download and build the source code of Apache Drill 0.7.0 and 0.8.0 from git.
  • Apache Drill 0.8.0
mvn clean install
  • Apache Drill 0.7.0
mvn clean install
SET UP DRILL DEVELOPMENT ENVIRONMENT IN ECLIPSE
  • Open Eclipse and Right click on Package Explorer and select Import.
  • Navigate to Maven and select Existing Maven Projects to Import the compiled source code of Drill 0.7.0
  • In case of any error in project, fix the errors.
  • Create a new Maven project (e.g., groupID - com.udf, artifactID - udf, version - 0.0.1-SNAPSHOT and package - jar)
  • Add the following lines in pom.xml file.
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.apache.drill.exec</groupId>
<artifactId>drill-java-exec</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
</dependencyManagement>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<version>2.4</version>
<executions>
<execution>
<id>attach-sources</id>
<!-- <phase>verify</phase> -->
<goals>
<goal>jar</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
  • Create a Java class and develop your UDA
CLEAN, COMPILE AND BUILD JAR WITH DEPENDENCIES
  • To Clean - Right click on your Maven project and select Run as -> Maven Clean
  • Goto Project in Eclipse and select clean
  • To compile and build Jar - Right click on your Maven project and select Run as -> Maven Install.
  • Now the jar file would be available in target folder under your project.
DEPLOY THE CUSTOM JAR IN APACHE DRILL
  • Stop the drillbit service
  • Copy the jar files (yourprojectname-0.0.1-SNAPSHOT-sources.jar and yourprojectname-0.0.1-SNAPSHOT.jar) from the target directory of your project and paste to $DRILL_HOME/jars/classb/ location.
  • Start the drillbit service.
  • Make sure that your jar is loaded successfully into the Drill by verifying the drill log files.
TEST YOUR CUSTOM UDA FUNCTION IN WEB UI OF THE APACHE DRILL

Wednesday, February 11, 2015