Sunteți pe pagina 1din 14

Dynamic Code Generation with Java Compiler API in Java 6

By Swaminathan Bhaskar
10/10/2009

In one of the previous articles, we were introduced to Scripting in Java 6. It was one way of
dynamically extending the capabilities of an application. Also, there is another way of extending an
application dynamically – by compiling and loading Java classes at runtime.

In Java 6, the javax.tools package exposes the Java compiler as an API. By default, the Java compiler
works with source from input file(s) and generates the corresponding class output file(s). By
implementing interfaces in the javax.tools package, we will be able to work with source from strings in
memory and generate class to byte array in memory.

But, why would we need to do this ? Imagine we have an asynchronous channel from where clients
can consume messages. Each client may have a different need and process only a subset of the
messages. The clients effectively need to filter on the content of the messages before processing. If the
filter criteria for all the clients is known and is a small predefined set, then we may be able to write the
filter classes for the predefined set. But, if the filter criteria various for each client and is subject to
change, it is better to implement the filter criteria as a dynamic extension. If the message rate is small
and the filter criteria is not that complex, then we may be able to leverage the scripting support in Java
6. But, if the message rates are high and the filter criteria complex, then scripting may not be a viable
option for efficiency/performance reasons. It would be more efficient to dynamically compile and
execute the filter criteria.

Before we proceed further, we want to state that in our examples we will be using an array of integers
to represent messages and define an interface for the filter criteria.

The following code listing defines the interface for the filter criteria:

/*
* Name: Bhaskar S
*
* Date: 10/10/2009
*/

package com.polarsparc.jdk.compiler;

public interface Filter {


public boolean filter(int num);
}

The following code listing illustrates an implementation of the filter criteria where the input integer is
an even integer and is greater than 500:

/*
* Name: Bhaskar S
*
* Date: 10/10/2009
*/

package com.polarsparc.jdk.compiler;

public class FilterEvenGt500 implements Filter {


@Override
public boolean filter(int num) {
if (num > 500 && (num % 2) == 0) {
return true;
}
return false;
}
}

The following code listing for ScriptVsCompiled illustrates an example that compares the efficiency of
using scripting versus compiled code:

/*
* Name: Bhaskar S
*
* Date: 10/10/2009
*/

package com.polarsparc.jdk.compiler;

import java.util.*;
import javax.script.*;

public class ScriptVsCompiled {


private static int[] nums;

public static void main(String[] args) {


init();
ScriptClient sc = new ScriptClient();
sc.init();
sc.execute();
CompiledClient cc = new CompiledClient();
cc.init();
cc.execute();
}

public static void init() {


try {
nums = new int[100];
Random rand = new Random();
for (int i = 0; i < 100; ++i) {
nums[i] = rand.nextInt(1000);
}
}
catch (Throwable ex) {
ex.printStackTrace(System.err);
System.exit(1);
}
}

private static class ScriptClient {


private ScriptEngine seng;

void init() {
ScriptEngineManager smgr = new ScriptEngineManager();
seng = smgr.getEngineByExtension("js");
if (seng == null) {
System.err.println("*** Could not find engine - Rhino !!!");
System.exit(1);
}
}

void execute() {
try {
Bindings bnds = seng.getBindings(ScriptContext.ENGINE_SCOPE);
long stm = System.currentTimeMillis();
for (int n : nums) {
bnds.put("num", n);
if ((Boolean)seng.eval("if (num > 500 && (num % 2) == 0) { true; } else
{ false; }")) {
System.out.printf("ScriptClient: %d\n", n);
}
}
long etm = System.currentTimeMillis();
System.out.printf("ScriptCliient execute time: %d ms\n", (etm-stm));
}
catch (Throwable ex) {
ex.printStackTrace(System.err);
System.exit(1);
}
}
}

private static class CompiledClient {


private Filter func;

void init() {
try {
Class<?> clazz = Class.forName("com.polarsparc.jdk.compiler.FilterEvenGt500");
func = (Filter) clazz.newInstance();
}
catch (Throwable ex) {
ex.printStackTrace(System.err);
System.exit(1);
}
}

void execute() {
try {
long stm = System.currentTimeMillis();
for (int n : nums) {
if (func.filter(n)) {
System.out.printf("CompiledClient: %d\n", n);
}
}
long etm = System.currentTimeMillis();
System.out.printf("CompiledClient execute time: %d ms\n", (etm-stm));
}
catch (Throwable ex) {
ex.printStackTrace(System.err);
System.exit(1);
}
}
}
}

Executing the above code, shows that using scripting capability of Java 6 for filtering is orders of
magnitude slower than using compiled code for filtering:

The client using the scripting capability (named ScriptClient) took about 160 ms to complete, while the
client (name CompiledClient) using the compiled filter class FilterEvenGt500 took 1 ms.

It is clear from the above execution that it will be more efficient to dynamically compile and execute
the filter criteria.

Ready for some exciting cool stuff !!! In the following sections we will explore the Java compiler API
exposed through the javax.tools package in Java 6.

The following code listing is a simple Java class called CompileMe that we will compile using the Java
compiler API:
/*
* Name: Bhaskar S
*
* Date: 10/10/2009
*/

package com.polarsparc.jdk.compiler;

public class CompileMe {


public static void main(String[] args) {
System.out.printf("Compiled with Java 6 JavaCompiler API\n");
}
}

The following code listing for BasicCompile illustrates the use of Java compiler API to compile the
simple Java class called CompileMe:

/*
* Name: Bhaskar S
*
* Date: 10/10/2009
*/

package com.polarsparc.jdk.compiler;

import java.util.*;
import javax.tools.*;
import javax.tools.JavaCompiler.CompilationTask;

public class BasicCompile {


public static void main(String[] args) {
try {
/* 1 */ JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();

/* 2 */ StandardJavaFileManager manager = compiler.getStandardFileManager(null, null, null);

/* 3 */ Iterable<? extends JavaFileObject> units = manager.getJavaFileObjects("src/com/polarsparc/


jdk/compiler/CompileMe.java");

/* 4 */ String[] opts = new String[] { "-d", "classes" };

/* 5 */ CompilationTask task = compiler.getTask(null, manager, null, Arrays.asList(opts), null,


units);
/* 6 */ boolean status = task.call();
if (status) {
System.out.printf("Compilation successful!!!\n");
}
}
catch (Exception ex) {
ex.printStackTrace(System.err);
System.exit(1);
}
}
}

In line /* 1 */, we get the reference to the implementation of the Java platform compiler called
JavaCompiler.

In line /* 2 */, we get the reference to StandardJavaFileManager, which provides an abstraction layer
for performing file operations such as reading an input source and writing compiled class output. Just
as with the javac command, the JavaCompiler also operates on file(s) using StandardJavaFileManager.

In line /* 3 */, we wrap the simple Java source file called CompileMe.java in a JavaFileObject. The
StandardJavaFileManager works on the input and the output files and provides them as objects of type
JavaFileObject.

In line /* 4 */, we are specifying the compiler options to use. In this case, we are specifying the target
directory of compile to be the dircetory “classes”. By default, the target directory is the current
directory.

In line /* 5 */, we get the reference to CompilationTask, which allows us to invoke the process of Java
source compilation. We get an handle to CompilationTask by invoking the getTask() method on the
reference to JavaCompiler from line /* 1 */. In this step, we associate the StandardJavaFileManager,
the compiler options, and the input source file wrapped in JavaFileObject.

In line /* 6 */, we finally invoke the compilation task. If the compilation succeeds, the compiled class
file will be under the directory named “classes”.

Now, open a terminal window and list all the files under the directory “classes”. We see no file called
“CompileMe.class” as illustrated below:

Open another terminal and execute the BasicCompile class as show below:

bswamina@shivam ~/Projects/Java/JavaCompiler $ ./bin/BasicCompile.sh


Compilation successful!!!

This will compile the java source “CompileMe.java” and now we see the class file as shown below:
With this, we have successfully demonstrated the use of the Java compiler API to generate java class
file from a java source file.

In the above example, the java code was sourced from a file. How do we handle the case where the java
code is generated dynamically at runtime as a String. One way would be to save the generated code to a
file and then use the above example to compile. The more interesting case would be to compile the
code directly from the String.

In the above example, we used StandardJavaFileManager to wrap the java source file as an object of
type JavaFileObject. Similarly, we need a class to wrap the generated code from a String into an object
of type JavaFileObject. We can achieve that by extending the concrete class SimpleJavaFileObject
from the javax.tools package.

The following code listing for StringJavaFileObject illustrates our custom JavaFileObject that allows
us to present java code from a String to the JavaCompiler for compilation:

/*
* Name: Bhaskar S
*
* Date: 10/10/2009
*/

package com.polarsparc.jdk.compiler;

import java.net.*;
import javax.tools.*;

/* 1 */ public class StringJavaFileObject extends SimpleJavaFileObject {


private String source;

/* 2 */ public StringJavaFileObject(String name, String source) {


/* 3 */ super(URI.create("string:///" + name.replace('.','/') + Kind.SOURCE.extension),
Kind.SOURCE);
this.source = source;
}

@Override
/* 4 */ public CharSequence getCharContent(boolean ignoreEncodingErrors) {
return this.source;
}
}

In line /* 1 */, we extend from the class SimpleJavaFileObject which is provided by the Java Compiler
API as a simple implementation of JavaFileObject. The constructor for SimpleJavaFileObject is
defined as protected and takes two arguments: an URI to the file it represents and the type of the file
(java source or compiled class) specified as a constant of type Kind.

In line /* 2 */, we define the constructor for our custom JavaFileObject implementation called
StringJavaFileObject. It takes two arguments: the full class name (including package name) for our
generated java source and the java code as a String.

In line /* 3 */, we invoke the constructor for the super class which is SimpleJavaFileObject in this case.
The line URI.create(“string:///” + name.replace('.','/') + Kind.SOURCE.extension) creates a standard
URL path. For example, given the full class name “com.abc.Foo” , this line will create a standard URL
path “com/abc/Foo.java”. The enum Kind (defined in JavaFileObject) identifies the type of the URI. In
our case, the type is java source and hence we use Kind.SOURCE.
In line /* 4 */, we override the method “getCharContent” to return the java source from the String.

The following code listing for StringCompile illustrates the use of our custom StringJavaFileObject to
compile and execute a Filter class implementation that is generated in a String:

/*
* Name: Bhaskar S
*
* Date: 10/10/2009
*/

package com.polarsparc.jdk.compiler;

import java.util.*;
import java.io.*;
import java.net.*;

import javax.tools.*;
import javax.tools.JavaCompiler.CompilationTask;

public class StringCompile {


public static void main(String[] args) {
try {
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();

StandardJavaFileManager manager = compiler.getStandardFileManager(null, null,


null);

/* 1 */ JavaFileObject strFile = new


StringJavaFileObject("com.polarsparc.jdk.compiler.FilterEven", generateJavaCode());

Iterable<? extends JavaFileObject> units = Arrays.asList(strFile);

String[] opts = new String[] { "-d", "classes" };

CompilationTask task = compiler.getTask(null, manager, null,


Arrays.asList(opts), null, units);
boolean status = task.call();
if (status) {
System.out.printf("Compilation successful!!!\n");

File classesDir = new File("classes");


URL[] classpath = new URL[] { classesDir.toURI().toURL() };
/* 2 */ URLClassLoader urlClassloader = new URLClassLoader(classpath,
StringCompile.class.getClassLoader());
/* 3 */ Class<?> clazz =
urlClassloader.loadClass("com.polarsparc.jdk.compiler.FilterEven");
/* 4 */ Filter filter = (Filter) clazz.newInstance();
/* 5 */ if (filter.filter(10)) {
System.out.println("10 is an even number");
}
/* 6 */ if (filter.filter(15) == false) {
System.out.println("15 is an odd number");
}
}
else {
System.out.printf("***** Compilation failed!!!\n");
}
}
catch (Exception ex) {
ex.printStackTrace(System.err);
System.exit(1);
}
}

private static String generateJavaCode() {


StringBuilder sb = new StringBuilder();
sb.append("package com.polarsparc.jdk.compiler;");
sb.append("public class FilterEven implements Filter {");
sb.append(" @Override");
sb.append(" public boolean filter(int num) {");
sb.append(" if (num % 2 == 0) {");
sb.append(" return true;");
sb.append(" }");
sb.append(" return false;");
sb.append(" }");
sb.append("}");
return sb.toString();
}
}

In line /* 1 */, we create an instance of StringJavaFileObject by specifying the full class name as
“com.polarsparc.jdk.compiler.FilterEven” and generating the java source for FilterEven by calling the
method “generateJavaCode”.

The steps to the compile the generated code is the same as was illustrated in BasicCompile. Once the
compilation proceeds successfully, a class file for FilterEven is generated under the “classes” directory.

In line /* 2 */, we create an instance of URLClassLoader for the “classes” directory so that we can load
the dynamically generated compiled class for FilterEven.

In line /* 3 */, we load the class for FilterEven using the URLClassLoader created in /* 2 */.

In line /* 4 */, we create an instance of FilterEven which implements the interface for Filter.

In line /* 5 */ and /* 6 */, we invoke the method “filter” on the FilterEven instance created in /* 4 */.

Executing the above code, shows that the compilation and execution of the dynamically generated Java
code is successful:

With this, we have successfully demonstrated the use of the Java compiler API to generate java class
file from dynamically generated java source from a String.

The next natural question to ask is: what if we make a mistake in the java source generation. Let us
change the method “generateJavaCode” in StringCompile as follows:

private static String generateJavaCode() {


StringBuilder sb = new StringBuilder();
sb.append("package com.polarsparc.jdk.compiler;");
sb.append("public class FilterEven implements Filter {");
sb.append(" @Override");
sb.append(" public boolean filter(int num) {");
sb.append(" if (num % 2 == 0) {");
sb.append(" return 2;");
sb.append(" }");
sb.append(" return false;");
sb.append(" }");
sb.append("}");
return sb.toString();
}
The line with the error is highlighted above. When we execute the StringCompile, we will see the
following result:

The JavaCompiler API fails to compile the generated java source and provides enough information on
the failure.

In the StringCompile example, the compiled class file for the dynamically generated java source is
saved under the “classes” directory. It would be most interesting if the compiled class is also generated
and loaded from memory via a byte array.

The JavaCompiler uses StandardJavaFileManager to perform both read and write on files using objects
of type JavaFileObject. The JavaCompiler reads the input source file and on successful compile writes
the output compiled class file. Just as we used the class StringJavaFileObject to wrap the generated
java source in a String, we will use a custom class that extends SimpleJavaFileObject to wrap the
compiled class bytes.

The following code listing for ByteArrayJavaFileObject illustrates our custom JavaFileObject that
allows the JavaCompiler to write compiled class bytes to:

/*
* Name: Bhaskar S
*
* Date: 10/10/2009
*/

package com.polarsparc.jdk.compiler;

import java.io.*;
import java.net.*;
import javax.tools.*;

public class ByteArrayJavaFileObject extends SimpleJavaFileObject {


private final ByteArrayOutputStream bos = new ByteArrayOutputStream();

public ByteArrayJavaFileObject(String name, Kind kind) {


super(URI.create("string:///" + name.replace('.', '/') + kind.extension), kind);
}

public byte[] getClassBytes() {


return bos.toByteArray();
}

@Override
public OutputStream openOutputStream()
throws IOException {
return bos;
}
}
The above code listing is similar to that of StringJavaFileObject, except that we are overriding the
method “openOutputStream” so that the JavaCompiler can use it to output compiled class bytes.

As indicated earlier, the StandardJavaFileManager is a default file manager that creates JavaFileObject
instances representing regular files from the file system and used by the JavaCompiler. In order to use
our StringJavaFileObject for input and ByteArrayJavaFileObject for output, we need to have a custom
JavaFileManager. We cannot extend the StandardJavaFileManager as it does not expose any public
constructor and is created internally by the JavaCompiler. Instead, we extend the delegating file
manager from the javax.tools package called ForwardingJavaFileManager that allows for
customization while delegating to the underlying default file manager the StandardJavaFileManager.

Once the JavaCompiler write the compiled class to a byte array, we will need a way to load the bytes
into the class loader. In order to do this we will need a custom class loader to load a class from class
bytes.

The following code listing for ByteArrayClassLoader illustrates our custom class loader which extends
the default java class loader to load the class bytes from the ByteArrayJavaFileObject:

/*
* Name: Bhaskar S
*
* Date: 10/10/2009
*/

package com.polarsparc.jdk.compiler;

import java.util.*;

public class ByteArrayClassLoader extends ClassLoader {


private Map<String, ByteArrayJavaFileObject> cache = new HashMap<String,
ByteArrayJavaFileObject>();

public ByteArrayClassLoader()
throws Exception {
super(ByteArrayClassLoader.class.getClassLoader());
}

public void put(String name, ByteArrayJavaFileObject obj) {


ByteArrayJavaFileObject co = cache.get(name);
if (co == null) {
cache.put(name, obj);
}
}

@Override
protected Class<?> findClass(String name)
throws ClassNotFoundException {
Class<?> cls = null;

try {
ByteArrayJavaFileObject co = cache.get(name);
if (co != null) {
byte[] ba = co.getClassBytes();
cls = defineClass(name, ba, 0, ba.length);
}
}
catch (Exception ex) {
throw new ClassNotFoundException("Class name: " + name, ex);
}
System.out.printf("Method findClass() called for class %s\n", name);
return cls;
}
}
In the above custom class loader, we maintain an internal cache of ByteArrayJavaFileObject for each
of the dynamically generated java source. The cache is update by our custom JavaFileManager for each
generated java source.

The following code listing for DynamicClassFileManager illustrates our custom JavaFileManager that
will be used by the JavaCompiler to read java source from a String and to write the compiled class to a
byte array:

/*
* Name: Bhaskar S
*
* Date: 10/10/2009
*/

package com.polarsparc.jdk.compiler;

import java.io.*;

import javax.tools.*;
import javax.tools.JavaFileObject.Kind;

/* 1 */ public class DynamicClassFileManager <FileManager> extends


ForwardingJavaFileManager<JavaFileManager> {
private ByteArrayClassLoader loader = null;

DynamicClassFileManager(StandardJavaFileManager mgr) {
super(mgr);
try {
/* 2 */ loader = new ByteArrayClassLoader();
}
catch (Exception ex) {
ex.printStackTrace(System.out);
}
}

@Override
/* 3 */ public JavaFileObject getJavaFileForOutput(Location location, String name, Kind kind,
FileObject sibling)
throws IOException {
ByteArrayJavaFileObject co = new ByteArrayJavaFileObject(name, kind);
loader.put(name, co);
return co;
}

@Override
/* 4 */ public ClassLoader getClassLoader(Location location) {
return loader;
}
}

In line /* 1 */, we extend from the class ForwardingJavaFileManager which is provided by the Java
compiler API as a simple implementation for delegating JavaFileManager. As indicated earlier, we
cannot extend SimpleJavaFileManager as it does not expose any public constructor. Instead, we use the
delegator ForwardingJavaFileManager which delegates to the underlying SimpleJavaFileManager.

In line /* 2 */, we create an instance of our custom class loader ByteArrayClassLoader.

In line /* 3 */, we override the method “getJavaFileForOutput”. This method is invoked by the
JavaCompiler when it is ready to write the compiled class bytes for the given java source input. In this
method, we create any instance of the custom ByteArrayJavaFileObject and save it in our custom class
loader ByteArrayClassLoader for the given class name indicated by the “name” argument.
In line /* 4 */, we override the method “getClassLoader” to return our custom class loader
ByteArrayClassLoader.

We now have all the necessary ingredients to create any java source and its corresponding class on the
fly at runtime. The following code listing for DynamicCompile puts all these ingredients together:

/*
* Name: Bhaskar S
*
* Date: 10/10/2009
*/

package com.polarsparc.jdk.compiler;

import java.util.*;
import javax.tools.*;
import javax.tools.JavaCompiler.CompilationTask;

public class DynamicCompiler {


private JavaCompiler compiler;
private DiagnosticCollector<JavaFileObject> collector;
private JavaFileManager manager;

public void init()


throws Exception {
compiler = ToolProvider.getSystemJavaCompiler();
collector = new DiagnosticCollector<JavaFileObject>();
manager = new
DynamicClassFileManager<JavaFileManager>(compiler.getStandardFileManager(null, null, null));
}

public Class<?> compileToClass(String fullName, String javaCode)


throws Exception {
Class<?> clazz = null;

StringJavaFileObject strFile = new StringJavaFileObject(fullName, javaCode);


Iterable<? extends JavaFileObject> units = Arrays.asList(strFile);
CompilationTask task = compiler.getTask(null, manager, collector, null, null, units);
boolean status = task.call();
if (status) {
System.out.printf("Compilation successful!!!\n");
clazz = manager.getClassLoader(null).loadClass(fullName);
}
else {
System.out.printf("Message:\n");
for (Diagnostic<?> d : collector.getDiagnostics()) {
System.out.printf("%s\n", d.getMessage(null));
}
System.out.printf("***** Compilation failed!!!\n");
}

return clazz;
}
}

The above code provides a convenience method “compileToClass” that takes a full class name and its
corresponding java code and returns the corresponding Class object for that class name. Pay close
attention to the code and you will see the DiagnosticCollector. Earlier, we saw the result of wrong code
generation. The error was reported by the JavaCompiler in a standard format. If we want customize
error reporting, then we can collect the error information by using the DiagnosticCollector.

Finally, we present an illustration that shows how to use the DynamicCompiler for dynamic code
generation. The following code listing for CompiledFilter generates two types of Filter implementation
based on the user preference:
/*
* Name: Bhaskar S
*
* Date: 10/10/2009
*/

package com.polarsparc.jdk.compiler;

import java.util.*;

public class CompiledFilter {


private static int[] nums;
private static Filter func;

public static void main(String[] args) {


if (args.length != 1) {
System.out.printf("Usage: java %s <DynFilterEvenGt500 | DynFilterOddLt500>\n",
CompiledFilter.class.getName());
System.exit(1);
}
init(args[0]);
execute();
}

public static void init(String name) {


try {
nums = new int[100];
Random rand = new Random();
for (int i = 0; i < 100; ++i) {
nums[i] = rand.nextInt(1000);
}

DynamicCompiler compiler = new DynamicCompiler();


compiler.init();

Class<?> clazz = null;


if (name.equals("DynFilterEvenGt500")) {
clazz = compiler.compileToClass("com.polarsparc.jdk.compiler." + name,
DynFilterEvenGt500());
}
else {
clazz = compiler.compileToClass("com.polarsparc.jdk.compiler." + name,
DynFilterOddLt500());
}

func = (Filter) clazz.newInstance();


}
catch (Throwable ex) {
ex.printStackTrace(System.err);
System.exit(1);
}
}

public static void execute() {


try {
long stm = System.currentTimeMillis();
for (int n : nums) {
if (func.filter(n)) {
System.out.printf("CompiledFilter: %d\n", n);
}
}
long etm = System.currentTimeMillis();
System.out.printf("CompiledFilter execute time: %d ms\n", (etm-stm));
}
catch (Throwable ex) {
ex.printStackTrace(System.err);
System.exit(1);
}
}
private static String DynFilterEvenGt500() {
StringBuilder sb = new StringBuilder();
sb.append("package com.polarsparc.jdk.compiler;\n");
sb.append("public class DynFilterEvenGt500 implements Filter {\n");
sb.append(" @Override\n");
sb.append(" public boolean filter(int num) {\n");
sb.append(" if (num > 500 && (num % 2) == 0) {\n");
sb.append(" return true;\n");
sb.append(" }\n");
sb.append(" return false;\n");
sb.append(" }\n");
sb.append("}\n");
return sb.toString();
}

private static String DynFilterOddLt500() {


StringBuilder sb = new StringBuilder();
sb.append("package com.polarsparc.jdk.compiler;\n");
sb.append("public class DynFilterOddLt500 implements Filter {\n");
sb.append(" @Override\n");
sb.append(" public boolean filter(int num) {\n");
sb.append(" if (num < 500 && (num % 2) != 0) {\n");
sb.append(" return true;\n");
sb.append(" }\n");
sb.append(" return false;\n");
sb.append(" }\n");
sb.append("}\n");
return sb.toString();
}
}

If the user chooses DynFilterEvenGt500, an even Filter implementation is generated and executed. On
the other hand, if the user chooses DynFilterOddLt500, an odd Filter implementation is generated. The
shows the result of the user choosing DynFilterOddLt500:

With this we conclude this article on Dynamic Code Generation in Java 6 – let the force be with you
Luke !!!

S-ar putea să vă placă și