Ola Bini: Programming Language Synchronicity

I've just spent my first three days on a project in Leeds. It's a pretty common Java project, RESTful services and some MVC screens. We have been using Mockito for testing which is a first for me. My immediate impression is quite good. It's a nice tool and it allows some very clean testing of stuff that generally becomes quite messy. One of the things I like is how it uses generics and the static typing of Java to make it really easy to make mocks that are actually type checked; like this for example:

Iterator iter = mock(Iterator.class);
stub(iter.hasNext()).toReturn(false);

// Call stuff that starts interaction

verify(iter).hasNext();

These are generally the only things you need to stub stuff out and verify that it was called. The things you don't care about you don't verify. This is pretty good for being Java, but there are some problems with it too. One of the first things I noticed I don't like is that interactions that isn't verified can't be disallowed in an easy way. Optimally this would happen at the creation of the mock, instead of actually calling the verifyNoMoreInteractions() afterwards instead. It's way to easy to forget. Another problem that quite often comes up is that you want to mock out or stub some methods but retain the original behavior of others. This doesn't seem possible, and the alternative is to manually create a new subclass for this. Annoying.

Contrast this to testing the same interaction with Mocha, using JtestR, the difference isn't that much, but there is some missing cruft:

iter = mock(Iterator)
iter.expects(:hasNext).returns(false)

# Call stuff that starts interaction

Ruby makes the checking of interactions happen automatically afterwards, and so you don't have any types you don't need to care about most stuff the way you do in Java. This also shows a few of the inconsistencies in Mockito, that is necessary because of the type system. For example, with the verify method you send the mock as argument and the return value of the verify-method is what you call the actual method on, to verify that it's actually called. Verify is a generic method that returns the same type as the argument you give to it. But this doesn't work for the stub method. Since it needs to return a value that you can call toReturn on, that means it can't actually return the type of the mock, which in turn means that you need to call the method to stub before the actual stub call happens. This dichotomy gets me every time since it's a core inconsistency in the way the library works.

Contrast that to how a Mockito like library might look for the same interaction:

iter = mock(Iterator)
stub(iter).hasNext.toReturn(false)

# Do stuff

verify(iter).hasNext

The lack of typing makes it possible to create a cleaner, more readable API. Of course, these interactions are all based on how the Java code looked. You could quite easily imagine a more free form DSL for mocking that is easier to read and write.

Conclusion? Mockito is nice, but Ruby mocking is definitely nicer. I'm wondering why the current mocking approaches doesn't use the method call way of defining expectations and stubs though, since these are much easier to work with in Ruby.

Also, it was kinda annoying to upgrade from Mockito 1.3 to 1.4 and see half our tests starting to fail for unknown reasons. Upgrade cancelled.

JtestR allows you to test your Java code with Ruby frameworks.

Homepage: http://jtestr.codehaus.org
Download: http://dist.codehaus.org/jtestr

JtestR 0.3 is the current release of the JtestR testing tool. JtestR integrates JRuby with several Ruby frameworks to allow painless testing of Java code, using RSpec, Test/Unit, Expectations, dust and Mocha.

Features:
- Integrates with Ant, Maven and JUnit
- Includes JRuby 1.1, Test/Unit, RSpec, Expectations, dust, Mocha and ActiveSupport
- Customizes Mocha so that mocking of any Java class is possible
- Background testing server for quick startup of tests
- Automatically runs your JUnit and TestNG codebase as part of the build

Getting started: http://jtestr.codehaus.org/Getting+Started

The 0.3 release has focused on stabilizing Maven support, and adding new capabilities for JUnit integration.

New and fixed in this release:
JTESTR-47 Maven with subprojects should work intuitively
JTESTR-42 Maven dependencies should be automatically picked up by the test run
JTESTR-41 Driver jtestr from junit
JTESTR-37 Can't expect a specific Java exception correctly
JTESTR-36 IDE integration, possibility to run single tests
JTESTR-35 Support XML output of test reports

Team:
Ola Bini - ola.bini@gmail.com
Anda Abramovici - anda.abramovici@gmail.com

As some of you might have noticed I am not extremely fond of everything the Java language. I have spent some time lately trying to figure out how I would change the language if I could. These changes are of course breaking, and would never be included in regular Java. I've had several names for it, but my current favorite is unJava. You can call it Java .314 or minijava if you want. Anyway, here's a quick breakdown of what I'd like to see done to make a better language out of Java without straying to far away from the current language:

• No primitives. No ints, bytes, chars, shorts, floats, booleans, doubles or longs. They are all evil and should not be in the language.
• No primitive arrays. Javas primitive arrays are not typesafe and are evil. With generics there is no real point in having them, especially since they interact so badly with generic collections. This point would mean that certain primitive collection types can't be implemented in the language itself. This is a price I'm willing to pay.
• Scala style generics, with usage-defined contra/co-variance and more flexible type bounds.
• No anonymous inner classes. There is no need for them with the next points.
• First class methods.
• Anonymous methods (these obviously need to be closures).
• Interfaces that carries optional implementation.
• No abstract classes - since you don't need them with the above.
• Limited type inference, to avoid some typing. Scala or C# style is fine.
• Annotations for many of the current keywords - accessibility specifically, but also things like transient, volatile and synchronized.
• No checked exceptions.
• No angle brackets for generics (they really hurt my eyes. are there XML induced illnesses? XII?). Square brackets look so much better.
• Explicit separation of nullable from non-nullable values.

These points are probably quite substantial together, but I still don't think the language would be that difference from Java in syntax and semantics. The usage patterns would be extremely different though. You wouldn't sacrifice any performance with these kinds of things - they wouldn't change the characteristics of the output that much, and I believe these things could make the language smaller, cleaner, and easier to work with.

My former colleague Lars Westergren recently posted a blog (here) about type inferencing, posing the question whether type inference would actually be good for Java, and if it would provide any benefits outside of just "less typing".

In short: no. Type inferencing would probably not do much more than save you some typing. But how much typing it would save you could definitely vary depending on the type of type inference you added. The one version I would probably prefer is just a very simple hack to avoid writing out the generic type arguments. One simple way of doing that would be to allow an equals sign inside of the angle brackets. In that case you could do this:

List<=> l = new ArrayList<String>();
List<String> l2 = new ArrayList<=>();

Of course, you can do it on more complicated expressions:

List<Set<Map<Class<?>, List<String>>>> l = new ArrayList<=>();

This would save us some real pain in the definition of genericized types, and it wouldn't strip away much stuff you need for readability. In the above examples it would just strip away one duplication, and you don't need that duplication to read it correctly. The one case where it might be a little bit harder to read would be if you defined a variable and assigned it somewhere else. In that case the definition would need to carry the type information, so the instantiation would use the <=> syntax. I think that would be an acceptable price to reduce the verbosity of Java generics.

Another kind of generics that would be somewhat useful is the kind added to C#, which is only local to a scope. That means there will be no type inferencing of member variables, method parameters or return values. Of course, that's the crux of Lars question, since this kind of type inference potentially removes ALL type information in the current text, since you can do:

var x = someValue.DoSomething();

At this point there is no easy way for you to know what the type of x actually is. Reading it like this, it looks a bit frightening if you're used to Java type tags, but in fact this is not what you would see. In most cases you have a small method - maybe 5-15 lines of code, where x is being used in some way or another. In many cases you will see methods called on x, or x used as argument to method calls. Both of these usages gives you clues about what it might be, but in fact you don't always need to know what type it is. You just need to know what you can do with it. And that's exactly what Java interfaces represent. So for example, do you know what class you get back from Collections.synchronizedMap()? No, and you shouldn't need to know. What you do know is that it's something that implements Map, and the documentation says that it is synchronized, but that is it. The only thing you know about it is that you can use it as a map.

So in practice, the kind of type inference C# adds is actually quite useful, clean, and doesn't cause too much trouble - especially if you have one of those fancy ideas that do method completion... =)

From another angle, there are some things that type inference could possible do, but that you will never see in Java. For example, say that you assign a variable to something, and later you assign that variable to some other value. If these two values are distinct types that doesn't overlap in the inheritence chain, you will usually get an error. But if you have an advanced type system, it will do unification for you. The basic versions will just find the most common supertype (the disjunction), but you can also imagine the compiler injecting a new type into your program that is the union of the two types in use. This will provide something similar to duck typing while still retaining some static type safety. If your type system allows multiple inheritence, the synthetic union type might even be a subclass of both the types in question.

So yeah. The long answer is that you can actually do some funky stuff with type inference that doesn't immediately translate to less typing. Although less typing and better abstractions is what programming languages are all about, right? Otherwise assembler provides everything we want.

Today I spent some time connecting two languages that are finding themselves popular for solving wildly different kinds of problems. I decided I wanted to see how easy it was and if it was a workable solution if you would want to take advantage of the strengths of both languages. The result is really up to you. My 15 minutes experiment is what I'll discuss here.

If you'd like, you can see this as a practical example of the sticky part where two languages meet, in language-oriented programming.

The languages under consideration is Ruby and Erlang. The prerequisite reading is this eminent article by my colleague Dennis Byrne: Integrating Java and Erlang.

The only important part is in fact the mathserver.erl code, which you can see here:

-module(mathserver).
-export([start/0, add/2]).

start() ->
Pid = spawn(fun() ->loop() end),
register(mathserver, Pid).

loop() ->
receive
 {From, {add, First, Second}} ->
 From ! {mathserver, First + Second},
 loop()
end.

add(First, Second) ->
mathserver ! {self(), {add, First, Second}},
receive
 {mathserver, Reply} ->Reply
end.

Follow Dennis' instructions to compile this code and start the server in an Erlang console, and then leave it there.

Now, to use this service is really easy from Erlang. You can really just use the mathserver:add/2 operation directly or remotely. But doing it from another language, in this case Ruby is a little bit more complicated. I will make use of JRuby to solve the problem.

So, the client file for using this code will look like this:

require 'erlang'

Erlang::client("clientnode", "cookie") do |client_node|
server_node = Erlang::OtpPeer.new("servernode@127.0.0.1")
connection = client_node.connect(server_node)

connection.sendRPC("mathserver", "add", Erlang::list(Erlang::num(42), Erlang::num(1)))

sum = connection.receiveRPC

p sum.int_value
end

OK, I confess. There is no erlang.rb yet, so I made one. It includes some very small things that make the interfacing with erlang a bit easier. But it's actually still quite straight forward what's going on. We're creating a named node with a specific cookie, connecting to the server node, and then using sendRPC and receiveRPC to do the actual operation. The missing code for the erlang.rb file should look something like this (I did the minimal amount here):

require 'java'
require '/opt/local/lib/erlang/lib/jinterface/priv/OtpErlang.jar'

module Erlang
import com.ericsson.otp.erlang.OtpSelf
import com.ericsson.otp.erlang.OtpPeer
import com.ericsson.otp.erlang.OtpErlangLong
import com.ericsson.otp.erlang.OtpErlangObject
import com.ericsson.otp.erlang.OtpErlangList
import com.ericsson.otp.erlang.OtpErlangTuple

class << self
def tuple(*args)
 OtpErlangTuple.new(args.to_java(OtpErlangObject))
end

def list(*args)
 OtpErlangList.new(args.to_java(OtpErlangObject))
end

def client(name, cookie)
 yield OtpSelf.new(name, cookie)
end

def num(value)
 OtpErlangLong.new(value)
end

def server(name, cookie)
 server = OtpSelf.new(name, cookie)
 server.publish_port

 while true
 yield server, server.accept
 end
end
end
end

As you can see, this is regular simple code to interface with a Java library. Note that you need to find where JInterface is located in your Erlang installation and point to that (and if you're on MacOS X, the JInterface that comes with ports doesn't work. Download and build a new one instead).

There are many things I could have done to make the api MUCH easier to use. For example, I might add some methods to OtpErlangPid, so you could do something like:

pid << [:call, :mathserver, :add, [1, 2]]

where the left arrows sends a message after transforming the arguments.

In fact, it would be exceedingly simple to make the JInterface API downright nice to use, getting the goodies of Erlang while retaining the Ruby language. And oh yeah, this could work on MRI too. There is an equivalent C library for interacting with Erlang, and there could either be a native extension for doing this, or you could just wire it up with DL.

If you read the erlang.rb code carefully, you might have noticed that there are several methods not in use currently. Say, why are they there?

Well, it just so happens that we don't actually have to use any Erlang code in this example at all. We could just use the Erlang runtime system as a large messaging bus (with fault tolerance and error handling and all that jazz of course). Which means we can create a server too:

require 'erlang'

Erlang::server("servernode", "cookie") do |server, connection|
 terms = connection.receive
 arguments = terms.element_at(1).element_at(3)
 first = arguments.element_at(0)
 second = arguments.element_at(1)

 sum = first.long_value + second.long_value
 connection.send(connection.peer.node, Erlang::tuple(server.pid, Erlang::num(sum)))
end

The way I created the server method, it will accept connections and invoke the block for every time it accepts a connection. This connection is yielded to the block together with the actual node object representing the server. The reason the terms are a little bit convoluted is because the sendRPC call actually adds some things that we can just ignore in this case. But if we wanted, we could check the first atoms and do different operations based on these.

You can run the above code in server, and use the exact same math code if you want. For ease of testing, switch the name to servernode2 in both server and client, and then run them. You have just sent Erlang messages from Ruby to Ruby, passing along Java on the way.

Getting different languages working together doesn't need to be hard at all. In fact, it can be downright easy to switch to another language for a few operations that doesn't suit the current language that well. Try it out. You might be surprised.

I'm attending QCon in London this whole week. I'll give a talk on Thursday called "Evolving the Java platform", talking about different things that might be a part of the next generation Java platform. On Friday I give the talk "JRuby: Power on the JVM", which is a more advanced JRuby talk than the regular introduction talks I usually give.

Hope to see you around!

I usually don't mention the G word in my blog - the results have a tendency to be less than palatable. This time I'm going to make an exception though. The reason is that this meme has been floating around as an argument for Groovy, and I really don't agree with it. In fact, the reason I'm writing about it is because I think the meme in itself generally can be bad for Groovy.

The thing I'm talking about is the saying that Groovy is just like Java. That you can start using Groovy directly without learning anything about Groovy and that basically all Java syntax is valid Groovy.

Now, the last part has never been really true, since there are a few valid Java constructs that are not supported in Groovy (anonymous classes, I'm looking at you). This might be fixed in the 1.5 version. I don't really know, but that's not the point. Since Groovy isn't using the Java parser there will always be a few small corner cases that are not valid in Groovy but Java accepts. This is not a problem, by the way. The only way it can be a problem is when you're using Groovy the wrong way, or you absolutely need to do something that Groovy doesn't support. Since one of the Groovy catch calls is that you shouldn't implement everything in Groovy, this can't really be a concern either. If you're using Groovy correctly, you should just drop down to the Java layer for that particular feature.

Now that that part is out of the way, let's take a look at the other parts of this idea. Namely that you should choose Groovy because it's just like Java, and you can start writing Java code directly in Java. These statements are definitely true, since you can absolutely pick up a language that way. In my mind, though, this misses the whole point of using another language on top of the JVM. If you're building an application and chooses to use Groovy for this instead of Java, isn't the reason that you're doing that the fact that you expect Groovy to give you something Java does not? And if that's the case, doesn't it sound like a disservice to people picking up Groovy to say that they can begin by just programming Java. There is no gain here. Really. In fact, the equivalent program in Groovy and Java will perform much worse on Groovy since there are several layers of dispatching in Groovy that you don't have in Java. The result? An application with exactly the same source code, but running 10 times slower.

Since this message is mostly used in marketing and introduction facilities, the people it's geared at are by definition newbies to the Groovy world. Many of them are totally new to dynamic languages. And the first thing they do is write Java in Groovy, find it slow and obviously, from that point don't understand why you should use Groovy. This is bad for Groovy. Groovy has some extremely powerful features and libraries that can really make your life on the JVM blissful (compared to using Java, for example). Groovy's integration with Java is top notch, which means that you can always fall back to Java when you need Java specific features. If you're using Groovy, I think you should use the full power of the language so you can actually get the full benefit of Groovy.

I feel that a few rants are about to burst in me. Let me take this first; this is not a rant, though. Just a very large exclamation:

Java Annotations are NOT a way to implement DSLs, and will never be.

Java annotations are really nice, but seriously, they are NOT manna from heaven.

This post is basically a follow up to Language explorations. That post had several statements about what I believe, and some of them have been a bit misunderstood, so I'll try to substantiate them a bit more here. Specifically, I want to talk about the viability of the Java language and whether "the stable layer" should be a static or dynamic language.

Lets begin with the stable layer. First, notice that I didn't have any specific definitions for the different layers when I wrote the post. In fact, I think that they are inherently a bit fuzzy. That's OK. DSL's are also quite fuzzy. The attributes I put at the stable layer is first of all that it shouldn't need to change much. In a smaller application, you can actually look at supporting structures as the stable layer, things like application servers or web servers. But for larger applications I find that you usually need your own stable kernel.

The reasons I choose static typing for stable layer is for several reasons. Performance is absolutely one of these, since everything will run on this base it's an advantage if that part is as performant as possible. Secondly, I am no dynamic language, TDD purist that says that static typing is totally unnecessary. Static type checking can be very helpful, but it stops you from having a malleable language. So the kernel should be stable, this means that as much checking as possible is necessary. None of these requirements explicitly preclude a dynamic language, but the specific features of dynamic languages doesn't really come to their rights at this level, and also leading to performance and static type checking suffering.

The more contentious part about my post seems to have been my off hand comment that I don't believe Java is really good for anything. I didn't give much reason in that post, and people have reacted in different ways, from saying that "Java is really good" to "Why are you involved in JRuby if you think Java is so bad?". So lets start with why Java is a bad language. (I am only talking about the language here, not the platform). Also, that Java is a bad doesn't say anything about the worse alternatives, so no comments along the lines of "if you think Java is so bad, why don't you use X instead".

In short:

• Java is extremely verbose. This is really a few different problems in one:
• No type inference
• Verbose generic type signatures
• Anonymous class implementations
• There is no way to create new kinds of abstractions:
• No closures
• Anonymous classes are a really clunky way to handle "block" functionality
• Broken generics implementation
• Language is becoming bloated

A few notes. That Java is becoming bloated is one of those things that people have been blogging about lately. After 1.5 came out, the complexity of the language got very much larger, without actually adding much punch for it. The improvements of generics were mostly negated by the verbosity of them. Fixing all the problems above will bloat the language even more. And at the same time, programming in Java is needlessly painful right now.

So, lets look at the question about why I'm working on JRuby. Well, first, I believe the JVM is a very good place to be at. It's the best platform out there, in my opinion. The libraries are great, the runtime is awesome, and it's available basically everywhere. The bytecodes of the JVM spec is good enough for most purposes. There is some tweaking that can be done (which we are looking at in JSR292), but mostly it's a very nice place. And working on JRuby is really one of the ways I've seen how bad Java as a language is. We are employing several different tricks to get away from the worst parts of it, though. Code generation is used in several places. We are generating byte codes at runtime. We are using annotations to centralize handling of JRuby methods. And we are moving parts of the implementation to Ruby. I believe that JRuby is important because it can run in the same environment as Java, but without the problems of Java.

What are the solutions to the problem with Java? There are basically two different ways. Either define a subset of Java, not necessarily totally compatible, that takes the best parts of Java syntax, does away with the problems and so on. That should be possible, but I don't know anyone who has done it. Or, you can go with an existing static language on the JVM. Here you have two choices - either one of the ports of existing extremely static languages (like ML or Haskell), or you can go with something like Scala. I haven't decided on the best solution here yet. The only thing I'm sure of is that Java in itself is a problem. I'm investigating Scala, but maybe Scala isn't the right answer either. We'll see.

As everyone knows, regular expressions are incredibly important in many programming tasks. So it pays to know some of the particulars of the regexp syntax. One example that bit me a while back was a simple oversight - something I did know but hadn't kept in mind while writing the bad code. Namely, the way the caret (^) works when used in a String with newlines in it. To be fair I've been using Java regexps for a while and that problem doesn't exist there.

To illustrate the difference, here is a program you can run in either MRI or JRuby. If running in JRuby you'll see that the Java version needs the flag MULTILINE to behave as Ruby does by default.

str = "one\nover\nyou"
puts "Match with ^"
str.gsub(/^o/) do |e|
 p $~.offset(0)
 e
end

puts "Match with \\A"
str.gsub(/\Ao/) do |e|
 p $~.offset(0)
 e
end


if defined?(JRUBY_VERSION)
 require 'java'
 regexp = java.util.regex.Pattern.compile("^o", java.util.regex.Pattern::MULTILINE)
 matcher = regexp.matcher(str)
 puts "Java match with ^"
 while matcher.find()
 p matcher
 end

 regexp = java.util.regex.Pattern.compile("\\Ao", java.util.regex.Pattern::MULTILINE)
 matcher = regexp.matcher(str)
 puts "Java match with \\A"
 while matcher.find()
 p matcher
 end
end

So, what's the lesson here? Don't use caret (^) and dollar ($) if you actually want to match the beginning or the end of the string. Instead, use \A and \Z. That's what they're there for.

I'm not sure if this is well known or not, so I've been meaning to write a quick notice about it. The short story is this: JRuby can run RSpec and RBehave. Why is this important? Well, you can write code that tests Java code using RSpec and RBehave, meaning that it will be possible to get much more readable tests, even for code living in Java land.

Even if your organization won't accept writing an application in Ruby, it would probably be easier to get the testing done in Ruby. And writing tests in an effective language means that you will either write more production code, or more tests. Either of those are a quite good outcome.

A quick example of this in action. To run this example, you need JRuby 1.0 or later, and the rspec gem:

require 'java'

describe java.util.ArrayList, " when first created" do
before(:each) do
 @list = java.util.ArrayList.new
end

it "should be empty" do
 @list.should be_empty
end

it "should be able to add an element" do
 @list.add "content"
end

it "should raise exception when getting anything" do
 lambda{ @list.get 0 }.should raise_error(java.lang.IndexOutOfBoundsException)
end
end

In this code the examples are not that realistic, but you can see that the RSpec code looks the same for Java code, as it does for Ruby code. Even the raise_error exception matcher works. You can run this:

jruby -S spec -fs arraylist_spec.rb

The RBehave test suite also runs, which means you can stop using JBehave now... =)

This is a perfect example of the intersection where JRuby's approach can be very powerful, utilizing the existing Ruby libraries to benefit your Java programming.

I have for a time argued that the JVM should become more like an Operating System, and Java the language for OS development. Other languages should run on top of the JVM for running applications. It seems I'm not alone in this line of thought. Robert Varttinen wrote some about it here. To me it seems like a compelling future. But it seems the next logical step for enterprise applications would be further virtualization. I would for example like my beans implemented in Ruby. But not only that, I would want all the business logic to be OS agnostic. I would like my Ruby logic to live on top of a JVM J2EE server, but that logic should be able to move, transparently, to a .NET-server and provide the same business logic at that place. What would be even better is if I didn't have to deploy it manually to all places, but the logic would just move to the places where it's needed. Will we see that anytime soon?

My blogging in the future will be a little bit fractured (or more fractured, some might say), since I have been invited to write at Inside Java for APress. The address is http://java.apress.com and I do recommend that you subscribe to the feed if you're interested in Java or associated technologies. My first posting was about the two closure proposals for Java 7, and I will try to focus my posting there to be more Java specific, mostly in article format. But if I write anything I deem to be exceptionally good, I promise to link from here to there.

Go subscribe now!

Yesterday marked the first one day conference in Sweden focused on Java. All in all, it was a resounding success. I am personally very happy about how well everything ran. The only part that wasn't really up to par was the panel discussion.

So, a quick rehash of the day. First Simon Phipps did his thing in the keynote, speaking much about where Sun's open source initiative comes from. I would have liked some more specific info on the future though, but I guess that's hard for him to say...

After that, Thorbiörn Fritzon from Sun in Sweden started of the Language track with talking about some new features in Mustang, specifically some Swing topics and the JSR 223 support (scripting). After that he showcased a really cool stack-based calculator he'd hacked during the weekend, where you could attach the functioning of the buttons to different scripts, and the scripts could be implemented in any language on the classpath that supported JSR 223. The final showdown of calculator buttons was first a script in Haskell (using Jaskell) that pushed a lambda on the stack. This lambda curried addition, with the first object on the stack. Then another script written in Python popped the lambda from the stack, popped the next value and called the lambda with the value from the stack, in effect doing an addition the longer way...

After that it was time for my presentation. It felt really good and I believe most in the audience was satisfied.

After that I sat in on Dan Bergh Johnsson from Omegapoint talking about domain driven design, and how to refactor to find your domain. I found it quite elementary, and nothing really new.

Last, Jonas Bonér talked about OpenTerracotta, which is an incredibly cool tool for clustering your JVM applications, and by doing that achieving many different cool things. Look it up! I wish there had been more time for Jonas to get in depth about the implementation of Terracotta, though. Most interesting thing he told us was about a situation where they wanted about 50-80 clients pounding in a cluster at the same time, and they used Terracotta for this too, using a CyclicBarrier that was synchronized over all the JVM's. Very nice.

The panel debate wasn't that interesting. Most of the questions asked by the moderator didn't really achieve any discussion. Open sourcing Java doesn't seem like such a controversial topic anymore.

After the debate, the conference was over, but then BEA provided a pub evening with entertainment, so we stayed and networked a little, talked with interesting people and generally had fun. After that, six of us continued to a pub in Stockholm, called Monk's Cafe, where we drank good Belgium beer and talked until our throats were sore. A very interesting day, no doubts.

A few weeks ago I noted that Sun in Sweden announced on the JavaForum meetup that they were going to have a full day Java conference in January. This conference is called JFokus and will have 4 different tracks with 4 speakers on each track. I will feature in one of these tracks, talking about testing Java code with JRuby and RSpec, and also how to build a DSL for doing database integration testing using Ruby. I believe it will be very interesting, and the rest of the day looks incredible nice too.

So, if you're in Sweden, Jan 30:th, look into it!

More information can be found at the official site: http://www.jfokus.se/.

Sometimes it feels like the progress I've done with the OpenSSL library is almost swallowed up by all the new functions I've found that needs to be implemented. It's an uphill battle.

implementation doesn't handle the But, I'm happy to say that the latest digression is finished. I have successfully implemented the X509_STORE-family of functions plus all dependencies. In the end I had to create a wrapper for X509Certificate and my own PEM-reader and PEM-writer, since BouncyCastle'sOpenSSL specific aux trust information that can be appended to these certificates.

But anyway, that means there is a Java implementation of X509_STORE, X509_STORE_CTX, X509_VERIFY_PARAM, X509_LOOKUP, X509_LOOKUP_METHOD, X509_TRUST, X509_PURPOSE and a whole slew of others too. About the only thing I didn't implement was X509_POLICY_TREE. That was just too much. I'll tackle that hurdle if it seems Ruby needs it.

So, what is the status then? Coming in at almost 10 000 lines of code there is only three real parts left to do. Or, three parts left that have MRI test cases, I should say. Since there are quite a few files not tested in the MRI implementation. Like DH keys. Wow. But I ignore that for now. The current goal is the OpenSSL::X509::Store-family, the OpenSSL::PKCS7-family, and the SSL-stuff, the rest of the way. So, wish me luck.

My interesting OpenSSL implementation exercise continues. I am now very close. Very, very close. I'm actually so close that SSLSocket and SSLServer actually works, provided that you use Anonymous Diffie-Hellman (which is suicidal, but that's another story). All of this have been submitted to my openssl-branch. What's missing is the X509-store and PKCS#7. And the X509-store doesn't really look good. Not good at all. It's needed for full SSL support. But the bad thing is this: there isn't any Java libraries that duplicate the functionality. Nada. At least not that I can find. The functionality needed is to read and write X509_store-formatted files and directories, to be able to add certificates and CRL's and to verify against these a certificate, based on various interesting OpenSSL rules.

I wouldn't say that I mislike OpenSSL. I wouldn't say that I hate it either. It's very impressive in many ways. But boy. It seems I have to port a substantial part of it to Java, and I'm not looking forward to it. I need to to do both a port, and add support for KeyStore and CertStore so the Java SSLEngine also can use the information. Will this be an interesting exercise? Oh yes.

So, without further ado, this is the plea of this blog post: If you know of any easier way to do this, please tell me. Now! (where "this" is the X509_STORE-family of functions.)

Every time I've written Java code lately, I've been painfully aware of how much unnecessary code I write every time. And most of this is Java's fault. This blog post is a very small thought experiment. TIJuAVA does not exist as software. Yet. If I someday have the time I would love to implement it, but there are more pressing needs right now.

So, what are the rules? Basically, all valid Java programs are valid TIJuAVA programs. Some valid TIJuAVA programs are not valid Java programs. Simply put, the main difference is that you don't need to declare a type for any local variables or member variables. Type declarations are only necessary in method declarations. You can declare local variables and member variables if you want to, and in certain very unlikely circumstances you will need too.

Let's take a very simple example. This code is taken from the JRuby source code, but I have added one or two things to make it easier to showcase:

package org.jruby.util.collections;

import java.util.ArrayList;
import java.util.Collection;
import java.util.Iterator;

public class IdentitySet {
 private items = new ArrayList();

 public void add(Object item) {
 items.add(item);
 }

 public void remove(Object item) {
 iter = items.iterator();
 while (iter.hasNext()) {
 storedItem = iter.next();
 if (item == storedItem) {
 iter.remove();
 }
 }
 }

 public boolean contains(Object item) {
 iter = items.iterator();
 while (iter.hasNext()) {
 storedItem = iter.next();
 if (item == storedItem) {
 return true;
 }
 }
 return false;
 }

 private Collection getItems() {
 return items;
 }

 private void something(java.util.AbstractSet inp) {
 val1 = inp;
 for(iter = val1.iterator();iter.hasNext();) {
 System.err.println(iter.next());
 }
 }
}

This code doesn't really show all that can be done with this approach, and if I were to show a real example, this blog would be unbearably filled with code. So, this is just a tidbit.

The TIJuAVA system would need to be implemented as a Java two-pass compiler. Basically, the first pass finds all variable names that need to have a type inferred, and then walks through the information it's got, basic on method signatures and methods called on the variable. In almost all cases it will be possible to come to one conclusion on which type to use. The compiler would then generate regular Java byte code, basically the same bytecode that would have been generated had you written the types by hand.

Of course, most people use IDE's to write code nowadays. Wizards and code generators and what not. So why something like this? Well, even though your IDE writes your code for you, it is still there, and you still have to understand it at some level. If not when writing, you would still need to read it. And boy does type declarations clutter things. Especially generics. And here is one interesting tidbit. Generic types would also be possible to infer in most cases.

Another thing that could be easily added is some kind of in-place literal syntax for lists and maps. This would be more like a macro feature, but the list syntax would mostly just be a call to Array.asList, which isn't to bad.

An objection that I anticipate is from people who think that the code will be less readable by removing the type pointers. This should be more of a problem when you have large methods, but everyone these days use refactorings so they won't have methods with a LOC over 20. And if that's the case, the local variables should be easily understood by the operations that are used on them.

So. Someday, when I have time, this may be reality. If anyone is interested, that is.

I just checked in a few updates to my openssl branch for JRuby. Boy is it tricky getting everything right. It seems like every DER format Java crypto emits differs from the OpenSSL DER output. And it's really incompatible. As an example I have been forced to reimplement the DER dumping for X509 certificates myself, and that's not the only place.

But the work is actually going forward; as fast as I can make it when I'm only doing this in my spare time and my regular work takes lots of time right now. I can't say for sure when it will be finished or usable, but I know for a fact that most of the MRI tests run now. What's missing is PKCS#7, X509 CRL's and X509 cert-stores, plus the regular SSL socket support. Not much, compared to what actually works.

But that leads to me to two issues. We have recently agreed that OpenSSL support will require BouncyCastle and Java 5. There is really no other way to get this working. 1.4.2 is fine for basic Digest support and some of the RSA/DSA support, but Java is sorely lacking in the ASN.1 and X509 department. Nothing whatsoever. Which is why we need BouncyCastle, which is fairly complete. I have only been forced to reimplement one or two central classes. Quite good. But SSL support is another story. As you may know, 1.4.2 has SSLSocket and SSLServerSocket. The problem is this: they aren't any good. As a first, they are blocking, and there isn't any support in 1.4.2 for NIO SSL sockets. Whoopsie. Which explains the requirement on Java 5. Tiger adds the SSLEngine class which can be used to implement NIO SSL, with the caveat that it heightens complexity. I have only taken a cursory look at this yet. Right now I want the other stuff working first, since there are so many dependencies on them.

But it's really going forward. Now, if I only had this as my day job, this would be finished in a few days... Alas, that's not the way it is. Expect further updates in a week or two.

I just found out something very disturbing and annoying. Java Property-files (and ResourceBundles in extension) will only load files in ISO-8859-1 format. Of course, you can use Unicode escape codes in the files, but that is not very convenient. Java is, over all, really good at internationalization, but this is one sore spot. And it doesn't seem to go away in Java 6 either. Isn't it highly ironic that ResourceBundles can't use regular UTF-8-files?