Bruce Eckel on Build Systems

In "Why We Use Ant", Bruce argues why we should continue use Ant, even though it has some significant flaws as a build system for large, complex projects. His arguments are:

1. It’s never as easy as it seems [writing a replacement for Ant].
2. It’s probably not the battle you should be fighting.
3. Lots of people have tried and failed.
4. Politics matter.

Number four is actually a very strong argument, especially in a group doing Java development.

Let me quote an interesting passage in Bruce’s article.

Another important proglem with Ant is that its XML-influenced declarative syntax is limited. A build system is really a little computer program. Normally, you look for conditions (a source file is out of date with its target is the primary one for make) and you execute commands, but sometimes you need to do things that are more sophisticated, conditional logic, that kind of thinkg. Both make and Ant fall down here because, while it’s possible to do these things it suddenly gets a lot more complicated because that’s not normally what they do.

Exactly! It was my need to dynamically generate some make targets that started my original make rant that sparked the rake project. make, by itself, just wasn’t up to the task

Bruce continues …

I suppost if I were give a budget and a bevy of programmers, I would ask them to take make, which is open-source, eliminate the silly tab-vs.-space thing, and meld it together with Python, also open-source. make would only retain the simplest of its functionalites (primariy the way you set up dependencies and execute simple commands), and as soon as you wanted to do something more complex you’d be able to drop seamlessly into Python syntax, using any Python libraries you want. I think that would probably be my dream build system.

Bruce, you really need to check out rake. Its exactly what you describe above (well, if you substitute Ruby for Python). And it didn’t take a bevy of programmers. The core functionality was an evening’s worth of work.

Dropping seamlessly into Ruby is trivial, because a Rakefile is a Ruby program, specified in standard (although idiomatic) Ruby syntax. Since we never leave Ruby, we have no parsers to deal with, leaving a very small code base for the core functionality. The core of Rake (dependency management, task execution and rule matching) is less than 200 lines of code. The minimal rake install (a single file) is around 500 lines of Ruby code and adds common file commands (system independent versions of cp, rm, mv, etc.) and a flexible FileList object that mimics Ant’s include and exclude semantics.

Martin Fowler on Build Systems

Martin Fowler echos many of Bruce’s comments and reiterates that a build system really needs a full blown programming language. Martin has been dabbling in Ruby, and it seems that he has discovered rake and has been using in it some small projects. Great!

A Build Language Needs to be a Programming Language

Why should your build system support a full blown programming language? Here is one small example taken from a real situation at work.

When we are ready to install a project at work, we create a staging directory containing the files to be installed into an environment (e.g. testing or production). As problems are discovered in testing, corrections are made in a development area and copied to the staging directory for reinstallation.

The following makefile copies 2 jar files from the developement area to the staging area. Yes, I know that a makefile is probably overkill for this problem, but in real life there are other tasks as well. For example, using make (or rake) allows me to ensure that a refresh step always occurs before an install step, but only if the refresh is needed.

So here is the makefile …

  # Makefile to copy jars

  DEV=/home/jim/development
  JARDIR=jars

  refresh: $(JARDIR)/a.jar $(JARDIR)/b.jar

  $(JARDIR)/a.jar: $(DEV)/AProject/lib/a.jar $(JARDIR)
          cp $@(DEV)/AProject/lib/a.jar $@

  $(JARDIR)/b.jar: $(DEV)/BProject/lib/b.jar $(JARDIR)
          cp $(DEV)/BProject/lib/b.jar $@

  $(JARDIR):
          mkdir -p $(JARDIR)

Notice that there are several duplications in the file. The copy command for the two jar files are remarkably similar, yet different enough that you can’t capture both cases with a generic make rule. Also, knowledge about a particular jar file is spread out in two places; once in the copy rule itself, and again in refresh task line.

First we will translate the above Makefile into an equivalent Rakefile. See the Rakefile documents for details on the syntax of a Rakefile.

  # Rakefile translated directly from the Makefile

  DEV = "/home/jim/development"
  JARDIR = "jars"

  task :refresh => [ "#{JARDIR}/a.jar", "#{JARDIR}/b.jar" ]

  file "#{JARDIR}/a.jar" => [ "#{DEV}/AProject/lib/a.jar", JARDIR ] do |task|
    cp "#{DEV}/AProject/lib/a.jar", task.name
  end

  file "#{JARDIR}/b.jar" => [ "#{DEV}/BProject/lib/b.jar", JARDIR ] do |task|
    cp "#{DEV}/BProject/lib/b.jar", task.name
  end

  directory JARDIR

All the duplication from the original makefile still exists in the rake version. Lets remove it by writing a function that will generate the dependency (and copy command) based on the jar file name and the project name.

  # Rakefile with (some) redundencies removed

  DEV = "/home/jim/development"
  JARDIR = "jars"

  task :refresh
  directory JARDIR

  def make_jar_copy_task(jarname, project)
    stagejar = "#{JARDIR}/#{jarname}.jar"
    devjar   = "#{DEV}/#{project}/lib/#{jarname}.jar"

    task :refresh => [ stagejar ]

    file stagejar => [ devjar, JARDIR ] do
      cp devjar, stagejar
    end
  end

  make_jar_copy_task("a", "AProject")
  make_jar_copy_task("b", "BProject")

Notice that the refresh task is now declared without any explicit dependencies. The individual jar file dependencies are added to the refresh task inside the make_jar_copy_task function.

If we had a lot of jar files, we could put them in an explicit list and replace the last two lines with something like this …

  jarfiles = [ ["a", "AProject"], ["b", "BProject"], ["c", "CProject"] ]
  jarfiles.each { |jar, proj|  make_jar_copy_task(jar, proj) }

Or perhaps we have to deduce the jar files from the current contents of the JARDIR directory. (Assume that projects is a lookup table mapping jarfiles to project names.)

  Dir["#{JARDIR}/*.jar"].each { |jar|
    jarname = File.basename(path).sub(/\.jar$/, "")
    make_jar_copy_task(jarname, projects[jarname])
  }

Or perhaps we need to pull our jar file names from a database …

  DBI.connect(DBI_STRING, USER, PASSWORD) do |db|
    sql = "SELECT jarfile, project FROM install_table WHERE install_date = ?",
    db.select_all(sql, INSTALL_DATE) do |row|
      make_jar_copy_task(row['jarfile'], row['project'])
    end
  end

Ok, that last example might have been a little over the top. The point is that builds can become arbitrarily complex, and anything less than a full programming language is just doesn’t cut it.

Bindings

In Ruby, bindings are explicitly made available in a Binding object. Invoking the binding method will produce a Binding object for the current local variables.

Given that bindings are objects, one would think that there would be a number of methods available to get, set and query the names and their values. Unfortunately, in Ruby that is not the case. When you pass a binding to eval, Ruby will use the binding to resolve open variable references while evaluating a string. Other than that, bindings are pretty opaque.

But that still gives us some useful functionality. If you want to know the value of a variable in binding, just evaluate the variable name …

  eval "a", vars         # Evaluate the value of "a" in the binding "vars"

To change a variable’s value in a binding, use …

  eval "a = 101", vars   # Bind "a" to the value 101.

Bindings and Local Scope

Using the binding method, we can capture the variable bindings available at a particular point in the code, and pass that binding to a different part of the code to be used there.

Here we pass the binding of the current scope to a method that uses it. This is pretty straight-forward.

  def value_of_a(vars)
    eval "a", vars
  end

  def my_scope
    a = 33
    value_of_a(binding)
  end

  my_scope               # => 33

Bindings can be returned from a function as well. This example shows that a binding continues to exist after the function that defines the binding has exited.

  def f
    a = 22
    b = 33
    binding
  end

  f_vars = f()

  eval "a", f_vars         # => 22
  eval "b", f_vars         # => 33
  eval "a = 101", f_vars
  eval "a", f_vars         # => 101

The bizzare feature of this example shows that not only does binding persist beyond the scope of the function that created them, but that you can modify these bindings just by evaluating an assignment within their context.

Blocks and Bindings

A block in Ruby is a chunk of code that can be called like a function. The block automatically carries with it the bindings from the code location where it was created. For example …

  a = 33
  block = lambda { a }
  def redefine_a(b)
    a = 44
    b.call
  end
  redefine_a(block)      # => 33

The block returns the value of a in the binding where the block was defined (a == 33), not the binding where the block was called (a == 44).

This combination of code block and binding is called a closure, and is a very powerful tool in a Ruby programmers toolbox.

Swapping Values

Recently on the ruby-talk mailing list, someone asked about writing a swap function where swap(a,b) would swap the values of the variables "a" and "b". Normally this cannot be done in Ruby because the swap function would have no reference to the binding of the calling function.

However, if we explictly pass in the binding, then it is possible to write a swap-like function. Here is a simple attempt:

  def swap(var_a, var_b, vars)
    old_a = eval var_a, vars
    old_b = eval var_b, vars
    eval "#{var_a} = #{old_b}", vars
    eval "#{var_b} = #{old_a}", vars
  end

  a = 22
  b = 33
  swap ("a", "b", binding)
  p a                          # => 33
  p b                          # => 22

This actually works! But it has one big drawback. The old values of "a" and "b" are interpolated into a string. As long as the old values are simple literals (e.g. integers or strings), then the last two eval statements will look like: eval "a = 33", vars". But if the old values are complex objects, then the eval would look like eval "a = #<SomeObject:0x401fef20>", vars. Oops, this will fail for any value that can not survive a round trip to a string and back.

Blocks as Getters and Setters

Let’s try a different approach to the "swap" problem. Instead of passing variable names and bindings to the swap function, let’s pass closures that do the work for us …

  def swap(get_a, get_b, set_a, set_b)
    temp = get_a.call
    set_a.call(get_b.call)
    set_b.call(temp)
  end

  a = 22
  b = 33
  swap(lambda{a}, lambda{b}, lambda{|v| a=v}, lambda{|v| b=v})
  p a           # => 33
  p b           # => 22

Wow! This works great! The values are swapped and arbitrary values are supported. The only down side is that the call to swap is extremely verbose.

Abstracting the Getter/Setter Code

We can shorten this by creating a getter/setter abstraction called a reference. To create a reference, we will need to supply the name of a variable along with a binding. But this is what we want the code to look like.

  a = 22
  ref_a = Reference.new(:a, binding)
  p ref_a.value        # => 22
  ref_a.value = 33
  p ref_a.value        # => 33
  p a                  # => 33

So changing the value of a reference should change the binding of the variable the reference refers to.

Here is a first pass at writing the Reference class …

  class Reference
    def initialize(var_name, vars)
      @getter = eval "lambda { #{var_name} }", vars
      @setter = eval "lambda { |v| #{var_name} = v }", vars
    end
    def value
      @getter.call
    end
    def value=(new_value)
      @setter.call(new_value)
    end
  end

Note that we create lambdas to handle the getting and the setting of values. This avoids the string interpolation problem mentioned above. Since we create the lambdas in the scope of the initialize fuction, we need to explicity pass in the calling scopes bindings so that the lambdas are created in the environment where are variables are defined. Leaving off the binding to eval will cause our getter and setter lambdas to modify the values of "a" and "b" in the scope of initialize instead of the calling scope.

Writing Swap

We are almost ready to write our swap function. First, let’s create a utility function (named ref) for creating references.

  def ref(&block)
    Reference.new(block.call, block.binding)
  end

ref takes a single block that returns the name of the value we are refering to. Since blocks automatically carry their binding with them, we get both the variable name and binding in a single argument. We use ref like this …

   aref = ref{:a}

Now swap can be written like this.

  def swap(aref, bref)
    aref.value, bref.value = bref.value, aref.value
  end

  a = 22
  b = 33
  swap(ref{:a}, ref{:b})
  p a                       # => 33
  p b                       # => 22

Summary

Using bindings, blocks and closures, we can have a great deal of control over the binding of names to objects within any scope. The basic ideas for the Reference class were developed in response to a Scheme/Python question on the C2 Wiki (see c2.com/cgi/wiki?SinisterSchemeSampleInRuby).

When I was first learning about programming (mumble mumble) years ago, variables were explained to me as shoeboxes. A variable name denoted a location in memory (a shoebox) which can hold things. You put values into the shoebox and they stay in there until you put something else in there. Each shoebox is exactly the right size to hold the type of data you put into it. Integers fit in int-sized shoeboxes and floating point numbers fit in float-sized shoeboxes. Pointers can be added to this model by envisioning labels (addresses) for each shoebox and storing a shoebox label in another shoebox.[1]

Assignment is the act of copying a new value into one of your shoeboxes.

This model works really well for understanding FORTRAN (the language I was learning at the time) and Algol, and reasonable well for C and C++. HOWEVER, this is exactly the wrong way to think about Ruby variables.

Consider the code …

    a = b = "Hello"

The shoebox model encourages us to think that the string is in the ‘a’ shoebox and another string is in the ‘b’ shoebox. Therefore we are surprised when modifying a changes b. To explain this within the shoebox model, we resort to talking about references and how the shoeboxes ‘a’ and ‘b’ don’t actually hold the string object, but hold a reference to it.

All of this is perfectly correct, but I believe there is a better model to use for Ruby variables.

In Ruby, the variable ‘a’ does not represent a shoebox that can store things. Instead, it represents an association between a name and an object. We call this association a "binding". Executing the code …

    a = "Hello"

binds the name ‘a’ to the string object "Hello". References to ‘a’ after the assignment will lookup the name ‘a’ in a dictionary of bindings and will find the object currently associated with ‘a’. We no longer need think of variables as shoeboxes containing objects, but as names associated (bound) to objects.[2]

I see a number of advantages to this point of view.

1. The distinction between immediate values (like Fixnum) and reference values (most everything else) becomes invisible.
2. Multiple assignment statements (like ‘a = b = "Hello"’) is explained in terms of binding the names ‘a’ and ‘b’ to the same object. No need to talk about pointers or references.
3. Assignment becomes an operation that is outside an object. Objects don’t care what names they are bound to. Therefore binding is not an operation on an object and won’t ever be a method on an object. This helps explain why "+=" is not a method and why "++" can’t be implemented as a method.

I found that adopting this view has helped me understand Ruby better. I hope it helps you as well. Comments and feedback are encouraged.

Footnotes

1. I used this technique to explain pointers when I taught a class in C, except I used styrofoam cups with labels instead of shoeboxes. I would write on yellow postit notes and drop them into the cups (variables). Each cup had a label, so to represent a pointer I would write another cup’s label on a postit and put it in a cup representing a pointer variable. It was actually a rather effective in demonstration.
2. Variables as name bindings is not unique to Ruby. I first encountered the concept when learning Lisp (what I learned after studying FORTRAN). I’ve also read threads in the Python newgroup where someone was offering a similar explaination of variable binding for Python. I’m sure there are many other languages where binding is a better model than shoeboxes.

Conference Introduction

Its Friday morning in Austin Texas, and the 3rd Annual International Ruby conference is just beginning. David Black welcomed everyone to the conference. He spoke a few words about RubyCentral, Inc, the organization that sponsors the conference.

Working in the Garden: Web Apps with Borges (Eric Hodel)

At last year’s conference, Avi Bryant demostrated an unusual web application framework that allowed linear programming to be applied to the rather non-linear web. Avi’s ruby framework was a partial port of his Smalltalk framework, Seaside. Seaside uses continuations to allow a user to return the application to an arbitrary earlier point in the program and to make a different decision.

Eric Hodel has done a complete port of Seaside to Ruby. The result is named "Borges". Eric demoed Borges by modeling SushiNet, a sales cart style application for ordering raw fish.

The Seaside/Borges model for web applications is really interesting. It seems to be a good fit for certain types of applications, especially for those that lead the user through a certain set of pages (e.g. the checkout pages for a web shopping site). Now if I could just get my head wrapped around continuations.

The State of XML Processing in Ruby (James Britt)

James points out that using XML these days is much like using ASCII, you really don’t care about the exact representation. You just use the APIs without worrying so much about the gory details. James gave us a survey of the currently available XML tools for Ruby, noting whether they were pure Ruby or C library based, whether they were streaming or DOM like, and gave a summary of there use. I’m not going go through his entire list, it was quite long. If fact, I was a bit surprised at the length of the list.

Object-relational mapping with Lafcadio (Francis Hwang)

Lafcadio is a OO mapping tool, allowing one to deal with objects in their Ruby code while accessing a relational database. Francis wrote Lafcadio for his own use, but appears to be fairly complete.

Lafcadio is specifically designed to support unit testing. Francis uses an interesting mechanism for flexibly controlling singletong instantiation in the framework.

Here are a couple of notes

• I would like to see Lafcadio support more databases. Francis indicated that it would not be hard to add DBI support, but currently MySql is the only supported out of the box.
• Francis talked about better abstractions for queries, but wasn’t aware of Ryan Pavlik’s Criteria library (see my Leaky Abstractions writeup. I think this would be a good match for Lafcadio.
• I asked Francis later about the origin of the name "Lafcadio". He indicated that it came from a children’s book by Shel Silverstein about a lion of the same name. Lafcadio the lion grew up in the jungle, but was transplanted into the world of men. Lafcadio longed for the jungle, but was unable to return after becoming acclimated to his new world. Lafcadio becomes a metaphor for the strange between world of objects coming from a database, but living in the world of objects.

Ruby World: Implemented (Steve Tuckner)

Steve intended to talk about the implementation his Ruby World project, but implementation of Ruby World was delayed. Since David Black (one of the conference organizers) wouldn’t let him back out of speaking, Steve decided to talk about using Ruby in a commercial project.

And that’s a good thing. Steve’s company is developing a fax device controller and Steve is using Ruby to write the PC-based scheduling and control software for outbound faxes. This is a native windows application that will actually ship with the product (the fax controller is not yet shipping).

It was refreshing to hear about a Ruby project that is product related.

Some random notes:

• Uses: Ruby, Vruby/Swin, win32ole, exerb
• Threads on windows are problematic and can suck 100% of the CPU. There was some discussion of how to solve this problem using separate ruby processes and allowing them to communicate within the box.
• Exerb will occasionally GP when built on certian machines. This problem has not reoccured since switching to Ruby 1.8.

As a wrap up, Steve shared some techniques for testing GUI code.

Generating Code in Ruby (Jack Herrington)

Jack is the author of Code Generation in Action. Jack talks about using code generation for anything from small one-liners to large MDA-style generators.

Some advice from Jack:

• First hand write the code that you will eventually generate.
• Generate only what you understand (avoid the wizard syndrome).
• Maintain the generator as part of the build process.

I only disagreed with Jack on one point. Jack seemed in favor of guarded blocks in generated code. Guarded blocks are areas where developers are not allowed to edit. I personally have a great dislike for guarded blocks and would rather have the generated code be in a completely separate file, but that’s just me.

I don’t have much details here, buts that’s probably because I was doing more listennig than note taking at that point in time.

Roundtable with Matz

Chad Fowler introduced Matz and opened the floor for questions. Matz set the ground rules in that he will talk about Rite and Ruby 2.0 tomorrow night. So no Ruby 2/Rite questions for tonight.

• Question: When is Ruby getting static typing?
  • Matz: Ummmmm .… I’m actually thinking about optional typing in Ruby 2.0, but based on signature rather than classes.
  • Comment: Strong Duck Typing?
• Question: What is that status of a packaging system for Ruby? I know you are a benevolent dictator, but perhaps this is a where we need less benevolence and more dictator.
  • Matz: I maintain Ruby itself but do not implement
  • Comment: But perhaps you are the right person to choose an implementation or the person to work on it.
  • Matz: Perhaps we can setup a Wiki and gather requirements.
  • Moderator: Perhaps

    Aside: This packaging question turns out to be one of the more significant questions asked during the roundtable. It caused a handful of us to lose a great deal of sleep over the weekend. More on this later.

• Question: Can you explain the reasoning behind the proc and Proc.new argument semantics?
  • There is a lengthy discussion about the difference. In the end, Matz agrees to deprecate "proc" in favor of "lambda" in order to differentiate block vs method argument semantics.

    Aside: I’m not sure I’m happy with this. I tend to use proc much more often than lambda, but I guess I can adjust. I suppose this shows what can happen when you can bend Matz’s ear.

• Question: What is the news about the Ruby language specification?
  • Matz defered to Rich Kilmer who talked about the possibility of using a stand-alone parser to be the arbitor of syntax.
• Question: Is the C API considered part of Ruby?
  • Matz: No, new versions might not support the current API, but I hope to provide some kind of backward compatibility.
• Question: Is there someway we can get more of the Japanese web pages translated to english?
  • Matz: I don’t know.
  • Comment: I’ve had good luck with Babelfish.
  • Comment: I have the a copy of the Ruby Hackers guide. Its in Japanese. It’s a great book, I haven’t read it.
• Question: What are you doing as the author of Ruby to ensure that Ruby doesn’t change and break are existing code?
  • Matz: Ruby 2.0 will be different from Ruby 1.8 and there will be some incompatibilities between the two, but Ruby 1.8 will remain as it is. Ruby 1.9 will generate warnings for future incompatibilities.
  • Comment: There are some advantages to being small and having a smaller installed base in that we can evolve the language
• Question: What is being done to improve Ruby on the Win32 platform?
  • Matz: The Win32 folks are working on issues to make the platform better. Perhaps we need to get away from standard IO on that platform.
• Question: Is Ruby going to follow Perl 6’s lead in regular expressions?
  • Matz: I have no plan to follow Perl6 in regular expressions, however if someone can convince me of a need for a particular feature I might change my mind. Ruby 1.9 will have a new regular expression engine that will unencumber it from license problems and gives better performance.
• Question: Would it be possible to optimize string and regex interpolations when the interpolated data is static?
  • Matz: Ruby optimization is very difficult. I have considered many things and I have rejected many things. If you have some good ideas, I would like to consider them.
• Question: What is the status of the local / block variable issue?
  • Matz: In Ruby 2.0, all block variables will be local to the block.
• Question: Do you get bored working on Ruby for 10 years?
  • Matz: I’m not bored yet.
• Question: What was the motiviation for using "#" for both comments and string interpolation?
  • Matz: Ummm … I don’t remember.
• Question: People at work ask me why they should use Ruby instead of Perl. What should I tell them?
  • Matz: I’m not the right person to answer that because I am not a Perl programmer.
  • Comment: It’s like the Matix. No one can tell what Ruby is, you have to experience it.

At this point people started sharing Ruby stories in the workplace. Matz closed by saying that we will review changes for Ruby 2.0 tomorrow evening. Yeah!

Post Meeting Talks

The package question that came up during the roundtable touched a nerve with a number of us. Rich Kilmer, Chad Fowler, Paul Brannan and I (Jim Weirich) headed for the hotel (Club Max) and setup our laptops in a corner (near an outlet). Chad had a cellular internet connection so we were able to pull down a number of packages and previous attempts for review. We finally decided we would hijack the RubyGems project started by Ryan Leavengood. Ryan presented RubyGems at the very first Ruby conference in 2001 and got a favorable reception. Unfortunately the work was never continued and the project languished. So, Ryan, if you are out there, we hope you don’t mind.

We setup the following rough goals:

• Single File, System Independent Distribution Format
• Simple installation
• Explicit versioning, possibly multiple version installed at the same time.
• Easy downloading, something like apt-get.

I don’t recall enumerating the goals in this form, but they evolved from the discussions as the night progressed. In particular, we waffled on installing gems or loading files directly from gems. We finally decided on going through an installation because shared native libraries couldn’t be loaded directly from a gem file.

That first night, Rich and Paul attacked the package format and metadata required for a gem. Chad and I started looking at compression libraries and packaging techniques. Zlib is now packaged with Ruby 1.8, so it will be easy to compress the individual files. Standard zip files were considered, especially when we found a zip file library in the RAA. But eventually we decided on a self-extracting format that begins with a Ruby script to do the extraction.

By the end of the evening my mind was turning to mush. Fortunately the bar began turning out lights at 2:00 am. I headed off to bed, but I’m pretty sure Rich kept going for a bit beyond that.

Next: One day down, Two to go …

Programming LOGO MINDSTORMS with Ruby (Shashank Date)

Having seen a LEGO Mindstorm robot at our CLUG meetings, I was very interested in seeing how Ruby works with the robots. The guts of the robot is the RCX brick that has inputs and outputs connected to the "nubs" on the top of the brick. Sensors and motors connect to the brick by snapping a sensor or motor block onto the nubs of the RCX brick.

The RCX brick sports 16 KB of ROM for BIOS-like functions and 32 KB of RAM containing a byte code interpreter. There is about 6 KB available for byte-coded user programs.

Shashank’s eleven year old son, Rutvik, demostrated the graphical "non-programming" environment by writing a simple program that moved the robot back and forth, switching directions on inputs from the light sensors and touch sensors.

Rutvik’s school has four dedicated PCs for programming the LEGO robots. To work with a particular robot, you must be sitting at the dedicated PC for that robot. Shashank’s goal is to use DRb to allow any PC in the lab to download programsm to any of the robots.

Using a second machine, Shashank ran a Ruby version of the program Rutvik used. The Ruby program on the client (rather than the RCX brick) and sent commands via DRb to the server machine, which in turn sent the command to the RCX robot via the IR tower.

…aside (David Black)

"Every conference needs a low point, and I’m always a team player". David confesses that he is standing in for a speaker who had to pull out of the conference. So David presented a program called aside that allows students to provide anonymous feedback to professors, and to allow professors to interact with anonymous students.

The purpose of David’s talk was not to present the program, but to open a discussion of applications vs frameworks and how to get some "half-baked" ruby programs out in the public eye. We discussed the role of RubyGarden and other web sites (e.g. CodedBliss) in the community. We had to cut the conversation short, but I’m sure the lunch time conversion

Building Environments for MUES (Michael Granger)

MUES stands for Multi-User Environment Server. MUES is the server for the FairyMUD project since 2000. It is an event based dispatcher for creating multi-user games. I’m really not up to speed on MUDs and such, but Michael’s design looked very clean and deep.

A Clueless Ruby Hacker Explores Security (Nathaniel Talbott)

Nathaniel talked about some basic issues around securing a DRb server using OpenSSL and other techniques. He stresses that security must be considered at a system level, not merely at a product level.

Teaching Ruby to Testers (Bret Pettichord)

Bret teaches a Scripting for Testers course with Brian Marick. They believe that some programming is required for automated testing, but that the programming should be easy. That’s why they use Ruby!

They take folk who have some programming experience, but that experience is stale. They might be familiar with procedural programming, but not with some of the newer paradigms. Bret and Brian use an Internet Explorer controller to programmatically control IE to interact with a web application. Bret demonstrated the code interactively.

Introducing Ruby to the Corporate World (Jim Freeze)

Jim’s perfect job involves getting paid to develope in Ruby (and not having to program in <fill in the blank>). Seriously, Jim shared some ideas for promoting Ruby in the workplace, something that most of us have dealt with in some for or another. Jim stresses the need to be honest about Ruby and its capabilities.

Jim has put together a course on Ruby for his company and is teaching it at several locations across the country.

Jim’s company also provides awards for "Productivity Wins", suggestions that save the company money. It was interesting to note that Ruby was involved in two of the first four wins in this contest and a third win copyied the architecture of a prior Ruby win.

Visions of the Future - Keynote Address (Yukihiro Matsumoto)

Finally, the moment we’ve all been waiting for, to hear news about Rite and Ruby 2. After an (short) introduction by David Black, Matz begins by saying that Ruby is "Good Enough" and that’s why we are here. So the real topic is "How Ruby Sucks", and what we will be doing about it.

First of all, Matz drew the distinction between what is Ruby 2 and what is Rite. Ruby 2 is the next version of the Ruby language. Matz plans to take advantage of the major version upgrade and introduce some things to clean up the language that are not necessarily backwards compatible. Rite refers specifically to the VM for Ruby 2.

So, with that in mind, here are some things that may by in Ruby 2.

Note:

The following is a paraphrase of the information Matz provided. The talk generally moved too fast for me to capture all the details, but I tried to capture the overall gist of the topic. If there are any inaccuracies, it is entirely my fault.

Note 2:

I discovered later that Chad Fowler had logged onto IRC during the keynote talk, and was streaming this same information into the IRC channel. Someone not at the conference was then taking the IRC information and updating the Ruby wiki pages in real time. That’s pretty cool.

• Local Variable scope. Block parameters will be block local, hiding any existing variables. Also, local variables created by "eval" will
• Instance Variables. @_foo will be truely private (not protected). Or maybe @foo will be private and @_foo protected. Matz hasn’t decided yet.
• Constant Lookup. New rule will check enclosing class, then all superclasses, then one level of nesting classes. This simplifies the rules for constant lookup.
• Class Variables. Now class variables will will be local to the class or module. Use accessors if you wish to access class variables from a larger scope.

  Question:

  How will class variables be different from class instance variables?

  Answer:

  Class instance variables are normal instance variables of the class object. Class variables will be more like static variables that are scoped by the class.

• Statements and Expressions. No implicit string concatenation (use "+"). Parenthesis will cause line continuations. Statement grouping by parenthesis will be disallowed, use begin/end instead.
• Multiple Values. Multiple value assignment is confusing. We can either separate arrays and multiple values strictly by syntax. Or we could use a subclass Array::Values < Array to return values. Matz hasn’t decided which solution he will go with. (There was a lot of questions and comments on this item).
• Method Visibilities. Private may be in a separate namespace from public/protected.
• Range in Condition. This is confusing. No one remembers the exact semantics (even Matz)
• Keyword Arguments. Finally! Here’s the scoop:

    def foo(a, b: 42, **keys)
      p [a,b,keys]
    end
    foo(1)               => [1,42,{}]
    foo(2, b: 5)         => [2,5,{}]
    foo(3, b: 4, c: 6)   => [3,4,{:c=>6}]
  

  Some Smalltalkers wanted an optional colon after foo. Matz said no.

• New Hash Literal. A new form of Hash literal will be available.

    { a: 1, b: 2, c: 3 }  == { :a => 1, :b => 2, :c => 3 }
  

• Method Combination. Pre, Post and Wrap method combinations will be available. These methods are modelled after the CLOS before/after/around modifiers.

   class Foo
     def foo:pre();  p "pre"; end
     def foo:post(); p "post"; end
     def foo:wrap(); p "wrap pre";   super;   p "wrap post"; end
     def foo();      p ;foo"; end
   end
  

  prints …

   wrap pre
   pre
   foo
   post
   wrap post
  

• Selector Namespace. It will be possible to modify the behaviors of selections within statically scoped namespace. Details are subject to change.
• Optional Type. Current strong typing experiments are class/module based. Matz is exploring options, but wants any solution to be signature based and efficient.

Matz is interested hearing about ideas, so he is encouraging RCRs (Ruby Change Requests) for a limited time (say March 2004). He wants more than mere "could we change it this way" type of requests, but RCRs should contain an abstract, motivation, proposal and rationale. Mats demonstrates an "proper" RCR#1 with the proposal to remove "proc" from the language.

After talking about Ruby 2, the language; Matz moved on to talk a bit about Rite, the VM for Ruby 2. Ruby 1.9 will be a testing ground for some items targeted for Rite. It sounds like the big push will be for dealing with M17N. The new RegEx engine (handling M17N) will be in 1.9.

You can find the slides at: www.rubyist.net/~matz/slides/rc2003/

Post Conference Activities

It was back to Club Max for another night of hacking on RubyGems. The group continued to work on it throughout the day during lunch and breaks. This evening David Black joined us.

Friday night we worked without a net … well, network. But tonight someone had enough presence of mind to snag Shashank’s network hub. Imagine this, four networked laptops in the corner of the bar with cables strung across several tables. The local club hoppers took it all in stride.

Tonight was an integration night. David supplied a version comparison module that he had worked on during the day. Rich integrated it with the code base on his laptop. I setup a CVS server on my laptop made it available to everyone. We did eventually check into that CVS system, but because of the small number of files, it was difficult to split up work in a meaningful way. Rich went ahead with the the software to create and extract gem files. Chad concentrated on the remote download and install portion. He even got a simple WebBrick server set up to serve out gem files loaded on a system.

What did I do? Well, my presentation was first thing in the morning, so I took the opportunity to finish that before doing anything else. I was able to integrate the Gem creation software into my general package task library in rake. I even slipped in a last minute slide into the Rake presentation showing how to generate gems.

It is really interesting to see how other developers go about writing software. I had paired with Rich earlier that day and the man is a code generating machine. Rich didn’t to TDD (at least not while I was pairing with him), but every step he takes is incremental and well tested by hand. He really knows how to break up a programming session into small tasks and tackle them one at a time and then put together the pieces into the final whole.

The night broke up a bit earlier than the night before. By the time we broke up, we had the beginnings of an actual working system. It was pretty exciting.

Next: the final day of the conference.

Building with Rake (Jim Weirich)

I did manage to squeeze in a single slide showing the creating of a GEM file from a GemSpec. This was a teaser from the RubyGem folk that Rich elaborated on later in his talk.

Toward a Refactoring Framework for FreeRide (Hal Fulton)

Hal shared some of his ideas for a Refactoring framework for FreeRide. He has been experimenting with the Scientilla editor widget and has some (very) rudimentary refactoring and code assists running. Truthfully, Hal was limited by the lack of a good Ruby-based parser for Ruby. Without that, a full featured rectoring browser will have to wait.

The discussion that followed Hal’s talk centered on the need for a good Ruby parser. There was some talk about Ripper, which is a port of the original parse.y file from the Ruby source base, but replaces explicit parse actions with a callback/event based mechanism to allow something other than the Ruby interpreter to use it. FreeRide is using this parser and there is some hope that it will be integrated back into the Ruby source tree so there will be only one parser. I can think of serveral projects that would benefit from this.

Ruby and the Java Debug Wire Protocol (Rich Kilmer)

I don’t think Rich could decide what topic he wanted to present at the conference this year. The main thrust of his presentation was about using Ruby to control Java VM via the Java Debug Wire Protocol. Rich uses a ruby based monitoring script to control the execution of over 100 (large) JVMs.

One thing that I found particularly interesting was how Rich generated the code for the low level debug packets. The JDWP is a binary protocol, with around 70 different packet types. There is an HTML writeup from SUM detailing all these different packet types. Rich transformed (by hand) the descriptions of the packets in HTML to more or less isomorphic Ruby code. In other words, He described the packets using Ruby instead of HTML. He then wrote a generated that loaded the Ruby descriptions of the packets and generated code to parse, send and receive the packets. The Ruby description and generator were several hundred lines of code, but the final generated code base was well over 1500 lines. Not a bad trade-off.

I’ve been thinking about Ruby as a declarative language for a bit. See onestepback.org/articles/buildingwithrake/appa.html for some of my thoughts in the area.

When Rich got to the demo portion of his talk, he pulled up a terminal window and tried to run his program which immediately failed because one of the Ruby JDWP libraries were not installed. "Oh look" says Rich in a mock serious voice, "I’ve got a GEM for that library here." Rich runs the GEM which installs the library. He then tries the demo again, but it fails because it is requiring the wrong version of the library. Rich fixes the version problem and the demo runs without a hitch.

What you missed if you blinked was that Rich inserted a mini-demo of the RubyGems project into the middle of his Ruby-JDWP demo. Although sneaky, the next part of the demo should he had even more tricks up his sleeve.

The demo consited of a JVM running some Java-2D demos and a GUI listing the JVM threads and command buttons to start and stop individual threads or the entire JVM. The GUI was implemented by Ruby code, and although I couldn’t place the toolkit used to generate it, I didn’t really think too much about it.

After the demo, Rich revealed that the GUI was done using the Alph project (Rich had been dropping hints about Alph all weekend long). The GUI was drawn in a Flash component and communicated with a controlling Ruby script that specified the components and buttons to appear on the page and then handled the callbacks when those buttons were clicked. The Flash portion of the GUI is a single file that implements all the components, but lets the Ruby script completely control it. You need the proprietary MacroMedia development kit to create that Flash file, but once created it can be used by any Ruby script. In other words, you can develope your GUI completely in Flash without using the proprietary Flash development kit. Cool!

Why is it named Alph? It comes from the poem:

  In Xanadu, did Kubla Khan
  A stately pleasure dome decree
  Where Alph the sacred river, ran
  Through caverns measureless to man
  Down to a sunless sea.

Rich was thinking of a GUI Utopia, and Alph the river runs right through that utopia. One of my favorite quotes from Rich during the talk:

"This is pretty much the fun thing you do in Open Source, is think of names." — Rich Kilmer

Post Conference Activities

The presentation portion of the conference ended around 11:30. Lunch was to be served at noon, so I hustled up to my room to check out of the hotel. After lunch, several of us decided to head to the airport to take advantage of the WIFI network, even though our flights were not scheduled for several hours.

At the airport, Chad and Rich put some final touches on the RubyGems code and checked it into RubyForge CVS tree. I did some cleanup on Rake and as soon as they check RubyGems in, I updated the copy on my laptop and generated a GEM for Rake 0.2.9. I then uploaded the GEM to the file area of RubyForge allowing me to claim the first GEM enabled application on RubyForge (we are ignoring for the moment that I forgot to use a "require_gem" in the source, so the fact that the Rake GEM won’t work out of the box is a small blight on that claim).

Rich and Chad’s flight was about a half an hour before mine, so after they left I hung around the ruby-lang IRC channel and answered some questions on RubyGems. I even started the gem_server on my laptop and allowed a couple people to download the Rake GEM directly through the gem_server before I to pack up to make a plane. What a great way to cap off the whole weekend.

Late Update: Presentation Source

Ryan Davis is hosting all the 2003 RubyConf presentations. You can find them at www.zenspider.com/Languages/Ruby/RubyConf2003.html

Speaking of which, I’ve uploaded the "Building with Rake" presentation I gave this morning at the conference. You can find it here.

I’ll get my notes from the conference upload once I have a chance to review them and polish the words a bit. Watch this space.

I’ve always liked Fowlers "UML Distilled" book because it drilled down to the really important parts of UML without getting caught up in a lot of the superfluous annotations. I definitely fall into the UML-as-sketch category. Now I wonder where UML is going.

If anyone has a heartache over losing Sys, I’ll provide an optional Sys module in a contrib directory. But for now, Sys is being deprecated.

I just love it when I can delete code!

The project is moving slowly. It is very difficult to come in once a month and work a few hours on a project, and then leave it for another month. We spend a good portion of the meeting remembering what we did last month, and just as we get up to speed, the meeting is almost over. It’s a good thing that the goal of the group is more to learn XP practices than to actually finish this project.

Tonight we concentrated on continuous integration. As soon as we had a new JUnit test working, we were to get the integration token and commit to our CVS repository.

If you are in the Cincinnati area and want to join us, feel free. We meet on the first Tuesday of the month. You can find more information here.