Archive for August, 2011

Mirah Office Hours: Bam!

Tuesday, August 30th, 2011

Last week was a busy week in Mirah land. We reopened github issues and consiliens awesomely went and moved over all the open issues from There were quite a lot of them and consiliens went through marking dups and closing issues he could not reproduce/were fixed. Plus, with the changes I checked in on Sunday during my office hours, the tests are passing again (on my machine anyway).

Office Hours.

I set high goals for myself and I met some of them. My main goal was to fix the issue where block arguments to a macro were scoped incorrectly and Bam! fix it I did.

I also wanted to start prepping for an 0.0.8 release, as well as fixing/verifying some of the issues that had been pulled over from google code.

On my second attempt to get blocks working with macros, I gave up on the deep dive approach and looked for the simplest thing that could possibly work. While I was diving in the code, I noticed that the block node was not a parameter. I’d figured out last week that all the parameters for the macro call were wrapped in a new ScopedBody, but I hadn’t realized that the block wasn’t and THAT was why look up was failing for it. Once I saw that, the answer was simple.

I fixed it by doing the same thing to the block body that was done to the rest of the parameters. I wrapped it in the correct scope before it got typed and that made the look ups work correctly. I’m not saying that my fix is the best way of ensuring that scoping works like you’d expect–but it is consistent with what the code was doing before, and isn’t a huge hack and it has tests–well one, anyway.

So, what does macro block scoping mean anyway?

Well, when you write a macro, you want to be able to reference variables defined where you called the macro from within the macro. Prior to the fix, this would only work with parameters. e.g. if you called my_macro some, expression, some and expression would be looked up in the outer scope. But this was not the case when you passed a block to the macro. In that case, the block would think it was in the macro’s scope. That meant that expressions in the block would look to the macro’s scope instead of where you’d expect–where the macro was called. That meant code like

a = 1
loop do
a += 1
break if a > 10

wouldn’t work because a += 1 would think it was in a different scope than a = 1, and wouldn’t know who or what a was supposed to be.

ARGV — Bout 1
I read this thread about command line args last week. It made me wonder how hard it would be to provide a way of accessing them in the body of a script. Right now mirah doesn’t do anything with additional commandline arguments. And, you can’t access them unless you are explicitly defining a main method on a class. I thought it would be nice to have an equivalent to Ruby’s ARGV in the implicit main method, for doing scripting. As things are right now, if you want to use commandline args, you need to define a main method explicitly:

class Main
def self.main args:string[] : void
puts args[0]

when what you’d like to do is more like

puts args[0]

which reads much better.

It turns out implementing that is a bit of a pain. The reason is because the implicit main method is not represented in the AST. Oh, the code that makes up the body of the implicit main method is in the AST, but it’s not there as a method body on a class. Generating the main method happens at code generation time, after the AST has been processed (more or less).

In order to access the argv in a script, the String[] args from Java’s public static void main(String[] args) need to be in the AST, or at least accessible to the typer when it is inferring the types of everything.

My first approach was to pull out all the AST nodes related to the implicit main(ie, everything not a class, interface, package definition, or import) and define a main method on the appropriate class with those nodes in the body. That actually worked ok, and allowed you to access argv because it was a parameter to the method whose body all those top level expressions ended up in.

I’m not sure this is the best way to do it, and it certainly isn’t the only one. But it was pretty straight forward. The problem I ran into was that in code like

puts args[0]

the class the main method hangs off of is never defined in the AST–which means you can’t define a new main method on it without creating it first. I didn’t get to that point. I don’t think it would be too hard though.

While, that is certainly a viable approach, you could also change the Script node so that argv returns something when you look it up. I’m not sure whether I like that better though, and I don’t quite know how I’d implement it that way, but it’d fun to try and see how it works out.

Mini Code Retreat at Rocky Mountain Ruby

Monday, August 29th, 2011

Tomorrow I’m helping facilitate a mini code retreat–one of the events for Rocky Mountain Ruby Conf.

It’ll be the same format as a full code retreat, only with half the sessions.

See you tomorrow!

Mirah Office Hours: Main Man

Thursday, August 25th, 2011

This last Sunday, I pulled the main method fix I’d worked out a few weeks ago into master. I threw out all the test refactoring I’d put together last week that didn’t work and came up with some better ones, using a better process. I also fixed the test suite so all the tests will run from rake test, even if there are failures.

My test suite changes aren’t perfect, but they get the job done. One aesthetic thing is that when there are failures in the jvm tests, you get messages for both the bytecode and javac test runs. It might be better if it just said there were jvm failures. But actually, thinking about it, it’s not bad as is because if there are errors in only one, you’d have a better idea of what to look for.

I also dove into the scoping code for macros to try to understand how to make them behave the way I thought they should. What I found is that there is already support for the behavior I’d like, but it’s hard to use from macros defined in Mirah rather than Ruby(read: I didn’t see a way, but there could be one).

Macros currently wrap their input nodes in what’s called a ScopedBody, that uses the outer scope for look up. Since the AST nodes determine their scoping by looking for the first scope in their parents, this works fine in most cases. But, when dealing with blocks it’s different. Blocks, (eg do … end) have their own scope so their lookup is handled differently. Macros implemented in Ruby can get around this by doing more AST manipulation, but that’s not particularly helpful when building macros that take blocks in Mirah.

I think figuring out how to make that work will be one of the things I’ll tackle this coming Sunday.

Mirah Office Hours: After the hiatus

Tuesday, August 16th, 2011

Lower Falls of the Yellowstone
I took last week off, I was a little busy vacationing and all. I had a very nice time and enjoyed not having internet or cell service. But this last Sunday I was back at it, hacking away on Mirah.

Getting Started

I started by pulling latest, because there had been some changes since the last time I checked out Mirah. Immediately I went from having 2 test failures to having 200 some errors with the message NativeException: java.lang.ClassNotFoundException: mirah.impl.Map$Extension4.

This had something to do with the bootstrap jar being upgraded, but I wasn’t sure what.
I tried updating my local Java from 1.6.0_22 to 1.6.0_24, that didn’t work–but it changed the error to RuntimeError: Compilation error.

I also ran rake clean and rake clobber and poked around for other stale files. It was weird because if I checked out the jar from before the change everything worked again.

Finally, I tried making a fresh clone of the repo and that worked–which I found a little strange. My suspicion is that somewhere I missed some build files that were screwing things up on the path, but I’m really not sure.

Somewhere in there I also blew away my rvm JRuby install and reinstalled it. When I did that, I also chmoded .rvm/hooks/after_use_jruby so nailgun wouldn’t cause issues like it did before.

Actually doing stuff

I looked at pull request #95, and couldn’t reproduce the issue it solved, so I left a comment to that effect.

On StackOverflow, I answered a question about Mirah’s metaprogramming support. I think I should work up a page like that to put in the wiki, so there is a canonical place to look for macro and metaprogramming information.

test reorg continues
I finished teasing apart the bytecode and javac jvm tests. Now, they run the same suite but use different helpers instead of sharing through subclassing. This means we can start to break apart the big test_jvm_compiler.rb file(2876 lines!) into more focused test files, which I think will be a big win for helping new people find a place to put new tests.

The only problem with the current implementation is that the rake test task is dependent on the sub tasks instead of calling them inside itself. This is a problem because the task stops at the last suite to fail, instead of doing like Rails does and running them all and telling you error in test:units or whatever.

I continued poking at the main method thing I’ve been working on, and managed to completely hose the test suite–clearly I’m doing something wrong somewhere. So I think I’ll take another crack at it next weekend. I think I was trying too hard to decompose the compile helper methods, rather than getting it working and then decomposing it.

Scoping Thoughts

I read the scoping discussion about macro scoping and formed an opinion about it.

The discussion boils down to how to handle scoping within macros. For macros to be useful, you need to be able to refer to variables from the outer scope. The problem is that just dropping the code the macro generates into the outer scope can be problematic when the code the macro generates has its own variables. Hmm, that explanation wasn’t great–how ’bout an example:

Here, if we use the outer scope without making temporary variables for the variables in the macro, we’ll get as output 4, which is probably not what we expected.

My Scoped Opinion

My thought is that anything assigned/declared to in the quote block in a macro should be turned into a variable local to the macro–eg foo’s bar becomes bar_1, or something. But, unquoted elements within the quote should use the outer scoping and not create temporary variables, so that when the macro is put in place, the function baz looks like:

That gets you the ability to reference things outside the macro, letting you manipulate them within it, but does not bleed temporary variables from the macro into the outer scope. Now, how to implement that, I don’t really have a good idea. An exercise for the reader.

I think it was a pretty productive four hours. See you next week!

Mirah Office Hours

Tuesday, August 2nd, 2011

This time I tried to tackle the improperly generated main bug that’s been a problem with java source generation for a while. It took me a couple of hours to figure out where the relevant code was because I’m not as familiar with that part of the compiler.

The problem was that when the main method was generated for a file with the same name as a class in it, the source for the class’s source code would be generated before the main method’s body.

huh? Lets see some examples

Say we have a file test.mirah that looks like this:

When we run mirahc -j test.mirah, we get this out:

Notice anything funny? The main method isn’t finished. This is because when we generate the main method, we compile the whole script–which includes the class. The problem is that we finish generating the class’s source code before we get to, leaving the main we added to the class unfinished.

My hackish solution to this was to check if we were generating a main or not in the class source builder, and not finish the class until the main method was finished. I did this by checking to see if we were in a main method generation in the class generator and not finishing the class source generation until after the main method was generated if that’s true. But of course, that by itself added new problems because you can have multiple classes in a .mirah file. So I added a klass method on the Mirah::JavaSource::MethodBuilder so I could check both whether we are in a main method and whether it is the main method of the current class.

After doing that, the generated code looks like this:

Much better

Since I haven’t written tests specifically for this yet, I put it on a branch on my fork of Mirah so I can get some other eyeballs on it. I’m also sure there’s a better approach than the one I took.

One other thing I found is that if the name of the .mirah file is different than any of the classes in it, it compiles correctly even w/o the patch. Which makes sense because in that case there is no class body for the class with the same name as the file.