Baroquebobcat

This week I decided to focus just on bugs. I picked a few that looked interesting and set about investigating.

I decided to look at #108, #150, #151, #68 and #52.

108 I realized, I hadn’t thought enough about. It’s really a design thing. Do you want to require that your closures match signatures with the methods the implement? What if you don’t care about the arguments because everything you want to do has to do with the closed scope and not with what you’re passed? I could see it both ways. On the one hand, requiring a matching signature ensures all the types and whatnot are correct. On the other hand, if you aren’t using the passed arguments, they are just boilerplate code.

I think we should allow closures to have no arguments where the method they are implementing has them for niceness. But it should be an all or nothing thing. Either no parameters, or parameters matching the signature of the method you expect to be implementing.

#52

I spent most of my time on this one. It seemed straight forward at first, but had a number of tricky bits that took some working out. The problem was that if you had a begin rescue end block, and treated it as an expression if the result types of the block itself and the rescue clauses differed, Mirah wouldn’t raise an exception, but the JVM would. For example (gist):

foo either is an int or a String, which would be fine in Ruby–but not in Mirah because Mirah has static typing.

So, I set about fixing it. The first thing was to define the valid behavior. What I came up with was this:

• if a begin rescue block is treated as an expression, check the compatibility of its result types
• if it’s not an expression, don’t check the result types

The second one is nicer because it lets you not care when you don’t want to care. For example (gist):

If we checked result types in both expressions and statements, the above would fail to compile, which would suck.

I started by writing a couple tests of the expected behavior. I wanted to get an inference error when I used the rescue as an expression, but not as a statement.

Then I went looking for other, similar pieces of functionality that I could use for a guide. I found that if elses also follow this pattern, so I wanted to look at how If AST nodes were inferred, trying to get a handle on how I would change rescues. I grepped through lib/ for If’s infer method and found it in lib/mirah/ast/flow.rb. What it does is get the if body’s result type, and the else’s result type and check to see if they are compatible (gist).

Rescue inference was a little more complicated than If because rescue’s have a lot more pieces. First, you have the body between the begin and the first rescue. Then, there can be multiple rescue clauses, each handling a different set of exceptions. Finally, if there’s an else, it runs after the body if there is no exception. There’s also ensure which is run after, but doesn’t return anything, so it shouldn’t affect result type comparisons (need to write a test for that …).

So, the types that need to be compatible are the that of the body if there isn’t an else or the else’s and all the rescue clauses. Kind of complicated. But pretty doable. I modified the Rescue node’s infer method to test compatibility of the body or the else and all the clauses, which worked for the tests I wrote but broke other tests.

It broke other tests because of how deferred inference works. When inference is reattempted, the typer assumes that the deferred node was evaluated as an expression, which breaks when it is a Rescue because its inference depends on how it was used.

So as a work around, I added an ivar to the Rescue node to cache how it was initially inferred (lib/mirah/flow.rb).

It’s a hack, but it resolved the issue. I’d like to look into how to change things to make expression vs statementness be something defined in the AST instead of decided at inference time, or make it more explicit that the expression-ness is determined by the parent node during inference.

This week I wanted to get back to my pet Sinatra clone–Shatner. Last time I tried working on it, I was trying to figure out how to get views working like they do in Ruby. I want to end up with a dsl like this(gist):

Where you can call edb :view_name just like you call erb :view_name in Sinatra, and it figures out what you want.

I tried to do this by injecting a call to the def_edb macro into the class’s AST node. That didn’t work. Because def_edb was getting expanded before the outer macro, instead of passing it a proper AST node representing a method name, I was passing it an Unquote, which it didn’t know what to do with.

This looked like a rather difficult thing to untangle, so I gave up on it for now and file a bug (#152).

I moved on to fixing some bugs.

I made a quick list of four boogs that looked promising

• #110 ICE with an empty block
• #108 closure generation
• #109 incorrectly attaching macros to the wrong node type
• #123 ICE from methods that have same name as previously defined macro

#110 ICE with empty block

Somewhere the AST node for the body of the block was being set to nil. This caused some later code to blow up when it attempted to iterate over the block’s children. I fixed it by ensuring that the body of a block is always set to some value.

I looked at some pretty interesting bits of the compiler while fixing this. Did you know that Mirah’s block support works in two ways? You can use it like you would in JRuby, and pass a block in place of an interface implementation, which is cool. And, you can also pass it a block that contains the definitions of the methods you want to implement. For example (gist)

#108 closure generation

I was able to reproduce this, and figure out roughly what’s going on, but haven’t fixed it yet. The problem is that we’re not generating the signature of the method a block passed as an interface correctly all the time. When the block has no arguments, but the method it’s supposed to implement does, Mirah generates a method with the same name but no arguments. Problem.

#109 incorrectly attaching macros to the wrong node type

I reproduced this but didn’t look into it. It’s interesting it only happens when the macro is used somewhere else in the code.

#123 ICE from methods that have same name as previously defined macro

The problem here was that a macro’s return type is InlineCode, and the MethodDefinition node didn’t know how to validate it’s own signature against a macro with the same name. I put in a condition that will cause inference to fail with an error that says the method has the same name as a macro. It’s not particularly elegant, but I don’t think it’s a bad thing for the compiler to do.

This week ended up mostly being researchy. I had a bunch of plans. I wanted to look at some of the work I’d done with implicit main methods–with an eye for changing from creating them at code generation time to modifying the AST and putting them in the right class. I also wanted to dig into making == use Java’s equals() instead of Java’s ==–which compares identity for objects.

Plans

• use AST modification for main method generation
• == –> .equals()
• eql? –> ==
• refactor intrinsic files
• research ivar inference(related #151)
• …
• boogs

I didn’t get to everything, but I touched a lot of it.

Main method

I had some old code that did this from the last time played around with it, so I tried starting there. It modified the JVM compiler’s define_main method to pull elements out of the AST and inject them into the main method of the class with the same name as the file. It worked as long as a class with a matching name was in the file and failed miserably when there wasn’t.

I tried different things to generate AST nodes for the ‘Script’ class that Mirah generates when there is no class of the same name as the file, but couldn’t get it to work. After taking a break and looking again I also realized that I was doing my editing in the wrong place.

The compiler is actually where the code generation happens and not where the AST is generated or transformed. It gets the AST after it has been transformed, as necessary, and after all the types have been inferred. Which makes it a bad place to try to extend Mirah’s behavior.

I think next time I’ll try to inject the main method AST transform as a plugin that adds functionality to the transformer.

equals

I poked around making == use Java’s equals method instead of using ==, which for objects compares their references to see if they point at the same thing. Getting == to work was actually pretty straight forward, I just replaced the code generation that does the java style == with a method call to equals. I didn’t push it yet because I haven’t added !=, which requires a little more knowledge about bytecode.

ivars

I wanted to see if I could figure out how to get public/protected instance variables from superclasses to be accessible from subclasses. I thought it would have something to do with look up mechanism, but I didn’t know where to start.

After looking around the various Java type lookup mechanisms I’m still a bit confused about how everything works. I think I’ll do a deeper look into it next week, focusing on this file (/lib/mirah/jvm/method_lookup.rb), which looks to be where most of the interesting type stuff happens.