From Ruby to Scala and back again: Better living through type-checking
This proposal by <a href="/users/timcowlishaw">Tim Cowlishaw</a> has been chosen by the community to be given at Ruby Manor 4.
updated about 4 years ago; latest suggestion about 4 years ago
I'd like to talk a bit about statically typed languages (especially Scala), and their advantages and disadvantages, from a Rubyist's perspective. This would involve:
A short introduction to type systems and the words we use to describe them (strong/weak, static/dynamic etc)
A short introduction to Scala (this should be very quick - it basically looks like Ruby with type annotations)
A motivating example, developed in parallel in Ruby and Scala, where we'd look at the kind of advantages and disadvantages that Scala's type system offers us, and any lessons it teaches us about how to improve our Ruby implementation. I'd pick this example to make sure it gave a fairly comprehensive overview of what's possible with type-level programming in Scala, it'd include type-classes, parameterised types, implicit conversions and dependent types, for instance, rather than just vanilla java-style type annotations on methods.
A summary of the ways in which thinking like a type-checker can give us more confidence in our tests, help us write cleaner, more understandable code, and refactor with more confidence in Ruby, as well as a discussion of the things we shouldn't rely on the compiler to do for us in Scala.
I'd close with a brief look at the way Scala combines some of the best aspects of functional and object oriented programming, and how these have influenced the Ruby implementation of our example that we developed earlier, and talk a bit about why I think the next language you learn should be Scala.
Finally, a question for the more mathematically / logically inclined than I (Tom in particular):
While I see this as a predominantly practical talk, I've been reading a little around type theory, in order to provide some context, and at least a small amount of rigour. One think I think it might be really useful to touch upon is the Curry-Howard correspondance, which (in a really general sense - I'm glossing over a lot of detail here) shows that statements about types are equivalent to statements of (a certain type of) logic. This I think is the key to why thinking about types can be useful even when you're not working in a statically typed language, as your unit tests are really just a series of logical propositions about the behaviour of the components of your system, and thinking about these components as types, can, via Curry-Howard, help you work out what propositions your test suite needs to assert about them in order to guarantee certain behaviours.
This has lead to another realisation that I think might be really interesting to talk about, (and which touches on Tom's idea earlier that types are tests for all programs, and tests, are tests for this program), but I just want to make sure it's correct first- that, while type systems allow you to make universally quantified statements about programs, eg: "this function returns an integer for all integer inputs" (modulo non-termination cough, splutter, mumble), unit tests necessarily only allow existentially quantified statements eg: "There exists an integer n for which this function returns another integer n", which we treat, for the purposes of testing, as indicative of some universal property that's not actually proved. This, I think, is key to how thinking about types can lead to better test suites, as they give us tools to reason about how well our (existential) tests provide coverage of the (universal) properties we actually want to assert.
Therefore, is this:
(a) a load of rubbish, and there's an obvious counterexample where a unit-test does make a universally-quantified statement about the behaviour of some program component, or some other glaring problem with my reasoning?
(b) Irrelevant / too abstract / not interesting?
(c) A useful way of thinking about types, testing and the relationship between the two?
I'd be really interested to hear any thoughts on this!
Regarding freedom of expression, this is a very interesting question. I don't tend to see Scala as being less expressive than Ruby, but this is almost certainly because I've learnt to express myself differently in each, whereas trying to write in a Ruby-ish style in Scala would undoubtedly be a frustrating experience (despite the superficial similarity in syntax). Surprisingly, duck-typing isn't an issue here- Scala has a very elegant way of allowing the same behaviour without sacrificing type-safety (the same mechanism also allows behaviour like Ruby 2's refinements, - I'll go into both of these things in the talk). In fact, the stuff that causes difficulty is perhaps less obvious - There's some fun stuff we can look at around extending Scala to have lists that behave like Ruby arrays, which really stretches the type system to its limit and is pretty cool. Really though, I'd like to make the point that while there's definitely some trade-off between safety and expression, it's not nearly as acute as you might think, and in many cases, the type-system can encourage ways of programming that are in fact more expressive, and which are also applicable in Ruby.
To address Nick's question about head-space, I think this is a really important point, my answer to which Steve has already touched upon. To me, one of the biggest changes in the way I approach programming in a statically typed language stems from the way it forces you to consider the roles and responsibilities your objects have, (and, crucially, to give them names) early on, as well as encouraging you to consider the interactions between them very explicitly. This I think is one of the aspects of programming in Scala that's absolutely beneficial when working in Ruby, and is certainly something I'd discuss in the talk.
Thanks for all the feedback guys! I'll split my response over a few posts to lessen the risk of hitting 'refresh' and losing everything.
Re: Livecoding, thanks for all the advice, I think I will fake it in this case, with a very well prepared case study shown through video / screenshots. I think that there may be quite a lot to pack into this talk, so therefore it'd be better to be able to concentrate on explanation, rather than making sure a live-programming experiment goes to plan.
Re -live coding. I live-coded during a talk at scotruby last year, and I'd recommend having as much prepared/recorded as you can.
It gives you much more opportunity to talk through the thinking, and the risk that you'll make one mistake that ends up snowballing is much lower.
I don't really have any experience of scala - so I'm interested to see your takeaways here.
One of the benefits I see with typed languages is the use of explicit types. I think this helps me to consider roles rather than concrete objects during TDD. I'd be interested to hear if you found that scala's type system helped you find abstractions more easily or not?
I'd love to see this talk happen! In particular I'm wondering if you can touch on the philosophy of Scala, as well as being specific around topics like strong typing.
Are you in the same mental head-space with Scala as you are with Ruby? In my eyes one of the most important aspects of Ruby is the freedom of expression. Does type strictness interfere with that? Duck typing?
Aha, Ok, I see what you mean. attr_accessor is probably a really good example actually, and I could more generally show which common ruby idioms translate to scala and which don't, and with what level of hackery to accomodate them.
There will never be a restriction on the set of programs that can be expressed: Scala is Turing-complete. The question is to what extent expressivity is compromised by the imposition of a static type system: I like being able to use
attr_accessorin Ruby, and I don’t want to have to call
Aha, I just had a look at James's talk and that looks like a very sensible approach. Will keep that in mind!
Regarding methodmissing / definemethod - I guess what I'm having trouble with is coming up with an example of a program where use of them is necessary. While it's true that they're language features that aren't (easily) expressible in a statically-typed language, I guess I see the use of them as a design decision that can be factored out, so therefore, this isn't really a restriction on the set of programs that can be expressed. I will give this some thought though, as I'm sure there's an example that is at least ridiculously impractical in the presence of a static type system which ruby makes easy.
Re: test suites vs type systems, I tend to think of a type system as a test suite for all possible programs in that language, whereas we also tend to want tests for our specific program.
Well, what about something that relies on
define_method, or an extreme application of duck typing, or whatever? (I’m afraid I haven’t done any Scala for almost a decade so I’m no longer au fait with what its type system can accommodate.)
Also @Tom, this time re: examples of valid ruby programs that would be rejected by a static type-checker: This is also a really good idea, but I'm having trouble coming up with a motivating example that's sufficiently concrete. If you've any suggestions I'd love to hear them!
Tom: RE the difference between a static type system and a test suite, I think this is a very important point. To me, the type system can only ever replace a subset of your test suite, as while it can assert that the system will behave in a certain way, and protect against regressions, it can't perform the other important functions of a test suite in BDD, namely to document the behaviour of the system, the end-user's acceptance criteria, and to drive the design and implementation from the outside in.
Additionally: static type systems are great, but their big downside is that they will always reject some of the programs that we actually want to write.
In the interests of balance, it would be interesting to see an example of a program that is easy to write in Ruby but difficult/impossible to write in Scala because of type system constraints.
A less meta remark: what’s the difference between a static type system and a test suite? Do we need both? (I have plenty of opinions of my own, but would love to hear yours!)
As a counterpoint, I find live-coding presentations more stressful to watch as an audience member, because there’s the constant threat of something weird happening and people’s time being wasted.
I like what James Coglan did: do some slides, then talk us through some “live coding” that’s actually a video, so that we get essentially all of the benefits of watching someone code but none of the stress of watching them fiddle about to get the terminal on the right display / get the right font size / work around a segfault etc.
I think I'd attempt to live-code as much as possible, or at least use some well-managed git-fu to fold in pre-prepared commits for most of the busy-work, live-coding the changes that are most instructive. This'd obviously require a fair bit of planning, but I tend to feel (as an audience member) that live-coding is far more instructive than just code on a slide. I'd be interested in hearing from other people on this though, as I'm happy to do either.
Very interested in this as Scala has been on my radar of languages I'd like to try out for a while.
Seeing it from a Rubyists perspective would be a nice intro, especially if its going to help improve my Ruby as well
This looks like a really well thought-out proposal.
When presenting the example, were you planning on live-coding, or will each stage be prepared in advance?