I have been evaluating Scala for quite some time now as an alternative to Java for enterprise development and one of the most common criticisms I have seen against Scala is that it is too complex for average developers. A quick Google search on Scala Complexity returns numerous links to blogs both defending and criticizing Scala for its perceived complexity. In my opinion one key aspect that has been ignored in this debate is that adoption of a framework/language/whatever in a large enterprise is almost never about technology per say, but has more to do with the culture of the organization.
To better understand the statement ‘Scala is too complex for average developers’, we will have to dig a little bit deeper into the prevalent culture in most IT departments in large enterprise today. This can be done by slightly tweaking the Type XY theories created and developed by Douglas McGregor at MIT in 1960s. In a nutshell Type X developers and Type Y developers are on the two opposite extreme of a spectrum.
Type X programmers could be described as:
- Individuals who lack ambition, dislike responsibility and prefer to be led.
- Individuals who desire job security and try to maintain status co by resisting change.
- Individuals who dislike work and avoid it where possible.
Type Y programmers could be described as:
- Individuals who are ambitious, self-motivated and exercise self-control.
- Individuals who consider effort at work as just like rest or play.
- Individuals who possess the ability for creative problem solving, but their talents are underused in most organizations.
Now with this new classification in mind, let’s see how our two programmer types fare against Scala and its perceived complexity.
Is Scala too complex for Type X programmers?
Scala is definitely too complex for Type X programmers. However
- Java 5 and beyond is also too complex for Type X programmers.
- Spring and Hibernate are also too complex for Type X programmers.
- Multithreading and thread pools are also too complex for Type X programmers.
In a nutshell, anything and everything that is new and require learning is too complex for Type X programmers.
Is Scala too complex for Type Y programmers?
Scala is definitely NOT too complex for programmers who are driven and self-motivated. The seduction of Scala for Type Y programmers in large enterprise is that Scala solves real world problems much more efficiently than Java. Some of the key benefits of Scala in large enterprise are:
- Much more elegant solutions to Multithreading problems than error prone Java alternatives.
- A lot less boilerplate code in the form of type inference, case classes and higher level functions.
- Ability to use existing Java libraries and infrastructure.
Lastly, Is Scala too complex for the management?
Management does not care about the complexity of programming language as long as it is enabling them to achieve faster time to market. In my opinion Scala does not require any additional investment from management as it can leverage all the existing Java infrastructure and is far more productive for their development team since it requires a lot less code than Java to solve similar problems.
So, is Scala really too complex for average developers?
In a large and conservative enterprise, a few Type Y programmers act as change agents by influencing both executive management and hordes of Type X programmers. A few years ago they evangelized Spring and Hibernate in their companies and there is no reason that same cannot be done with Scala. I believe Scala in particular and any new technology in general should just focus on Type Y programmers since Type X programmers will always resist change no matter what the change is. Scala is probably too complex for average developers however the opinions of average developers generally don’t matter. Scala needs to convert that small minority of enterprise developers who are typically responsible for bringing organization wide changes.In this regard Scala is doing a fine job attracting intelligent and self motivated programmers and I have no doubt that it will continue to gain traction in enterprise.
I only read the title. Yes, it is too complex for the average (i.e. shit) developer.
ReplyDeleteThe bottom line is Scala 'can be' too complex for anyone, except type Z, the language designers and other PhD academics. I am definitely a type Y programmer, and I really like Scala and I can write proficient code in it, but it took a lot of effort to get there. Also, while I can read a lot of Scala code, there is definitely a lot of Scala code I cannot read, typically when it is written by certain type Z programmers. On the other hand I have no trouble writing Scala code that type X programmers can easily understand.
ReplyDeleteAt this moment in time Computing Science really needs Scala as it is pushing our boundaries of thinking and imagination. It is one of the most provocative computer languages I have ever seen.
Algol 68 was provocative when it came out, but not widely adopted. C++ was provocative when it came out, and hugely adopted, resulting in billions of lines of some of the most terrible code I have ever seen in my life, especially if it has COM in it. As long as Microsoft do not adopt Scala the language will probably remain useful.
Do I worry about the complexity of Scala - definitely. Am I excited about the promise of Scala - emphatically yes!
I am unequivocally a type Y programmer and I cannot seriously use Scala as it is. One of my very highest desires in a language -- in fact THE highest desire -- is NOT efficient solution representation, but CLEAR AND READABLE and unambiguous code without a lot of surprises. Scala makes things much more "efficient" (in terms of economy of expression), but at the expense of making it much harder to grok quickly. I find that tradeoff to be unworkable, particularly on large projects. If there was a way to turn off some of the sharp edges like operator overloading, which I generally detest (and the core Scala libraries are a perfect case in point as far as I'm concerned), I would be a HUGE Scala advocate because it really does solve a litany of Java problems I've hated for years. The problem with this article is that it's just propaganda, repeating the same things every other pro-Scala blog is repeating. "If you don't like Scala, you're lazy." On the contrary, I'm not intellectually lazy or unadventurous AT ALL. And I would LOVE to switch from Java to a language with Scala's capabilities. I just don't like the intuitive readability of Scala code or quite a few of the most powerful and dangerous features. It's a great academic language that demonstrates what's possible in a modern language. Unfortunately, it's an academic language by virtue of it's feature set and syntax. What I'd like to finally get through someone's head in the Scala community is that I don't hate Scala. I hate the readability of the Scala syntax and the features that make code difficult to read and predict. Go write a new syntactic front end for it that's much simpler and files off all the sharp edges and so forth I'll jump right on the bandwagon. It's that simple. I'm not lazy. I DO want a change. I just don't want this exact feature set and syntax. What I want is LESS, not more. You'd think someone could do that...
ReplyDeleteComplaints about Scala's syntax seem very strange to me; in the absence of pathological operator overloading, I feel like it reads a lot like Ruby does. The degree to which familiarity makes Scala syntax feel like the most natural and correct way to do things cannot be understated: postfix type annotations, square brackets for type parameters, etc. are unfamiliar at first, but after using the language for a while it really does start to appear beautiful. There are still things to complain about with Scala, but syntax isn't one of them. :)
ReplyDelete@jonathan: while I agree that some of the code people write on Scala is hard to read, don't you think that this can happen with *any* language? Would you mind giving some examples where the core scala library is hard to understand? I don't mean the internals of it, but what end users like you or me use?
ReplyDeleteI think that Scala gives your the choice of writing code that average developers can read or code that is very cryptic.
Regards,
Diego
There are two types of people in the world: those who divide people into two groups and those who don't…
ReplyDeleteA girl asked me how to implement the "group by" in VBA, then I show her some code(http://daily-scala.blogspot.com/2009/10/groupby-collection-processing.html). She knows a little bit of java and SQL, and can understand the code easily. But she cannot understand the define of groupBy:
ReplyDelete"def groupBy[K](f : (A) => K) : Map[K, This]"
Scala is not complex, programming is. Java just makes it all the more harder. Now toughen up ya bladdy princesses, pull ya head in and learn some shit. No more whinging.
ReplyDelete@Tony Morris Sorry, but this is pretty close to the C++ mess: http://groups.google.com/group/scala-user/browse_thread/thread/939ed4ea3b7c65ad/3c34a700c0bd2860?lnk=gst&q=val+1+%3D+2+compiles#3c34a700c0bd2860
ReplyDeleteJava has a low barrier to enter, but also a low limit on what you can achieve (in terms of expressiveness, fewer lines of code, and more).
ReplyDeleteScala has a low barrier to enter and a high limit on what you can achieve.
A Low barrier? Yes. You can program Java-like in Scala pretty easily!
So why does it seem complex? Because when someone blogs about something, it is about the high-end that can be achieved. The low barrier is just to mundane to blog about...
@Iron9light
ReplyDeleteWell For List the method is defined like:
def groupBy[K](f : (A) => K ) : Map[K,List]
so what does this tell us? You need to pass in a function which performs on every elemen of the List of type A and this function will give you a corresponding value for each entry of type K . So you supply the function and thus define the type of K. The results are grouped into entries of the map by a common value for K. If you try some examples in te REPL and use a List of Strings with the length function you can See VERY clearly (IMHO) how it works and what the types tell you.
@Andras Schheinert
ReplyDeleteHow do you know from the signature, that the function is applied to every element of the list?
goddard,
ReplyDeleteNo it isn't. It's an essential property of every turing-complete system. Haskell has it too, however, in the Haskell case, this theme arises regularly where a newcomer doesn't quite understand why it is an essential property, then they complain that it is somehow a language bug or oversight, like you have done.
In the Haskell case, I just usually kick back and watch, knowing or perhaps hoping, that the newcomer can learn something (I enjoy learning, even when it is not me). In the Scala case, being the first time this issue has arisen, I am not so sure.
I do hold hope that someone points this out to [scala-user], but for reasons unstated, that is not going to be me. I'll just sit back and watch.
public void f() { f(); } // OMG I can't believe Java allows this!
ReplyDelete@ido
ReplyDeleteThe groupBy method is a Method on the type List[A], your List of type A in that case. It seems to me logical if you have a discriminator function (f) this method can only make sense if you use that function on all elements on the list because you want to group the whole list and not just some elements of it? Thus you have to apply your function on all elements.
@Andreas
ReplyDeletethe point I wanted to make is just, what can be really concluded from _just_ the signature and what has to be documented in addition in the comments or in the name of the function.
I have the feeling that looking at the signature alone is not enough. Although your reasoning is convincing.
def indexWhere (p: (A) ⇒ Boolean): Int
also takes a function, but this is only applied to the elements as long as its result is false, so not to all the elements.
@ido yes I think its a mix of very precise observation of: input parameter, output parameter and the name of the method. Of course there still can be surprises. But IMO the REPL is very nice to test such things.
ReplyDeleteido: This example of the signature is okay, I'd find much harder (those wit implicit manifests etc) - most of the programmers had to go through basic calculus or mathematics course. Yes, it requires a little bit of intellectual effort.
ReplyDeleteI am currently working on a Scala implementation of a program I wrote in Erlang. At this point, I would say Scala is much more difficult than Erlang (but then I have been using Erlang for years and this my first real project in Scala). That being said, I remember the my first Ruby program and how easy that was.
ReplyDeleteOne of the key characteristics I like in a language is "no surprises". For me at least, Erlang and Ruby score very high in this category. Scala, on the other hand, is full of surprises. It has been a lot like Christmas; some of the presents have been great and some have great disappointments. But I'm going to keep opening them.
Nice text and nice replies (esp. Erik Kolotyluk).
ReplyDeleteUnlike most, I'm approaching Scala from a C++ and Lua standpoint. To me, it seems an amalgamation of the expressiveness and power of those two languages. I do like Erik's 'XYZ' approach. Then again, there are things that only "library writers" should do. In C++, I put template programming in this. Awful to develop, nice to use. Done by Z and Y, to the Y and X.
The reason for me to spend time learning Scala is the supposed good networking and performance. And over Lua it wins in having static type checking. Any real projects *really* benefit from that.
Eschneef: *Great* comparison of Scala with Xmas. Agreed.
Ittay: Yes. I found this blog on the basics of Scala good: http://www.codecommit.com/blog/scala/scala-for-java-refugees-part-1