Charming Scala

Scala is one of the frontrunners of the JVM languages (though it has Javascript and .NET backend too). Everyone is talking about Scala (and FP in general) and we all hear the usual stuff about lots of heavy things like functional programming and how it helps in the world of parallelism/concurrency and its conciseness. Yes, FP is great and there are tons of books and articles which will talk about referential transparency, side effects minimization, immutable approach to programming, combinators, etc. but I am going to talk about the charming Scala.Well, I did programming in many languages (and keep learning new ones) but till recently programmed in Clojure and Python. They are great languages but I found something missing when we embarked on building a massive system with large code base. First, static languages give you the peace of mind  Рthat contracts will not be violated at runtime. Compiler works hard to ensure people do not change contracts at will. Then there is performance that the compiler gives you by producing optimized code by virtue of type knowledge. Then there is comfort in using that knowledge of types to write code faster (and be really productive) with IDE features like context aware code assist, refactoring, etc. Testing effort can focus on domain level functionality. Any problems are immediately highlighted. As humans, we always tend to overlook issues and it is nice and reassuring that my IDE is preventing me from using any API inappropriately or writing wrong code. For big teams, writing and maintaining large code bases, static language definitely helps. BTW, I use IntelliJ Scala Plugin which is awesome . The REPL has all facilities that you get in the main editor. For example, using a class which has not been imported into REPL is automatically done so on its usage. I use IDEAVim being a Vim guy (best of both worlds).

So back to Charming Scala. Scala has this image of being very complex. Yes, a language which covers multiple paradigms from imperative to functional to object oriented (though OOP could be both imperative and functional) will have certain complexities. But I would argue that every language has its deep ends be that Ruby, Python and Javascript (you could smother someone with implementing OOP and other powerful paradigms in Javascript).

Scala is beautiful for everyday tasks (including your scripting needs). In the forthcoming series, we will show how neat and powerful Scala can be for your everyday tasks (as much as Ruby and Python). By the end of series, you will see that writing your normal scripts in Scala is as much a pleasure as it is in Ruby and Python (or for the Javascript guys who love node).

We at Xenovus even write our shell scripts in Scala (and you would not find that ridiculous after a while with faster JVM bootup and a persistent compile server running in background and with option to transparently save your script bytecode without any explicit compilation , main method etc. – your scripts are going to run real fast).

And Scala will give you the opportunity to “scale your abilities” as a programmer (coz that is what we want – isn’t it?). One of my favorite hobbies is to read Haskell, Erlang, ML/F#, Lisp/Scheme etc. books and rewrite those in Scala. It is so much fun.

To summarize Scala is

  1. Concise – we’ll show some fun examples below.
  2. With its type inferencing, my daily scripts are as much pleasure to write in Scala as it was in Python
  3. I can learn powerful programming paradigms
  4. I have the whole JVM ecosystem (and for that matter Scala ecosystem is pretty muscled too) – batteries included !
  5. I am reassured that I do not have to learn a new language or recode my hacks when I need to write massively scalable code.
  6. Yes, I am a polyglot as much. I learn from other languages and then Scala helps me with implementing almost anything those languages provide.

The following examples show what is possible in Scala for those who code using Clojure, F#, Haskell, Erlang, etc as these are typical functional idioms that cut across languages. We’ll explain Scala syntax, concepts and idioms in future articles of the series. For now it is just a few code samples. The future series will showcase useful shell scripts and other snippets.

One of my favorite function in Clojure is iterate.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
// powers of two sequence
scala>  Stream.iterate(1)(_*2).take(10).toList
res2: List[Int] = List(1, 2, 4, 8, 16, 32, 64, 128, 256, 512)

// take the first ten non-negative integers
scala>  Stream.iterate(1)(_+1).takeWhile(_ <= 10).toList
res3: List[Int] = List(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)

// Pascal's triangle. It is very simple
// Break the input sequence into adjacent elements i.e. (1,2,3) into (1,2), (2,3)
// map over them using sum
// stick 1 at the edges
def pascalTriangle(): Stream[Vector[Int]] = {
  Vector(1) #:: Stream.iterate(Vector(1,1))(1 +: _.sliding(2).map(_.sum).toVector :+ 1)
}
scala> pascalTriangle.take(10).foreach(println)
Vector(1)
Vector(1, 1)
Vector(1, 2, 1)
Vector(1, 3, 3, 1)
Vector(1, 4, 6, 4, 1)
Vector(1, 5, 10, 10, 5, 1)
Vector(1, 6, 15, 20, 15, 6, 1)
Vector(1, 7, 21, 35, 35, 21, 7, 1)
Vector(1, 8, 28, 56, 70, 56, 28, 8, 1)
Vector(1, 9, 36, 84, 126, 126, 84, 36, 9, 1)

How about tail recursion?

Let us start with the obligatory factorial

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
def factorial(x:BigInt): BigInt = {
  import annotation.tailrec
  @tailrec
  def helper(x: BigInt, accum: BigInt): BigInt = {
    x match {
      case _ if x == 1 => accum
      case _ => helper(x-1, x*accum)
    }
  }
  helper(x, 1)
}
scala> println(factorial(1000))
402387260077093773543702433923003985719374864210714632543799910429938512398629020592044208486969404800479988610197196058631666872994808558901323829669944590997424504087073759918823627727188732519779505950995276120874975462497043601418278094646496291056393887437886487337119181045825783647849977012476632889835955735432513185323958463075557409114262417474349347553428646576611667797396668820291207379143853719588249808126867838374559731746136085379534524221586593201928090878297308431392844403281231558611036976801357304216168747609675871348312025478589320767169132448426236131412508780208000261683151027341827977704784635868170164365024153691398281264810213092761244896359928705114964975419909342221566832572080821333186116811553615836546984046708975602900950537616475847728421889679646244945160765353408198901385442487984959953319101723355556602139450399736280750137837615307127761926849034352625200015888535147331611702103968175921510907788019393178114194545257223865541461062892187960223838971476088506276862967146674697562911234082439208160153780889893964518263243671616762179168909779911903754031274622289988005195444414282012187361745992642956581746628302955570299024324153181617210465832036786906117260158783520751516284225540265170483304226143974286933061690897968482590125458327168226458066526769958652682272807075781391858178889652208164348344825993266043367660176999612831860788386150279465955131156552036093988180612138558600301435694527224206344631797460594682573103790084024432438465657245014402821885252470935190620929023136493273497565513958720559654228749774011413346962715422845862377387538230483865688976461927383814900140767310446640259899490222221765904339901886018566526485061799702356193897017860040811889729918311021171229845901641921068884387121855646124960798722908519296819372388642614839657382291123125024186649353143970137428531926649875337218940694281434118520158014123344828015051399694290153483077644569099073152433278288269864602789864321139083506217095002597389863554277196742822248757586765752344220207573630569498825087968928162753848863396909959826280956121450994871701244516461260379029309120889086942028510640182154399457156805941872748998094254742173582401063677404595741785160829230135358081840096996372524230560855903700624271243416909004153690105933983835777939410970027753472000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Let us go to Fibonacci again in tail recursive manner.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
def fibSequence(n: Int): Vector[BigInt] = {
  assert(n >= 0)
  @tailrec 
  def helper(x: BigInt, accum: Vector[BigInt]): Vector[BigInt] = {
    val result = x match {
      case _ if x == 0 => accum :+ BigInt(0)
      case _ if x == 1  => accum :+ BigInt(1)
      case _ => accum :+ (accum.takeRight(2).sum)
    }
    if(x == n) result else helper(x+1,result)
  }

  helper(0, Vector[BigInt]())
}
scala> println(fibSequence(100))
Vector(0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597, 2584, 4181, 6765, 10946
, 17711, 28657, 46368, 75025, 121393, 196418, 317811, 514229, 832040, 1346269, 2178309, 3524578, 5702887, 9227465, 14930352, 24157817, 39088169, 63245986, 102334155, 165580141, 267914296, 433494437, 701408733, 1134903170, 1836311903, 2971215073, 4807526976, 7778742049, 12586269025, 20365011074, 32951280099, 53316291173, 86267571272, 139583862445, 225851433717, 365435296162, 591286729879, 956722026041, 1548008755920, 2504730781961, 4052739537881, 6557470319842, 10610209857723, 17167680177565, 27777890035288, 44945570212853, 72723460248141, 117669030460994, 190392490709135, 308061521170129, 498454011879264, 806515533049393, 1304969544928657, 2111485077978050, 3416454622906707, 5527939700884757, 8944394323791464, 14472334024676221, 23416728348467685, 37889062373143906, 61305790721611591, 99194853094755497, 160500643816367088, 259695496911122585, 420196140727489673, 679891637638612258, 1100087778366101931, 1779979416004714189, 2880067194370816120, 4660046610375530309, 7540113804746346429, 12200160415121876738, 19740274219868223167, 31940434634990099905, 51680708854858323072, 83621143489848422977, 135301852344706746049, 218922995834555169026, 354224848179261915075)

Or for those who have seen Haskell’s succinct Fibonacci, Scala offers you this:

1
2
3
scala> val fibs: Stream[BigInt] = BigInt(0) #:: BigInt(1) #:: ((fibs zip fibs.tail) map {case (x,y) => x + y})
scala>  fibs.take(13).toList
res27: List[BigInt] = List(0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144)

Note we are using BigInt to avoid overflows and for demonstration purpose primarily.

Also note that @tailrec annotation is not required. We use it to make the compiler check whether we have really written the function in a tail recursive manner (coz we might think we have written it but may have erred. It is nice to have compiler looking over the shoulder).

And now for some fun stuff using higher order functions.
Let us find the prime numbers between 2 and 100
Note there are other algorithms like Sieve of Erastothenes which are more efficient
Here we’ll show how the use of small functions and higher order functions lead to readable code.

1
2
3
4
5
6
7
8
def factors(x: Int) = {
  (1 to x/2).filter(x % _ == 0)
}

def isPrime(x: Int) : Boolean = factors(x).length == 1

scala> println(2 to 100 filter isPrime)
Vector(2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97)

Let us look at frequency count of words in a document.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
val txt =
  """
    |Scala is an acronym for Scalable Language.
    |This means that Scala grows with you. You can play with it by typing one-line expressions and
    |observing the results. But you can also rely on it for large mission critical systems, as many companies,
    |including Twitter, LinkedIn, or Intel do.
  """.stripMargin

"""[.-,]""".r.replaceAllIn(txt, "").trim.split("""s+""").map(_.toLowerCase).foldLeft(Map[String, Int]()) (
  (accum, e) => {
    val count = if(accum.contains(e)) accum(e) + 1 else 1
    accum + (e -> count)
  }
)
res40: scala.collection.immutable.Map[String,Int] = Map(acronym -> 1, mission -> 1, for -> 2, this -> 1, is -> 1,
 companies -> 1, oneline -> 1, but -> 1, systems -> 1, play -> 1, do -> 1, linkedin -> 1, it -> 2, as -> 1, expressions -> 1, or -> 1, observing -> 1, scalable -> 1, critical -> 1, that -> 1, you -> 3, results -> 1, also -> 1, can -> 2, on -> 1, intel -> 1, language -> 1, by -> 1, scala -> 2, with -> 2, means -> 1, grows -> 1, typing -> 1, an -> 1, large -> 1, twitter -> 1, many -> 1, rely -> 1, including -> 1, and -> 1, the -> 1)

As we can see from the above examples, Scala allows us to write powerful and idiomatic functional code.

Before concluding, let us switch gears and do some fun stuff.

How about some string interpolation which is type checked and has code assist. Simply awesome! Just download IntelliJ and try it in the REPL (the next article will be on setting up your tools).

1
2
3
4
5
6
7
8
scala>  val product = "My Cool Bag"
product: String = My Cool Bag

scala>  val price = 100
price: Int = 100

scala>  s"Product $product cost $price but we'll give you at discounted price ${0.9*price}" 
res41: String = Product My Cool Bag cost 100 but we'll give you at discounted price 90.0

Now on to another example. Assume that you want to do some analysis on stock prices. You can get stock data from Yahoo finance using the following url pattern:
“http://ichart.finance.yahoo.com/table.csv?s=MSFT” // change it to any ticker you want

Our goal is to calculate the average stock (opening) price by year.

The file format is CSV. The first line is the header line. Our fields of interest are the first two ones.

In our case, we have saved the content to a local file but with minor change you can fetch data from the net.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
val filePath = "/Users/fpguy/tmp/scala/msft.csv"
val stockUrl =  "http://ichart.finance.yahoo.com/table.csv?s=MSFT"

def fetchFields(line: String, fieldIndices: Set[Int]) = {
  val fields = line.split(",")
  fields.zipWithIndex.filter(t => fieldIndices(t._2)).map(_._1)
}

def convertFields(fields: Array[String]): (Int, Double) = {
  val yr = fields(0).take(4).toInt
  val price = fields(1).toDouble
  (yr, price)
}

val indices = Set(0, 1) // fields that we want
val rows = Source
  .fromFile(filePath) // for fetching from the net: Source.fromURL(stockUrl)
  .getLines // lines iterator
  .drop(1) // drop the header line
  .map(fetchFields(_, indices)) // select the fields. In our case first and second field
  .map(convertFields) // this is a collection of 2-Tuple (yr, price-for-the-day)
  .foldLeft(TreeMap[Int, Vector[Double]]())(
  (accum, e) => {
    val (yr, price) = e
    val lst = if (accum.contains(yr)) accum(yr) :+ price
    else Vector[Double]()
    accum + (yr -> lst) // Remember this is an immutable sorted map
  }
) // Now we have a sorted map with key = year and value = Vector of prices.
  .map {
  case (yr, prices) => (yr, prices.sum / prices.length)
}
  .foreach(println)

(1986,33.713645320197045)
(1987,82.91809523809522)
(1988,55.538809523809526)
(1989,62.516294820717064)
(1990,76.40841269841272)
(1991,93.2979365079365)
(1992,98.24391304347824)
(1993,83.50436507936507)
(1994,67.76374501992035)
(1995,83.02235059760974)
(1996,115.46162055335957)
(1997,120.85861111111085)
(1998,107.82745019920324)
(1999,105.82804780876502)
(2000,76.55780876494026)
(2001,62.37279352226712)
(2002,54.587250996015925)
(2003,29.284143426294822)
(2004,27.133705179282867)
(2005,25.87103585657369)
(2006,26.251760000000004)
(2007,30.410039999999995)
(2008,26.729166666666664)
(2009,22.90350597609561)
(2010,27.071912350597636)
(2011,26.045976095617547)
(2012,29.840682730923707)
(2013,30.8691975308642)

Finally for some shell script.

Assume we want to extract a particular field say, a filename from directory listing and sort it.

$ ls -l | ./myScala.sh  | sort -f

This is how myScala.sh looks

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
#!/bin/sh
exec scala "$0" "$@"
!#
import scala.io.Source

val stdin = Source.stdin

stdin
  .getLines
  .map(_.split("""s+"""))
  .drop(1) // header line
  .map( r => r(r.length - 1))
  .foreach(println)

This concludes our first article which gave a view into the diverse world of Scala –¬† it can be used to write massive systems to simple shell scripts. At Xenovus, we are using the typical Scala stack of Akka, Play, Slick and rest of the ecosystem.

In case you are interested in exploring the world of functional programming and specifically Scala, and be part of our team to build a product based on these technologies, please drop us a mail at Xenovus