Apache Spark has attracted broad attention in both academia and industry. When people talk about Spark, the first thing that comes to mind is the Resilient Distributed Datasets (RDDs), which lets programmers perform in-memory com-putations on large clusters in a fault-tolerant manner. While RDD is certainly a great contribution, an overlooked aspect of Spark lies in its harness of functional programming con-cept (closure, lazy computation, and delegate) in distributed system building. We believe that this is what makes Spark flexible to support different data processing scenario under one common platform and to execute the distributed opera-tion efficiently. We also believe that Spark has just scratched the surface, and the broader use of functional computing concept can fundamentally change how distributed system is built.
Apache Spark has attracted broad attention in both academia and industry. When people talk about Spark, the first thing that comes to mind is the Resilient Distributed Datasets (RDDs), which lets programmers perform in-memory com-putations on large clusters in a fault-tolerant manner. While RDD is certainly a great contribution, an overlooked aspect of Spark lies in its harness of functional programming con-cept (closure, lazy computation, and delegate) in distributed system building. We believe that this is what makes Spark flexible to support different data processing scenario under one common platform and to execute the distributed opera-tion efficiently. We also believe that Spark has just scratched the surface, and the broader use of functional computing concept can fundamentally change how distributed system is built.
Apache Spark has attracted broad attention in both academia and industry. When people talk about Spark, the first thing that comes to mind is the Resilient Distributed Datasets (RDDs), which lets programmers perform in-memory com-putations on large clusters in a fault-tolerant manner. While RDD is certainly a great...