Scala里的WordCount 案例

7.7.5 普通 WordCount 案例

在这里插入图片描述

package chapter07object TestWordCount__简单版 {def main(args: Array[String]): Unit = {//单词计数：将集合中出现的相同单词计数，进行计数，取计数排名的前三的结果val stringList = List("Hello Scala Hbase kafka", "Hello Scala Hbase", " Hello Scala", "Hello")//1.将每个字符串转换成一个一个单词val wordList:List[String] = stringList.flatMap(str => str.split(" "));println(wordList)println("------------------------------------------------------------------------------------------")//2.将相同的单词放置在一起,则需要用Map进行(k,v)操作val wordSame:Map[String,List[String]] = wordList.groupBy(word => word)println(wordSame)println("------------------------------------------------------------------------------------------")//3.对相同的单词进行计数//(word,list) => (word,count)val wordToCount:Map[String,Int] = wordSame.map(tuple=>(tuple._1,tuple._2.size))println(wordToCount)println("------------------------------------------------------------------------------------------")//4.对计数完成后的结果进行排序（采用降序）val sortList:List[(String,Int)] = wordToCount.toList.sortWith{(A,B) =>{A._2>B._2}}println(sortList)println("------------------------------------------------------------------------------------------")//5.对排序后的结果取前3名、val resThreeList:List[(String,Int)] = sortList.take(3)println(resThreeList)}}

7.7.6 复杂 WordCount 案例

TestWordCount复杂版方式01

package chapter07object TestWordCount__复杂版__方式01 {def main(args: Array[String]): Unit = {val tupleList = List(("Hello Scala Spark World ", 4), ("Hello Scala Spark", 3), (" Hello Scala", 2), ("Hello", 1))val stringList:List[String]  = tupleList.map(t => (t._1 + " ") * t._2)val words:List[String] = stringList.flatMap(s=>s.split(" "))//在map中，如果传进来什么就返回什么，不要用 _ 省略val groupMap:Map[String,List[String]] = words.groupBy(word => word)//val groupMap:Map[String,List[String]] = words.groupBy(_)// //(word,list) => (word,count)val wordToCount:Map[String,Int] = groupMap.map(t=>(t._1,t._2.size))val wordCountList:List[(String,Int)] = wordToCount.toList.sortWith{(left,right) =>{left._2 > right._2}}.take(3)tupleList.map(t=>(t._1+" ")*t._2).flatMap(_.split(" ").groupBy(word=>word).map(t=>(t._1,t._2.size)))println(wordCountList)}
}

TestWordCount复杂版方式02

package chapter07object TestWordCount__复杂版__方式02 {def main(args: Array[String]): Unit = {val tuples = List(("Hello Scala Hbase kafka",4),( "Hello Scala Hbase",3),(" Hello Scala",2),( "Hello",1))val wordToCountList:List[(String,Int)] = tuples.flatMap{t =>{val strings:Array[String] = t._1.split(" ")strings.map(word=>(word,t._2))}}// Hello, List((Hello,4), (Hello,3), (Hello,2), (Hello,1))// Scala, List((Scala,4), (Scala,3), (Scala,2)// Spark, List((Spark,4), (Spark,3)// Word, List((Word,4))val wordToTuoleMap:Map[String,List[(String,Int)]] = wordToCountList.groupBy(t=>t._1)val stringToInts:Map[String,List[Int]] = wordToTuoleMap.mapValues{datas=> datas.map(t=>t._2)}stringToInts//---------------------------------------------------------------------val wordToCountMap:Map[String,List[Int]] = wordToTuoleMap.map{t=>{(t._1,t._2.map(t1 => t1._2))}}val wordToTotalCountMap:Map[String,Int] = wordToTotalCountMap.map(t=>(t._1,t._2))println(wordToCountMap)}}