大資料進階之路——Scala入門

概述

https://www.scala-lang.org/

Scala combines object-oriented and functional programming in one concise, high-level language. Scala’s static types help avoid bugs in complex applications, and its JVM and JavaScript runtimes let you build high-performance systems with easy access to huge ecosystems of libraries.

• 學習Scala的意義：
  Spark、Kafka、Flink
  優雅
  開發速度快
  融合到生態圈

安裝

1. 安裝 Java 8

2. 下載 download scala 網址：https://www.scala-lang.org/download/2.11.8.html

3. 解壓 unzip scala

4. 配置環境變數(可選)
   Windows 需配置兩個 Path中： D:\scala\bin 和 D:\scala\jre\bin
   

5. 查看是否生效

Linux或Mac中操作步驟：
1.tar -zxvf scala-2.11.8.tgz -C  解壓路徑
2.到解壓目錄下 pwd  復制整個路徑 
3.將上面的路徑 添加到環境變數中
	vi ~/.bash_profile
		export SCALA_HOME=復制的路徑
		export PATH=$SCALA_HOME/bin:$Path
	保存
	source ~/.bash_profile
	echo $SCALA_HOME

下載之后的scala目錄下的bin目錄中 有普通檔案 和 .bat檔案
.bat檔案是在Windows中用的，Linux或Mac中用不到，所以可以刪掉 rm *.bat

Java VS Scala

Java

public class HelloWorld {
	public static void main(String[] args) {
		System.out.println("Hello World..");
	}
}

Scala(每行代碼并不強求使用;結束，但是Java是必須的)

Object HelloWorld {
	def main(args : Array[String]) {
		println("Hello World..")
	}
}

val 和 var

• val：值

final
val 值名稱：型別=xx
val a = 1  (不可變)
val a : int = 1

• var：變數

var 值名稱：型別=xxx
var b = 1
var b : int = 1

基本資料型別

• Byte/Char
• Short/Int/Long/Float/Double
• Boolean
  只有Float宣告時比較特別
• var c : Float = 1.1f

scala> b=20
b: Int = 20

scala> val b:Int =10
b: Int = 10

scala> val c:Boolean=true
c: Boolean = true

scala> val d =1.1
d: Double = 1.1

scala> val e:Float=1.2f
e: Float = 1.2

lazy在Scala中的應用

lazy var d : int = 1;
延遲加載，只有在第一次使用時才加載

讀取檔案并以字串形式輸出

	import scala.io.Source._
	var info = fromFile("...").mkString

如果用lazy var info = fromFile("…").mkString，開始是檢測不到錯誤的，要小心使用

*注意：當一個變數宣告為lazy，只有當你第一次操作時才會去真正訪問，如果不去訪問，即使寫錯了，也不會發現

開發工具IDEA

Maven

1.下載IDEA和Maven
2.進入IDEA，新建專案 選擇Maven 勾選create from archetype 選擇scala-archetype simple-> 正常創建(注意Maven倉庫位置)
3.IDEA默認是不支持Scala的，需要下載Scala插件
	File -> settings -> Plugins -> install JetBrains plugin -> scala
	之后就可以new 一個Scala類了
4.新建測驗類，運行報錯
	洗掉pom.xml中<arg>make:transitive</args>

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>org.example</groupId>
  <artifactId>untitled5</artifactId>
  <version>1.0-SNAPSHOT</version>
  <inceptionYear>2008</inceptionYear>
  <properties>
    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.target>1.8</maven.compiler.target>
    <encoding>UTF-8</encoding>
    <scala.version>2.11.8</scala.version>
    <spark.version>2.3.0</spark.version>
  </properties>

  <dependencies>
    <dependency>
      <groupId>org.scala-lang</groupId>
      <artifactId>scala-library</artifactId>
      <version>${scala.version}</version>
    </dependency>

    <!--引入Spark Core的依賴-->
    <!-- https://mvnrepository.com/artifact/org.apache.spark/spark-core -->
    <dependency>
      <groupId>org.apache.spark</groupId>
      <artifactId>spark-core_2.11</artifactId>
      <version>2.3.0</version>
    </dependency>

    <!-- https://mvnrepository.com/artifact/org.apache.spark/spark-sql -->
    <dependency>
      <groupId>org.apache.spark</groupId>
      <artifactId>spark-sql_2.12</artifactId>
      <version>2.4.3</version>
    </dependency>

  </dependencies>

  <build>
    <sourceDirectory>src/main/scala</sourceDirectory>
    <testSourceDirectory>src/test/scala</testSourceDirectory>
    <plugins>
      <plugin>
        <groupId>org.scala-tools</groupId>
        <artifactId>maven-scala-plugin</artifactId>
        <version>2.15.0</version>
        <executions>
          <execution>
            <goals>
              <goal>compile</goal>
              <goal>testCompile</goal>
            </goals>
            <configuration>
              <args>
                <arg>-dependencyfile</arg>
                <arg>${project.build.directory}/.scala_dependencies</arg>
              </args>
            </configuration>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.6</version>
        <configuration>
          <useFile>false</useFile>
          <disableXmlReport>true</disableXmlReport>
          <!-- If you have classpath issue like NoDefClassError,... -->
          <!-- useManifestOnlyJar>false</useManifestOnlyJar -->
          <includes>
            <include>**/*Test.*</include>
            <include>**/*Suite.*</include>
          </includes>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>