Packages

  • package root
    Definition Classes
    root
  • package org
    Definition Classes
    root
  • package apache
    Definition Classes
    org
  • package spark

    Core Spark functionality.

    Core Spark functionality. org.apache.spark.SparkContext serves as the main entry point to Spark, while org.apache.spark.rdd.RDD is the data type representing a distributed collection, and provides most parallel operations.

    In addition, org.apache.spark.rdd.PairRDDFunctions contains operations available only on RDDs of key-value pairs, such as groupByKey and join; org.apache.spark.rdd.DoubleRDDFunctions contains operations available only on RDDs of Doubles; and org.apache.spark.rdd.SequenceFileRDDFunctions contains operations available on RDDs that can be saved as SequenceFiles. These operations are automatically available on any RDD of the right type (e.g. RDD[(Int, Int)] through implicit conversions.

    Java programmers should reference the org.apache.spark.api.java package for Spark programming APIs in Java.

    Classes and methods marked with Experimental are user-facing features which have not been officially adopted by the Spark project. These are subject to change or removal in minor releases.

    Classes and methods marked with Developer API are intended for advanced users want to extend Spark through lower level interfaces. These are subject to changes or removal in minor releases.

    Definition Classes
    apache
  • package mllib

    RDD-based machine learning APIs (in maintenance mode).

    RDD-based machine learning APIs (in maintenance mode).

    The spark.mllib package is in maintenance mode as of the Spark 2.0.0 release to encourage migration to the DataFrame-based APIs under the org.apache.spark.ml package. While in maintenance mode,

    • no new features in the RDD-based spark.mllib package will be accepted, unless they block implementing new features in the DataFrame-based spark.ml package;
    • bug fixes in the RDD-based APIs will still be accepted.

    The developers will continue adding more features to the DataFrame-based APIs in the 2.x series to reach feature parity with the RDD-based APIs. And once we reach feature parity, this package will be deprecated.

    Definition Classes
    spark
    See also

    SPARK-4591 to track the progress of feature parity

  • package linalg
    Definition Classes
    mllib
  • package distributed
    Definition Classes
    linalg
  • DenseMatrix
  • DenseVector
  • Matrices
  • Matrix
  • QRDecomposition
  • SingularValueDecomposition
  • SparseMatrix
  • SparseVector
  • Vector
  • VectorUDT
  • Vectors

sealed trait Matrix extends Serializable

Trait for a local matrix.

Annotations
@SQLUserDefinedType() @Since( "1.0.0" )
Linear Supertypes
Serializable, Serializable, AnyRef, Any
Known Subclasses
DenseMatrix, SparseMatrix
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. Matrix
  2. Serializable
  3. Serializable
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def apply(i: Int, j: Int): Double

    Gets the (i, j)-th element.

    Gets the (i, j)-th element.

    Annotations
    @Since( "1.3.0" )
  2. abstract def asML: ml.linalg.Matrix

    Convert this matrix to the new mllib-local representation.

    Convert this matrix to the new mllib-local representation. This does NOT copy the data; it copies references.

    Annotations
    @Since( "2.0.0" )
  3. abstract def colIter: Iterator[Vector]

    Returns an iterator of column vectors.

    Returns an iterator of column vectors. This operation could be expensive, depending on the underlying storage.

    Annotations
    @Since( "2.0.0" )
  4. abstract def copy: Matrix

    Get a deep copy of the matrix.

    Get a deep copy of the matrix.

    Annotations
    @Since( "1.2.0" )
  5. abstract def numActives: Int

    Find the number of values stored explicitly.

    Find the number of values stored explicitly. These values can be zero as well.

    Annotations
    @Since( "1.5.0" )
  6. abstract def numCols: Int

    Number of columns.

    Number of columns.

    Annotations
    @Since( "1.0.0" )
  7. abstract def numNonzeros: Int

    Find the number of non-zero active values.

    Find the number of non-zero active values.

    Annotations
    @Since( "1.5.0" )
  8. abstract def numRows: Int

    Number of rows.

    Number of rows.

    Annotations
    @Since( "1.0.0" )
  9. abstract def transpose: Matrix

    Transpose the Matrix.

    Transpose the Matrix. Returns a new Matrix instance sharing the same underlying data.

    Annotations
    @Since( "1.3.0" )

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @native() @throws( ... )
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  10. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  12. val isTransposed: Boolean

    Flag that keeps track whether the matrix is transposed or not.

    Flag that keeps track whether the matrix is transposed or not. False by default.

    Annotations
    @Since( "1.3.0" )
  13. def multiply(y: Vector): DenseVector

    Convenience method for Matrix-Vector multiplication.

    Convenience method for Matrix-Vector multiplication.

    Annotations
    @Since( "1.4.0" )
  14. def multiply(y: DenseVector): DenseVector

    Convenience method for Matrix-DenseVector multiplication.

    Convenience method for Matrix-DenseVector multiplication. For binary compatibility.

    Annotations
    @Since( "1.2.0" )
  15. def multiply(y: DenseMatrix): DenseMatrix

    Convenience method for Matrix-DenseMatrix multiplication.

    Convenience method for Matrix-DenseMatrix multiplication.

    Annotations
    @Since( "1.2.0" )
  16. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  17. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  18. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  19. def rowIter: Iterator[Vector]

    Returns an iterator of row vectors.

    Returns an iterator of row vectors. This operation could be expensive, depending on the underlying storage.

    Annotations
    @Since( "2.0.0" )
  20. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  21. def toArray: Array[Double]

    Converts to a dense array in column major.

    Converts to a dense array in column major.

    Annotations
    @Since( "1.0.0" )
  22. def toString(maxLines: Int, maxLineWidth: Int): String

    A human readable representation of the matrix with maximum lines and width

    A human readable representation of the matrix with maximum lines and width

    Annotations
    @Since( "1.4.0" )
  23. def toString(): String

    A human readable representation of the matrix

    A human readable representation of the matrix

    Definition Classes
    Matrix → AnyRef → Any
  24. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  25. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  26. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped