Packages

sealed abstract class State[S] extends AnyRef

:: Experimental :: Abstract class for getting and updating the state in mapping function used in the mapWithState operation of a pair DStream (Scala) or a JavaPairDStream (Java).

Scala example of using State:

// A mapping function that maintains an integer state and returns a String
def mappingFunction(key: String, value: Option[Int], state: State[Int]): Option[String] = {
  // Check if state exists
  if (state.exists) {
    val existingState = state.get  // Get the existing state
    val shouldRemove = ...         // Decide whether to remove the state
    if (shouldRemove) {
      state.remove()     // Remove the state
    } else {
      val newState = ...
      state.update(newState)    // Set the new state
    }
  } else {
    val initialState = ...
    state.update(initialState)  // Set the initial state
  }
  ... // return something
}

Java example of using State:

// A mapping function that maintains an integer state and returns a String
Function3<String, Optional<Integer>, State<Integer>, String> mappingFunction =
   new Function3<String, Optional<Integer>, State<Integer>, String>() {

     @Override
     public String call(String key, Optional<Integer> value, State<Integer> state) {
       if (state.exists()) {
         int existingState = state.get(); // Get the existing state
         boolean shouldRemove = ...; // Decide whether to remove the state
         if (shouldRemove) {
           state.remove(); // Remove the state
         } else {
           int newState = ...;
           state.update(newState); // Set the new state
         }
       } else {
         int initialState = ...; // Set the initial state
         state.update(initialState);
       }
       // return something
     }
   };
S

Class of the state

Annotations
@Experimental()
Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. State
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def exists(): Boolean

    Whether the state already exists

  2. abstract def get(): S

    Get the state if it exists, otherwise it will throw java.util.NoSuchElementException.

    Get the state if it exists, otherwise it will throw java.util.NoSuchElementException. Check with exists() whether the state exists or not before calling get().

    Exceptions thrown

    java.util.NoSuchElementException If the state does not exist.

  3. abstract def isTimingOut(): Boolean

    Whether the state is timing out and going to be removed by the system after the current batch.

    Whether the state is timing out and going to be removed by the system after the current batch. This timeout can occur if timeout duration has been specified in the StatSpec and the key has not received any new data for that timeout duration.

  4. abstract def remove(): Unit

    Remove the state if it exists.

    Remove the state if it exists.

    State cannot be updated if it has been already removed (that is, remove() has already been called) or it is going to be removed due to timeout (that is, isTimingOut() is true).

  5. abstract def update(newState: S): Unit

    Update the state with a new value.

    Update the state with a new value.

    State cannot be updated if it has been already removed (that is, remove() has already been called) or it is going to be removed due to timeout (that is, isTimingOut() is true).

    Exceptions thrown

    java.lang.IllegalArgumentException If the state has already been removed, or is going to be removed

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @native() @throws( ... )
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  10. final def getOption(): Option[S]

    Get the state as a scala.Option.

    Get the state as a scala.Option. It will be Some(state) if it exists, otherwise None.

    Annotations
    @inline()
  11. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  12. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  13. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  14. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  15. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  16. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  17. final def toString(): String
    Definition Classes
    State → AnyRef → Any
    Annotations
    @inline()
  18. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  19. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  20. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @throws( ... )

Inherited from AnyRef

Inherited from Any

Ungrouped