HighlandJS methods

.flatMap

What it does: Highland's .flatMap takes a function that **receives each data chunk** and **returns another stream**. All these resulting streams are then merged (flattened) into a single output stream.

Use case: It is particularly useful when you want to perform asynchronous operations for each data chunk and then combine all these operations into one single stream. If your transformation returns a stream of streams, flatMap will flatten that into a single stream.

Type Signature: Essentially, .flatMap expects a function of type (A) => Stream B and turns a Stream A into a Stream B.

Example:

_([1, 2, 3]).flatMap((x) => _([x, x * 10]))
// Output will be a single stream: [1, 10, 2, 20, 3, 30]

.through

What it does:: Highland's .through is more general and expects a function that **takes a Highland stream and **returns another Highland stream**__. The function can perform any operation: filter elements, map them, transform the stream in more complex ways, etc.

The function that is passed in:

• will be called for each value in the stream. The function is called with the value as the first argument and a callback as the second argument. The callback must be called for the stream to continue.
• must be a function that takes a stream and returns a stream to work with .through

Use case: You use .through when you have a reusable piece of transformation logic that you want to apply to a stream. This logic could be anything from simple mapping to complex conditional transfor involving multiple steps. .through is designed to extend a pipeline by applying a function to a stream.

Type Signature: .through expects a function of type (StreamA) => StreamB. This function takes a Highland stream as input and returns a new Highland stream. This allows you to easily chain multiple transformations and operations.

Example:

const double = (stream: Highland.Stream<number>) => {
  return stream.map((x) => x * 2)
}

_([1, 2, 3]).through(double)
// Output will be a single stream: [2, 4, 6]

Differences between .flatMap and .through

1. Function Argument:

• flatMap expects a function that takes a data chunk and returns a stream.
• through expects a function that takes a stream and returns a stream.

2. Flattening:

• flatMap automatically flattens the output if the transformation results in a stream of streams.
• through doesn't automatically do this; you would need to explicitly flatten the stream if necessary.

3. Flexibility:

• through is more flexible because it allows you to perform more general transformations, whereas flatMap is specialized for flattening.

4. Pipeline Composition:

• through is more suited for middleware-like pipeline stages, where you take a stream, perform multiple transformations, and return a new stream.
• flatMap is more suited for "pointwise" transformations where each data chunk is independently transformed into a stream.

.filter

What it does:: The .filter method takes a function as an argument. This function should return a boolean value (true or false). The stream will then only include items for which this function returns true.

The function that is passed in:

• will be called for each value in the stream. The function is called with the value as the first argument and a callback as the second argument. The callback must be called for the stream to continue.
• must be a function that takes a stream and returns a stream to work with .through

Use case: .filter is particularly useful when you want to remove unnecessary or undesired items from a data stream. This is helpful when you have a stream of mixed content and you're only interested in a subset of it.

Type Signature: The function provided to .filter has the following type signature:

a -> boolean

Here, a is the type of each chunk of data in the stream.

Example:

import * as _ from 'highland'

const isEven = (n: number) => n % 2 === 0

_([1, 2, 3, 4, 5, 6])
  .filter(isEven)
  .toArray((array) => {
    console.log(array) // Output will be [2, 4, 6]
  })