Create a new IterableMapper
Iterated over concurrently in the mapper function.
Function which is called for every item in input.
Expected to return a Promise or value.
The mapper should handle all errors and not allow an error to be thrown
out of the mapper function as this enables the best handling of errors
closest to the time that they occur.
If the mapper function does allow an error to be thrown then the
stopOnMapperError option controls the behavior:
- stopOnMapperError: true - will throw the error
out of next or the AsyncIterator returned from [Symbol.asyncIterator]
and stop processing.
- stopOnMapperError: false - will continue processing
and accumulate the errors to be thrown from next or the AsyncIterator
returned from [Symbol.asyncIterator] when all items have been processed.
IterableMapper options
Private _activePrivate _asyncPrivate _currentPrivate Readonly _errorsPrivate _initialPrivate _isPrivate _isPrivate _iteratorPrivate _mapperPrivate _optionsPrivate _resolvingPrivate _unreadPrivate arePrivate bubblePrivate sourceGet the next item from the source iterable.
This is called up to concurrency times in parallel.
If the read queue is not full, and there are source items to read, each instance of this will keep calling a new instance of itself that detaches and runs asynchronously (keeping the same number of instances running).
If the read queue + runners = max read queue length then the runner will exit and will be restarted when an item is read from the queue.
Private startARunnerPrivate startPrivate throwGenerated using TypeDoc
Iterates over a source iterable with specified concurrency, calling the
mapperon each iterated item, and storing themapperresult in a queue of specified max size, before being iterated / read by the caller.Remarks
This allows performing a concurrent mapping with back pressure (won't iterate all source items if the consumer is not reading).
Typical use case is for a
prefetcherthat ensures that items are always ready for the consumer but that large numbers of items are not processed before the consumer is ready for them.