Table of content

• Overview
• Type Parameters
• Documentation
• Installation
• Debugging
• Example Stream
• Contributions
• The Golden rules of the game
• Pressure
• Concept: Entry
• Features summary
  • Concurrency
• Collectors
• Golang, Receivers and Functions
• Known limitations

Overview

Making Go come to functional programming.

This is a research project in functional programming which I hope will prove useful!

ƒuego brings a few functional paradigms to Go. The intent is to save development time while promoting code readability and reduce the risk of complex bugs.

I hope you will find it useful!

Have fun!!

(toc)

Type Parameters

Starting with version 11, ƒuego uses Go 1.18's Type Parameters.

It is a drastic design change and fundamentally incompatible with previous versions of ƒuego.

Use version 10 or prior if you need the pre-Go1.18 version of ƒuego that is based on interface Entry.

(toc)

Documentation

The code documentation and some examples can be found on godoc.

The tests form the best source of documentation. ƒuego comes with a good collection of unit tests and testable Go examples. Don't be shy, open them up and read them and tinker with them!

Note: Most tests use unbuffered channels to help detect deadlocks. In real life scenarios, it is recommended to use buffered channels for increased performance.

(toc)

Installation

Download

go get github.com/seborama/fuego

Or for a specific version:

go get gopkg.in/seborama/fuego.v12

Import in your code

You can import the package in the usual Go fashion.

To simplify usage, you can use an alias:

package sample

import ƒ "gopkg.in/seborama/fuego.v12"

...or import as a blank import:

package sample

import _ "gopkg.in/seborama/fuego.v12"

Note: dot imports should work just fine but the logger may be disabled, unless you initialised the zap global logger yourself.

(toc)

Debugging

Set environment variable FUEGO_LOG_LEVEL to enable logging to the desired level.

(toc)

Example Stream

strs := []string{
    "a",
    "b",
    "bb",
    "bb",
    "cc",
    "ddd",
}
    
Collect(
  NewStreamFromSlice[string](strs, 100).
    Filter(isString).
    Distinct(stringHash),
  GroupingBy(
    stringLength,
    Mapping(
      stringToUpper,
      Filtering(
        stringLengthGreaterThan(1),
        ToSlice[string](),
      ),
    ),
  ),
)

// result: map[1:[] 2:[BB CC] 3:[DDD]]

(toc)

Contributions

Contributions and feedback are welcome.

For contributions, you must develop in TDD fashion and ideally provide Go testable examples (if meaningful).

If you have an idea to improve ƒuego, please share it via an issue.

And if you like ƒuego give it a star to show your support for the project - it will put a smile on my face! :blush:

Thanks!!

(toc)

The Golden rules of the game

1. Producers close their channel.

2. Consumers do not close channels.

3. Producers and consumers should be running in separate Go routines to prevent deadlocks when the channels' buffers fill up.

(toc)

Pressure

Go channels support buffering that affects the behaviour when combining channels in a pipeline.

When the buffer of a Stream's channel of a consumer is full, the producer will not be able to send more data through to it. This protects downstream operations from overloading.

Presently, a Go channel cannot dynamically change its buffer size. This prevents from adapting the stream flexibly. Constructs that use 'select' on channels on the producer side can offer opportunities for mitigation.

(toc)

Features summary

Streams:

• Stream:
  • Filter
  • Map / FlatMap
  • Reduce
  • GroupBy
  • All/Any/None -Match
  • Intersperse
  • Distinct
  • Head* / Last* / Take* / Drop*
  • StartsWith / EndsWith
  • ForEach / Peek
  • ...
• ComparableStream
• MathableStream

Functional Types:

• Optional
• Predicate

Functions:

• Consumer / BiConsumer
• Function / BiFunction
• StreamFunction

Collectors:

• GroupingBy
• Mapping
• FlatMapping
• Filtering
• Reducing
• ToSlice
• ToMap*

Check the godoc for full details.

(toc)

Concurrency

As of v8.0.0, a new concurrent model offers to process a stream concurrently while preserving order.

This is not possible yet with all Stream methods but it is available with e.g. Stream.Map.

Notes on concurrency

Concurrent streams are challenging to implement owing to ordering issues in parallel processing. At the moment, the view is that the most sensible approach is to delegate control to users. Multiple ƒuego streams can be created and data distributed across as desired. This empowers users of ƒuego to implement the desired behaviour of their pipelines.

Stream has some methods that fan out (e.g. ForEachC). See the godoc for further information and limitations.

I recommend Rob Pike's slides on Go concurrency patterns:

• Go Concurrency Patterns, Rob Pike, 2012

As a proof of concept and for facilitation, ƒuego has a CStream implementation to manage concurrently a collection of Streams.

(toc)

Collectors

A Collector is a mutable reduction operation, optionally transforming the accumulated result.

Collectors can be combined to express complex operations in a concise manner. Simply put, a collector allows the creation of bespoke actions on a Stream.

ƒuego exposes a number of functional methods such as MapToInt, Head, LastN, Filter, etc... Collectors also provide a few functional methods.

But... what if you need something else? And it is not straightforward or readable when combining the existing methods ƒuego offers?

Enters Collector: implement your own requirement functionally! Focus on what needs doing in your streams (and delegate the details of the how to the implementation of your Collector).

(toc)

Golang, Receivers and Functions

Some tests (e.g. TestCollector_Filtering) are using receiver Getter methods in guise of Function[T, R any]. Here is the explanation how this is possible.

Function[T, R any] is defined as func(T) R.

A method Getter is typically defined as func (T) Property() R {...}.

With t as the receiver, Go allows t.Property() be called as T.Property(t). This is a func(T) R and hence a Function[T, R any].

Example - TestCollector_Filtering:

employee.Department(employees[0]) is the same as employees[0].Department(), and of course, they both return a string.

The first syntax has one advantage for our purpose though: it is a Function[T, R any].

(toc)

Known limitations

• several operations may be memory intensive or poorly performing.

No parameterised method in Go

Go 1.18 brings typed parameters. However, parameterised methods are not allowed.

This prevents the Map() method of Stream from mapping to, and from returning, a new typed parameter.

To circumvent this, we need to use a decorator function to re-map the Stream.

This can lead to a leftward-growing chain of decorator function calls that makes the intent opaque:

ReStream(
  ReStream(is, Stream[int]{}).Map(float2int),
  Stream[string]{}).Map(int2string)
// This is actually performing: Stream.Map(float2int).Map(int2string)

ƒuego includes a set of casting functions that reduce the visually leftward-growing chain of decorators while preserving a natural functional flow expression:

C(C(C(
  s.
    Map(float2int_), Int).
    Map(int2string_), String).
    Map(string2int_), Int).
    ForEach(print[int])
// This is actually performing: s.Map(float2int).Map(int2string).Map(string2int).ForEach(print)

While not perfect, this is the best workable compromise I have obtained thus far.

(toc)

Performance issues when using numerous parameterised methods in Go 1.18

As a result of this issue, an experiment to add MapTo<native_type>() Stream[<native_type>] is disabled.

Instead, use function CC (ComparableStream Cast) to access Min(), Max(), and MC (MathableStream Cast) to access Sum().

(toc)