Skip to content

Commit 71d2e5c

Browse files
ekkinoxekkinox
authored
feat(fxworker): Provided module (#54)
1 parent 7af5255 commit 71d2e5c

24 files changed

+1780
-0
lines changed

‎.github/workflows/coverage.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@ jobs:
4040
- "fxmetrics"
4141
- "fxorm"
4242
- "fxtrace"
43+
- "fxworker"
4344
steps:
4445
- name: Checkout
4546
uses: actions/checkout@v3

‎.github/workflows/fxworker-ci.yml

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
name: "fxworker-ci"
2+
3+
on:
4+
push:
5+
branches:
6+
- "feat**"
7+
- "fix**"
8+
- "hotfix**"
9+
- "chore**"
10+
paths:
11+
- "fxworker/**.go"
12+
- "fxworker/go.mod"
13+
- "fxworker/go.sum"
14+
pull_request:
15+
types:
16+
- opened
17+
- synchronize
18+
- reopened
19+
branches:
20+
- main
21+
paths:
22+
- "fxworker/**.go"
23+
- "fxworker/go.mod"
24+
- "fxworker/go.sum"
25+
26+
jobs:
27+
ci:
28+
uses: ./.github/workflows/common-ci.yml
29+
secrets: inherit
30+
with:
31+
module: "fxworker"

‎README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ Yokai `Fx modules` are the plugins for your Yokai application.
3232
| [fxmetrics](fxmetrics) | Fx module for [prometheus](https://github.com/prometheus/client_golang) |
3333
| [fxorm](fxorm) | Fx module for [orm](orm) |
3434
| [fxtrace](fxtrace) | Fx module for [trace](trace) |
35+
| [fxworker](fxworker) | Fx module for [worker](worker) |
3536

3637
They can also be used in any [Fx](https://github.com/uber-go/fx) based Go application.
3738

‎fxworker/.golangci.yml

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
run:
2+
timeout: 5m
3+
concurrency: 8
4+
5+
linters:
6+
enable:
7+
- asasalint
8+
- asciicheck
9+
- bidichk
10+
- bodyclose
11+
- containedctx
12+
- contextcheck
13+
- cyclop
14+
- decorder
15+
- dogsled
16+
- dupl
17+
- durationcheck
18+
- errcheck
19+
- errchkjson
20+
- errname
21+
- errorlint
22+
- exhaustive
23+
- forbidigo
24+
- forcetypeassert
25+
- gocognit
26+
- goconst
27+
- gocritic
28+
- gocyclo
29+
- godot
30+
- godox
31+
- gofmt
32+
- goheader
33+
- gomoddirectives
34+
- gomodguard
35+
- goprintffuncname
36+
- gosec
37+
- gosimple
38+
- govet
39+
- grouper
40+
- importas
41+
- ineffassign
42+
- interfacebloat
43+
- logrlint
44+
- maintidx
45+
- makezero
46+
- misspell
47+
- nestif
48+
- nilerr
49+
- nilnil
50+
- nlreturn
51+
- nolintlint
52+
- nosprintfhostport
53+
- prealloc
54+
- predeclared
55+
- promlinter
56+
- reassign
57+
- staticcheck
58+
- tenv
59+
- thelper
60+
- tparallel
61+
- typecheck
62+
- unconvert
63+
- unparam
64+
- unused
65+
- usestdlibvars
66+
- whitespace

‎fxworker/README.md

Lines changed: 234 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,234 @@
1+
# Fx Worker Module
2+
3+
[![ci](https://github.com/ankorstore/yokai/actions/workflows/fxworker-ci.yml/badge.svg)](https://github.com/ankorstore/yokai/actions/workflows/fxworker-ci.yml)
4+
[![go report](https://goreportcard.com/badge/github.com/ankorstore/yokai/fxworker)](https://goreportcard.com/report/github.com/ankorstore/yokai/fxworker)
5+
[![codecov](https://codecov.io/gh/ankorstore/yokai/graph/badge.svg?token=ghUBlFsjhR&flag=fxworker)](https://app.codecov.io/gh/ankorstore/yokai/tree/main/fxworker)
6+
[![Deps](https://img.shields.io/badge/osi-deps-blue)](https://deps.dev/go/github.com%2Fankorstore%2Fyokai%2Ffxworker)
7+
[![PkgGoDev](https://pkg.go.dev/badge/github.com/ankorstore/yokai/fxworker)](https://pkg.go.dev/github.com/ankorstore/yokai/fxworker)
8+
9+
> [Fx](https://uber-go.github.io/fx/) module for [worker](https://github.com/ankorstore/yokai/tree/main/worker).
10+
11+
<!-- TOC -->
12+
* [Installation](#installation)
13+
* [Features](#features)
14+
* [Documentation](#documentation)
15+
* [Dependencies](#dependencies)
16+
* [Loading](#loading)
17+
* [Configuration](#configuration)
18+
* [Registration](#registration)
19+
* [Override](#override)
20+
<!-- TOC -->
21+
22+
## Installation
23+
24+
```shell
25+
go get github.com/ankorstore/yokai/fxworker
26+
```
27+
28+
## Features
29+
30+
This module provides a workers pool to your Fx application with:
31+
32+
- automatic panic recovery
33+
- automatic logging
34+
- automatic metrics
35+
- possibility to defer workers
36+
- possibility to limit workers max execution attempts
37+
38+
## Documentation
39+
40+
### Dependencies
41+
42+
This module is intended to be used alongside:
43+
44+
- the [fxconfig](https://github.com/ankorstore/yokai/tree/main/fxconfig) module
45+
- the [fxlog](https://github.com/ankorstore/yokai/tree/main/fxlog) module
46+
- the [fxtrace](https://github.com/ankorstore/yokai/tree/main/fxtrace) module
47+
- the [fxmetrics](https://github.com/ankorstore/yokai/tree/main/fxmetrics) module
48+
- the [fxgenerate](https://github.com/ankorstore/yokai/tree/main/fxgenerate) module
49+
50+
### Loading
51+
52+
To load the module in your Fx application:
53+
54+
```go
55+
package main
56+
57+
import (
58+
"context"
59+
60+
"github.com/ankorstore/yokai/fxconfig"
61+
"github.com/ankorstore/yokai/fxgenerate"
62+
"github.com/ankorstore/yokai/fxlog"
63+
"github.com/ankorstore/yokai/fxmetrics"
64+
"github.com/ankorstore/yokai/fxtrace"
65+
"github.com/ankorstore/yokai/fxworker"
66+
"github.com/ankorstore/yokai/worker"
67+
"go.uber.org/fx"
68+
)
69+
70+
func main() {
71+
fx.New(
72+
fxconfig.FxConfigModule, // load the module dependencies
73+
fxlog.FxLogModule,
74+
fxtrace.FxTraceModule,
75+
fxtrace.FxTraceModule,
76+
fxmetrics.FxMetricsModule,
77+
fxgenerate.FxGenerateModule,
78+
fxworker.FxWorkerModule, // load the module
79+
fx.Invoke(func(pool *worker.WorkerPool) {
80+
pool.Start(context.Background()) // start the workers pool
81+
}),
82+
).Run()
83+
}
84+
```
85+
86+
### Configuration
87+
88+
Configuration reference:
89+
90+
```yaml
91+
# ./configs/config.yaml
92+
app:
93+
name: app
94+
env: dev
95+
version: 0.1.0
96+
debug: true
97+
modules:
98+
worker:
99+
defer: 0.1 # threshold in seconds to wait before starting all workers, immediate start by default
100+
attempts: 3 # max execution attempts in case of failures for all workers, no restart by default
101+
metrics:
102+
collect:
103+
enabled: true # to collect metrics about workers executions
104+
namespace: app # workers metrics namespace (default app.name value)
105+
subsystem: worker # workers metrics subsystem (default worker)
106+
```
107+
108+
Notes:
109+
110+
- the workers logging will be based on the [fxlog](https://github.com/ankorstore/yokai/tree/main/fxlog)
111+
module configuration
112+
- the workers tracing will be based on the [fxtrace](https://github.com/ankorstore/yokai/tree/main/fxtrace)
113+
module configuration
114+
115+
### Registration
116+
117+
This module provides the possibility to register
118+
several [Worker](https://github.com/ankorstore/yokai/blob/main/worker/worker.go) implementations, with
119+
optional [WorkerExecutionOption](https://github.com/ankorstore/yokai/blob/main/worker/option.go).
120+
121+
They will be then collected and given by Fx to
122+
the [WorkerPool](https://github.com/ankorstore/yokai/blob/main/worker/pool.go), made available in the Fx container.
123+
124+
This is done via the `AsWorker()` function:
125+
126+
```go
127+
package main
128+
129+
import (
130+
"context"
131+
132+
"github.com/ankorstore/yokai/fxconfig"
133+
"github.com/ankorstore/yokai/fxgenerate"
134+
"github.com/ankorstore/yokai/fxlog"
135+
"github.com/ankorstore/yokai/fxmetrics"
136+
"github.com/ankorstore/yokai/fxtrace"
137+
"github.com/ankorstore/yokai/fxworker"
138+
"github.com/ankorstore/yokai/worker"
139+
"go.uber.org/fx"
140+
)
141+
142+
type ExampleWorker struct{}
143+
144+
func NewExampleWorker() *ExampleWorker {
145+
return &ExampleWorker{}
146+
}
147+
148+
func (w *ExampleWorker) Name() string {
149+
return "example-worker"
150+
}
151+
152+
func (w *ExampleWorker) Run(ctx context.Context) error {
153+
worker.CtxLogger(ctx).Info().Msg("run")
154+
155+
return nil
156+
}
157+
158+
func main() {
159+
fx.New(
160+
fxconfig.FxConfigModule, // load the module dependencies
161+
fxlog.FxLogModule,
162+
fxtrace.FxTraceModule,
163+
fxtrace.FxTraceModule,
164+
fxmetrics.FxMetricsModule,
165+
fxgenerate.FxGenerateModule,
166+
fxworker.FxWorkerModule, // load the module
167+
fx.Provide(
168+
fxworker.AsWorker(
169+
NewExampleWorker, // register the ExampleWorker
170+
worker.WithDeferredStartThreshold(1), // with a deferred start threshold of 1 second
171+
worker.WithMaxExecutionsAttempts(2), // and 2 max execution attempts
172+
),
173+
),
174+
fx.Invoke(func(pool *worker.WorkerPool) {
175+
pool.Start(context.Background()) // start the workers pool
176+
}),
177+
).Run()
178+
}
179+
```
180+
181+
To get more details about the features made available for your workers (contextual logging, tracing, etc.), check
182+
the [worker module documentation](https://github.com/ankorstore/yokai/tree/main/worker).
183+
184+
### Override
185+
186+
By default, the `worker.WorkerPool` is created by
187+
the [DefaultWorkerPoolFactory](https://github.com/ankorstore/yokai/blob/main/worker/factory.go).
188+
189+
If needed, you can provide your own factory and override the module:
190+
191+
```go
192+
package main
193+
194+
import (
195+
"context"
196+
197+
"github.com/ankorstore/yokai/fxconfig"
198+
"github.com/ankorstore/yokai/fxgenerate"
199+
"github.com/ankorstore/yokai/fxhealthcheck"
200+
"github.com/ankorstore/yokai/fxlog"
201+
"github.com/ankorstore/yokai/fxmetrics"
202+
"github.com/ankorstore/yokai/fxtrace"
203+
"github.com/ankorstore/yokai/fxworker"
204+
"github.com/ankorstore/yokai/healthcheck"
205+
"github.com/ankorstore/yokai/worker"
206+
"go.uber.org/fx"
207+
)
208+
209+
type CustomWorkerPoolFactory struct{}
210+
211+
func NewCustomWorkerPoolFactory() worker.WorkerPoolFactory {
212+
return &CustomWorkerPoolFactory{}
213+
}
214+
215+
func (f *CustomWorkerPoolFactory) Create(options ...worker.WorkerPoolOption) (*worker.WorkerPool, error) {
216+
return &worker.WorkerPool{...}, nil
217+
}
218+
219+
func main() {
220+
fx.New(
221+
fxconfig.FxConfigModule, // load the module dependencies
222+
fxlog.FxLogModule,
223+
fxtrace.FxTraceModule,
224+
fxtrace.FxTraceModule,
225+
fxmetrics.FxMetricsModule,
226+
fxgenerate.FxGenerateModule,
227+
fxworker.FxWorkerModule, // load the module
228+
fx.Decorate(NewCustomWorkerPoolFactory), // override the module with a custom factory
229+
fx.Invoke(func(pool *worker.WorkerPool) {
230+
pool.Start(context.Background()) // start the custom worker pool
231+
}),
232+
).Run()
233+
}
234+
```

‎fxworker/define.go

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
package fxworker
2+
3+
import "github.com/ankorstore/yokai/worker"
4+
5+
// WorkerDefinition is the interface for workers definitions.
6+
type WorkerDefinition interface {
7+
ReturnType() string
8+
Options() []worker.WorkerExecutionOption
9+
}
10+
11+
type workerDefinition struct {
12+
returnType string
13+
options []worker.WorkerExecutionOption
14+
}
15+
16+
// NewWorkerDefinition returns a new [WorkerDefinition].
17+
func NewWorkerDefinition(returnType string, options ...worker.WorkerExecutionOption) WorkerDefinition {
18+
return &workerDefinition{
19+
returnType: returnType,
20+
options: options,
21+
}
22+
}
23+
24+
// ReturnType returns the worker definition return type.
25+
func (w *workerDefinition) ReturnType() string {
26+
return w.returnType
27+
}
28+
29+
// Options returns the worker definition execution options.
30+
func (w *workerDefinition) Options() []worker.WorkerExecutionOption {
31+
return w.options
32+
}

0 commit comments

Comments
 (0)