When proxying requests and responses between the client(s) and the database in the proxy object, the traffic hooks are called. These hooks allow the plugins to inspect and possibly modify the traffic, provided that they register to those hooks. The return value of the hook functions contain two essential fields (key-value) in the req.Fields or resp.Fields:
request (bytes): the binary request from the client(s).response (bytes): the binary response from the database or injected by the plugin(s).The plugins can inspect and return a modified request and/or response. Then, GatewayD decides what to do with the modified request and/or response.
Plugins can also influence the traffic flow. To enable this, they should return an extra field to signal GatewayD to terminate the request. The terminate (boolean) is a "signal" that dictates an "action" to be taken by GatewayD.
GatewayD detects the terminate signal and decides whether to act on the signal or not based on a policy, called the termination policy. The termination policy takes either of the following values:
stop (default): terminates the request and returns the response injected into the request.Fields by the plugin to the client.continue: disregards the signal and continues the traffic flow, thus forwarding the packet to the database.The terminate signal should be injected into the request.Fields and returned by the OnTrafficFromClient hook function. And only the OnTrafficFromClient can return this signal (plus a normal or error response) to influence the traffic flow.
terminate Termination policy (stop/continue) terminate the request by returning a response Solution
An action system should be developed to account for actions other than just termination of the request:
terminate signal is returned by the plugin's OnTrafficFromClient. The termination policy controls whether to run the action or not. Act system gatewayd#451.terminate and they should be always synchronous, while external actions queue jobs and run them asynchronously via runners (plugins). External actions can be either sync (?) or async. Make actions extensible via plugins and/or a scripting language gatewayd#467.terminate action has a quick side-effect of terminating the request, yet calling an API might take more time (and possibly even fail), so it isn't ideal to run the action on the main goroutine. Note that the sync actions exposed by plugins should be called immediately and not queued. Add queueing for async actions gatewayd#464.gatewayd and it should support multiple queueing systems. Add distributed runner for running actions gatewayd#472.flowchart LR
subgraph GatewayD
direction LR
subgraph Proxy
PassThroughToServer
end
PassThroughToServer -- calls --> h
subgraph PX["Plugin X"]
h["OnTrafficFromClient"] --returns--> v1S
end
subgraph v1S["*v1.Struct"]
rFBt["req.Fields['signals'] = []Signal{
Terminate({'value': true}),
Log({'level':'debug','msg':'...'}),
Call({'method':'get','url':'...'})}"]
end
rFBt --are--> Signals
subgraph PE["Policy Engine"]
subgraph TP["Terminate policy"]
IT{"Signal.terminate == true"} -->|true| ST
IT -->|false| sendToS["Send request to server"]
ST{"Policy.terminate == 'stop'"} -->|true| sendToC["Send error/response to client"]
ST -->|false| sendToS
end
subgraph LP["Log policy"]
IL{"'log' in Signal"} -->|true| PL
IL -->|false| X["Discard log signal"]
PL{"Policy.log == true"} -->|true| Log
PL -->|false| X
end
subgraph CP["Call policy"]
IC{"'call' in Signal"} -->|true| PC
IC -->|false| W["Discard call signal"]
PC{"Policy.call == true"} -->|true| Y["Call an API"]
PC -->|false| W
end
end
Signals -. Signal .-> Actions
Log -.- Async
X -.- Async
sendToC -.- Sync
sendToS -.- Sync
W -.- Async
W -.- Sync
Y -.- Async
Signals -- passes through --> IT
Signals -- passes through --> IL
Signals -- passes through --> IC
Actions --> Sync
Actions --> Async
Async --> Queue
subgraph JQ["Job Queue"]
Queue --> Consumer
Consumer --> Worker
Worker --> Result
end
Sync -->|sync| Worker
Result -->|response| h
h -->|response| PassThroughToServer
end
style Actions fill:#fff,color:black
style Log fill:#fff,color:black
style X fill:#fff,color:black
style sendToC fill:#fff,color:black
style sendToS fill:#fff,color:black
style W fill:#fff,color:black
style Y fill:#fff,color:black
style GatewayD fill:#fff,color:black
style PX fill:#fff,color:black
style v1S fill:#fff,color:black
style JQ fill:#fff,color:black
Loading Sequence diagram
sequenceDiagram
Client ->> GatewayD: sends a query
GatewayD ->> Plugins: calls a hook (onTrafficFromClient)
Plugins ->> GatewayD: return single or multiple non-contradicting signal(s)
GatewayD ->> GatewayD: signal is mapped to an action
GatewayD ->> GatewayD: policies control whether to run or discard the action
par runs sync action(s) (e.g. terminate)
GatewayD ->> GatewayD: decides what to do with the request or response
GatewayD -->> Client: terminates traffic (decision is final)
and queues async action(s) (e.g. call a webhook)
GatewayD ->> Queue: queue job for calling a webhook
Queue ->> GatewayD: queued or failed and logged
end
Loading Actions
The following is the list of built-in and custom actions.
GatewayD sync actions Signal Policy Actionterminate (bool) signal.terminate == true && policy.terminate == "stop" Terminate request (current) (change to drop?) disconnect Terminate connection reject Drop the request and reset the TCP connection transmit/tx Bypass plugins and just relay the request/response allow Allow request/response deny Deny request/response block Block client (time-based or permanent) discard Discard a request/response reset Reset the connection either way (disconnect?) fallthough No action (is it needed?) route Route traffic to a specific server forward Forward traffic to a specific server upgrade Upgrade connection to TLS police Apply rate limits to the traffic set Set/update session parameter noset Prevent session parameter update create create a new connection to the database (connect?) limit Limit client or request/response (should it be a custom action?) error Return error response (should it be a response?) request Return a (modified) request response Return a (modified) response GatewayD async actions Signal Policy Action log Log audit trail (or request/response) metric Record a metric queue Queue request/response conntrack Connection tracking mirror Mirror traffic to another server inspect Store traffic for inspection (submit for inspection?) record Store all traffic for inspection quarantine Like record, but for malicious packets (?) call Call an API or webhook (HTTP request?) Plugin or custom actions Signal Policy Action cache Cache request/response alert Log and/or trigger an alert notify Notify a plugin of something (or call a service?) run Run an action (should it be a custom action?) webhook Call a webhook rotate Rotate keys and secrets reauth Reauthenticate the client either way ebpf Run eBPF program to block a client in the kernel wasm Run WebAssembly filter publish/produce Publish a (batch of) message(s) (to Kafka or any other streaming/messaging system) subscribe/consume Consume a (batch of) message(s) (to Kafka or any other streaming/messaging system) Custom debugger actions Signal Policy Action debug Start debugging breakpoint Set a breakpoint on an action/step pause Pause for user input step Run a single step after user input (maybe next?) continue Continue processing abort Abort processing Action storage/channel
Currently everything is passed around at the root level of the request.Fields. The action can be passed through in the context (metadata) to avoid clashing with the keys injected by HandleClientMessage (specifically the Terminate message) from the PostgreSQL wire protocol parser. This needs to be abstracted away in something like gRPC metadata. The metadata is passed around in the context object, which is unidirectional from the client (GatewayD) to the server (plugins). However, the plugins can only pass data to GatewayD using the returned request.Fields (and the error, which is not designed for this task). This means that a custom field should be created to handle the action(s).
Note to self: Policies dictate actions. For example, a simple policy would be equality to a value (query.where.id == 1). Policies can be stored anywhere, including a policy engine like OPA.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4