Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

Commit 2b3591a

Browse files
[SDP] FlowPlanner
1 parent 85fcddf commit 2b3591a

File tree

4 files changed

+40
-2
lines changed

4 files changed

+40
-2
lines changed
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# CompleteFlow
2+
3+
`CompleteFlow` is...FIXME

‎docs/declarative-pipelines/Flow.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Flow
2+
3+
`Flow` is...FIXME

‎docs/declarative-pipelines/FlowPlanner.md

Lines changed: 31 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,42 @@
11
# FlowPlanner
22

3-
## Plan { #plan }
3+
`FlowPlanner` is used to [plan a flow into a FlowExecution](#plan).
4+
5+
`FlowPlanner` is created alongside a [GraphExecution](GraphExecution.md#flowPlanner) (i.e., when [PipelineExecution](PipelineExecution.md) is requested to [start a pipeline](PipelineExecution.md#startPipeline)).
6+
7+
## Creating Instance
8+
9+
`FlowPlanner` takes the following to be created:
10+
11+
* <span id="graph"> [DataflowGraph](DataflowGraph.md)
12+
* <span id="updateContext"> [PipelineUpdateContext](PipelineUpdateContext.md)
13+
* <span id="triggerFor"> [Flow-to-Streaming-Trigger Conversion Function](#triggerFor)
14+
15+
`FlowPlanner` is created alongside a [GraphExecution](GraphExecution.md#flowPlanner).
16+
17+
### Flow-to-Streaming-Trigger Conversion Function { #triggerFor }
18+
19+
```scala
20+
triggerFor: Flow => Trigger
21+
```
22+
23+
`FlowPlanner` is given a function to convert a [Flow](Flow.md) into a streaming `Trigger` ([Spark Structured Streaming]({{ book.structured_streaming }}/Trigger/)) when [created](#creating-instance).
24+
25+
The `triggerFor` function is the [streamTrigger](GraphExecution.md#streamTrigger) function of the owning [GraphExecution](GraphExecution.md).
26+
27+
## Plan DataflowGraph { #plan }
428

529
```scala
630
plan(
731
flow: ResolvedFlow): FlowExecution
832
```
933

10-
`plan`...FIXME
34+
`plan` creates a [FlowExecution](FlowExecution.md) (for the given [ResolvedFlow](ResolvedFlow.md)) as follows:
35+
36+
FlowExecution | ResolvedFlow
37+
-|-
38+
[BatchTableWrite](BatchTableWrite.md) | [CompleteFlow](CompleteFlow.md)
39+
[StreamingTableWrite](StreamingTableWrite.md) | [StreamingFlow](StreamingFlow.md)
1140

1241
---
1342

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# ResolutionCompletedFlow
2+
3+
`ResolutionCompletedFlow` is...FIXME

0 commit comments

Comments
(0)

AltStyle によって変換されたページ (->オリジナル) /