Configuration as Code, use YAML to write automated workflows that run on Deno, with any Deno modules, Typescript/Javascript codes
With denoflow, you'll use yaml
to write automated workflow that runs on Deno, with any Deno modules, Typescript/Javascript code in your workflow. Denoflow will execute your workflow as you want. You can think of it as configuration-as-code.
Consider it an alternative to Zapier or IFTTT, for example fetching articles at regular intervals via RSS and then performing some tasks for non-repeating articles.
It's still at a very early stage, use with care!
If we need some GUI features, the ideal result is having some app schemas, using rjsf to render a web gui, gui can help us to generate yaml configs.
The ideal runtime is using cloud serverless platform, or CI platform, like Github Actions, Gitlab CI, self-hosted with Deno, or any Docker runtime.
Deno Deploy is not supported yet, because it doesn't support Code generation from strings, Error Detail:
Code generation from strings disallowed for this context
See Github Actions Example: test.yml
Now we can only write yaml by ourself, and actually, it's not that hard.
Join our Discord chat channel to discuss about Denoflow!
Stable deno land version see Denoflow
Try and exploring denoflow with the Online Playground
I have written actionsflow before, it must run in github actions, or local Docker, for workflow is too heavy, I found Deno features make it more suitable for doing flexible workflow based on yaml configuration, hope Denoflow can become a simple but powerful workflow assistant.
Install Deno first.
Fetch from Hacker News API to webhook example:
mkdir workflows
touch workflows/fetch.yml
sources:
- from: https://deno.land/x/[email protected]/mod.ts
use: get
args:
- https://test.owenyoung.com/slim.json
itemsPath: data.hits
key: objectID
limit: 1
steps:
- run: console.log('item', ctx.item)
# Open: <https://requestbin.com/r/enyvb91j5zjv9/23eNPamD4DK4YK1rfEB1FAQOKIj> , See live webhook request.
- from: https://deno.land/x/[email protected]/mod.ts
use: post
args:
- https://enyvb91j5zjv9.x.pipedream.net/
- ${{ctx.item}}
- headers:
'Content-Type': 'application/json'
Open: https://requestbin.com/r/enyvb91j5zjv9/23eNPamD4DK4YK1rfEB1FAQOKIj , See live webhook request.
deno run --allow-read --allow-net --allow-write --allow-env --allow-run https://deno.land/x/denoflow/cli.ts run
Or simplly with all permissions:
deno run -A https://deno.land/x/denoflow/cli.ts run
Denoflow will scan the
workflows
directory and run all valid.yml
files.
latest version:
https://denopkg.com/denoflow/denoflow@main/cli.ts
Try it at Online Playground
If you prefer to use fetch
:
sources:
- use: fetch
args:
- https://test.owenyoung.com/slim.json
run: return ctx.result.json()
itemsPath: hits
key: objectID
limit: 1
steps:
- use: fetch
args:
- https://enyvb91j5zjv9.x.pipedream.net/
- method: POST
headers:
Content-Type: application/json
body: ${{JSON.stringify(ctx.item)}}
Try it at Online Playground
touch workflows/rss.yml
sources:
- from: https://deno.land/x/[email protected]/sources/rss.ts
args:
- https://actionsflow.github.io/test-page/hn-rss.xml
limit: 1
steps:
- use: fetch
args:
- ${{env.DISCORD_WEBHOOK}}
- method: POST
headers:
Content-Type: application/json
body: ${{ JSON.stringify({content:ctx.item.title.value}) }}
Try it at Online Playground
Or, if you prefer more raw
way:
sources:
- use: fetch
args:
- https://actionsflow.github.io/test-page/hn-rss.xml
run: |
const rss = await import("https://deno.land/x/rss/mod.ts");
const xml = await ctx.result.text();
const feed = await rss.parseFeed(xml);
return feed.entries;
limit: 1
steps:
- use: fetch
args:
- ${{ctx.env.DISCORD_WEBHOOK}}
- method: POST
headers:
'Content-Type': 'application/json'
body: ${{ JSON.stringify({content:ctx.item.title.value}) }}
deno run --allow-read --allow-net --allow-write --allow-run --allow-env https://deno.land/x/denoflow/cli.ts run
Try it at Online Playground
A simple script generated exmaple:
sources:
- run: return [{id:"1"}]
force: true
steps:
- run: console.log("item",ctx.item);
Try it at Online Playground
More examples are in workflows directory, you can submit your awesome workflows.
Try Online Playground to explore workflow.
sources?
: where to fetch the data, Source[]
, can be one or more sources. Every source should return an array of items. e.g. [{"id":"1"}]
, the item key can be specified in key
field.
from
?: import ts/js script from url
or file path
use
?: run moduleName
from above from
, or if from
is not provided, run global function like fetch
, args
will be passed to the function, the return value will be attached to ctx.result
and ctx.sources[index].result
, if use
is a class, then ctx.result
will be the instance of the class. use
can also be Deno.cwd
things, to call Deno functions.run
?: run ts/js code, you can handle use
result here. Return a result that can be stringified to json. The return value will be attached to ctx.result
and ctx.sources[index].result
itemsPath
?: the path to the items in the result, like hits
in https://test.owenyoung.com/slim.json
key
?: the key to identify the item, like objectID
in https://test.owenyoung.com/slim.json
, if not provided, will use id
, denoflow will hash the id, then the same item with id
will be skipped.reverse?
, boolean
, reverse the itemsfilter?
, string
, script code, should handle ctx.item
-> return true
or false
cmd
: string
, exec a shell command after all other task, the return value will be attached to ctx.cmdResult
and ctx.sources[index].cmdResult
post?
: post script code, you can do some check, clean, things here, change ctx.statefilter
? filter from all sources items, handle ctx.items
, expected return a new boolean[]
,
from
?: import ts/js script from url
or file path
use
?: run moduleName
from above from
, or if from
is not provided, run global function like fetch
, args
will be passed to the function, the return value will be attached to ctx.result
and ctx.sources[index].result
, if use
is a class, then ctx.result
will be the instance of the class. use
can also be Deno.cwd
things, to call Deno functions.run
?: run ts/js code, you can handle use
result here.handle ctx.items
, expected return a new boolean[]
, flag which item will be used. e.g. run: return ctx.items.map(item => item.title.value.includes('test'))
cmd
?: string
, exec a shell command after all other task, the return value will be attached to ctx.cmdResult
and filter.cmdResult
post?
: post script code, you can do some check, clean, things here, change ctx.statesteps
? the steps to run, Step[]
, can be one or more steps.
from
?: import script from url
or file path
use
?: run moduleName
from above from
, or if from
is not provided, run global function like fetch
, args
will be passed to the function, the return value will be attached to ctx.result
and ctx.sources[index].result
, if use
is a class, then ctx.result
will be the instance of the class. use
can also be Deno.cwd
things, to call Deno functions.run
?: run ts/js code, you can handle use
result here. Return a result that can be stringified to json. the result will be attached to the ctx.steps[index].result
cmd
?: exec shell commands, will be run after run
, the result will be attached to the ctx.steps[index].cmdResult
post?
: post script code, you can do some check, clean, things here, change ctx.statepost
? final post script code, run after all steps done, you can do some check, clean, things here. You can use all steps params here.deno install -n denoflow --allow-read --allow-net --allow-write --allow-run --allow-env https://deno.land/x/denoflow/cli.ts
Then, you can run it with denoflow run
, or denoflow run <files>
deno cache --reload https://deno.land/x/denoflow/cli.ts
denoflow/0.0.17
Usage:
$ denoflow run [...files or url]
Options:
--force Force run workflow files, if true, will ignore to read/save state
--debug Debug mode, will print more info
--database Database uri, default json://data
--limit max items for workflow every runs
--sleep sleep time between sources, filter, steps, unit seconds
--stdin read yaml file from stdin, e.g. cat test.yml | denoflow run --stdin
-h, --help Display this message
If you are not yet familiar with YAML syntax, take 5 minutes to familiarize yourself with.
You can use ${{variable}}
in any fields to inject variables into your workflow, we inject ctx
variable in template and script. For example:
steps:
- if: ${{ctx.items.lengh>10}}
run: console.log(ctx.item);
All ctx
variables, See Interface
You can simply use ctx.state
to get or set state, for example:
let currentState = ctx.state || {};
let sent = ctx.state.sent || [];
if(sent.includes(ctx.item.id)){
sent.push(ctx.item.id);
}
ctx.state = {
sent
};
// deno flow will save the state for you , next you can read it.
return;
The state will be saved to data
folder in json
format. You can also use sqlite to store the state. Just set database: sqlite://data.sqlite
in your workflow config file.
Cause denoflow designed for serverless, simple, so it self can't schedule a workflow. You can use cron
or other trigger to schedule a workflow. For example:
*/15 * * * * deno run --allow-read --allow-net --allow-write --allow-env --allow-run https://deno.land/x/denoflow/cli.ts run workflows/schedule15.yml
Like above, denoflow can't handle webhook directly, you can forward the webhook to denoflow, For github actions example:
Webhook.yml
:
sources:
- run: return [ctx.env.event]
force: true
steps:
- run: console.log("item",ctx.item);
.github/workflows/webhook.yml
:
name: Webhook
on:
repository_dispatch:
workflow_dispatch:
jobs:
denoflow:
runs-on: ubuntu-latest
concurrency: denoflow
steps:
- name: Check out repository code
uses: actions/checkout@v2
- uses: denoland/setup-deno@v1
with:
deno-version: v1.x
- env:
event: ${{toJSON(github.event)}}
run: deno run --allow-read --allow-net --allow-write --allow-env --allow-run https://deno.land/x/denoflow/cli.ts run workflows/webhook.yml
If you write scripts directly, it will make maintenance very confusing, each script has to handle its own de-duplication and process management, using configuration files to write more readable, maintainable, to achieve a certain degree of low code.
clean
command