# Windmill > Windmill is an open-source and self-hostable workflow engine and developer platform to build, orchestrate, and monitor internal software at scale. Build workflows, internal tools, AI agents and automations. Full flexibility of code with 20+ languages, public and private library imports, and a local development experience optimized for AI (Claude Code, Cursor, CLI & VS Code extension). Enterprise-grade integrations with Postgres, Snowflake, Kafka, DuckDB and 50+ services. Enterprise-grade security with SSO, SAML, SCIM and audit logs. Built in Rust, optimized for scale. Website: https://www.windmill.dev Documentation: https://www.windmill.dev/docs GitHub: https://github.com/windmill-labs/windmill Cloud: https://app.windmill.dev Hub: https://hub.windmill.dev OpenAPI: https://app.windmill.dev/openapi.html This file contains the entire Windmill documentation as a single document. A curated per-page index with one-line descriptions is available at https://www.windmill.dev/llms.txt. ## Browser automation Source: https://www.windmill.dev/docs/advanced/browser_automation # Browser automation Windmill makes it easy to perform browser automation tasks, such as testing or web scraping. :::info Not sure what a worker group is? You should probably [read about it first](../../core_concepts/9_worker_groups/index.mdx). ::: By default, a worker group named `reports` is available which will handle jobs with the `chromium` tag. Workers assigned to this group will install chromium on start (learn more about [init scripts](../../core_concepts/9_worker_groups/index.mdx#init-scripts)). You have to set the worker group of at least one worker to `reports`. There is a sample worker container definition called `windmill_worker_reports` in the `docker-compose.yml` file which you can uncomment to quickly start a worker with the right worker group. The chromium binary will be available on these workers at `/usr/bin/chromium`. You will need to disable the sandbox to run it inside windmill workers. You can do this by passing the `--no-sandbox` flag. :::caution Running chromium without the sandbox is a security risk. Make sure you trust the website you are visiting. ::: To run jobs on a chromium-equipped worker, you have to select the `chromium` tag in the settings of the script or flow step. [Learn how here](../../core_concepts/9_worker_groups/index.mdx). ## Examples ### Playwright (Bun) ```typescript const page = await browser.newPage(); await page.goto("https://google.com"); const title = await page.title(); await browser.close() return title } ``` ### Puppeteer (Bun) ```typescript const page = await browser.newPage(); await page.goto("https://google.com"); const title = await page.title(); await browser.close(); return title; } ``` --- ## Canonical deployment setups Source: https://www.windmill.dev/docs/advanced/canonical_deployment_setups # Collaboration and deployment stages This page describes how to set up collaboration and deployment in Windmill as a progression of four stages, from simplest to most advanced. Each stage is additive: it builds on the previous one and adds one capability. Most teams settle at stage 2 or stage 3 — stage 4 is only needed when you need a separate production environment with a promotion workflow. | Stage | What you add | Git required | Forks | Workspaces | |-------|--------------|--------------|-------|------------| | [Stage 1](#stage-1--shared-workspace) | Shared workspace, `u/` → `f/` | No | No | 1 | | [Stage 2](#stage-2--add-forks-and-the-merge-ui) | Forks, merge UI, protection rulesets | No | Yes | 1 | | [Stage 3](#stage-3--add-git-sync) | Bi-directional git sync, local development | Yes | Yes | 1 | | [Stage 4](#stage-4--add-multi-workspace-promotion) | Multi-workspace (dev/staging/prod) | Yes | Yes | 2+ | :::info Edition requirements - **Stage 1** works on any edition. - **Stage 2** relies on [workspace forks](../20_workspace_forks/index.mdx) and [protection rulesets](../../core_concepts/56_protection_rulesets/index.mdx). Forks are available on all self-hosted editions but count toward the global 3-workspace limit on Community Edition. On [Windmill Cloud](https://app.windmill.dev/), forks require a [paid plan](../20_workspace_forks/index.mdx#forks-on-windmill-cloud). - **Stage 3** adds [git sync](../11_git_sync/index.mdx), available on [Cloud and Enterprise Self-Hosted](/pricing), and on Community Edition for workspaces with **up to 2 users**. - **Stage 4** requires git sync and is [Cloud or Enterprise Self-Hosted](/pricing) only. ::: ## Stage 1 — Shared workspace Everyone on the team works in the same workspace. Developers iterate in their own [user space](../../core_concepts/16_roles_and_permissions/index.mdx#path) (`u//`) and, once an item is ready to be shared, move it to a shared [folder](../../core_concepts/8_groups_and_folders/index.mdx) (`f//`). This is the simplest possible setup and the right starting point for almost every team. ### Architecture - One workspace (for example `main`) - Developers own `u//` and iterate there - Shared items live under `f//`, with folder-level [permissions](../../core_concepts/16_roles_and_permissions/index.mdx) granting write access to the relevant group - No git, no forks, no protection rulesets ### Setup 1. Create a workspace and invite your team. 2. Create one or more [folders](../../core_concepts/8_groups_and_folders/index.mdx#folders) that represent your projects, and assign write permission to the groups that should own each project. 3. That's it — developers can start building. ### Developer workflow 1. A developer creates a script, flow or app under `u//`. Items in user space are private to them by default. 2. They iterate freely, test, and share drafts with teammates via the [draft system](../../core_concepts/0_draft_and_deploy/index.mdx). 3. When the item is ready, they move it from `u//` to the appropriate `f//`. Moving an item into a folder transfers ownership to that folder's permissions. 4. Other members of the folder's group can now edit, run, and depend on the item. ### When to move to stage 2 - Someone broke an item in the shared workspace and you wish it had been isolated. - You want a review gate before changes reach the shared workspace. - Developers want sandboxes to test risky changes without affecting anyone else. ## Stage 2 — Add forks and the merge UI Stop editing the shared workspace directly. Instead, developers create a [workspace fork](../20_workspace_forks/index.mdx) for each change, iterate in the fork, and merge back via the [merge UI](../20_workspace_forks/index.mdx#merge-workspaces-from-the-ui-merge-ui). [Protection rulesets](../../core_concepts/56_protection_rulesets/index.mdx) enforce this: direct deploys to the shared workspace are blocked except for a small `wm_deployer` group. This is still a pure UI workflow — no git, no CI/CD, no external tools. Stage 3 will add git sync on top, and forks integrate seamlessly with it when you get there. :::note Forks are optional Forks are not a prerequisite for stage 3. Git sync works on a plain shared workspace, so if you already want git history and local development, you can skip this stage and go straight to [stage 3](#stage-3--add-git-sync). Adopt forks later if you need isolated sandboxes and a review gate. ::: ### Architecture - One shared workspace (same as stage 1), now called `prod` or `main` - [Protection rulesets](../../core_concepts/56_protection_rulesets/index.mdx) block direct deploys; a `wm_deployer` group has bypass permissions - Developers create [workspace forks](../20_workspace_forks/index.mdx) to develop features - Changes merge back through the [merge UI](../20_workspace_forks/index.mdx#merge-workspaces-from-the-ui-merge-ui) ![UI only forks](./workspace_forks_ui_only.png) ### Setup #### 1. Create the `wm_deployer` group 1. Open the workspace settings and go to **Groups**. 2. Create a group called `wm_deployer` and add the users who are allowed to deploy directly or approve merges. #### 2. Enable protection rulesets 1. Go to **Workspace settings** → **Protection Rulesets**. 2. Add a rule that enables **Disable direct deployment**. 3. Add the `wm_deployer` group to the bypass list. This blocks everyone except the deployer group from modifying `f/` items directly. Developers must now use forks. ### Developer workflow 1. A developer creates a [workspace fork](../20_workspace_forks/index.mdx#fork-creation) from the shared workspace via the workspace menu. 2. They iterate, test, and deploy freely *inside* the fork — triggers are not copied over to avoid unwanted executions. 3. When ready, they open the fork home page, click **Review & Deploy Changes**, and review the diff in the [merge UI](../20_workspace_forks/index.mdx#merge-workspaces-from-the-ui-merge-ui). 4. A member of `wm_deployer` approves the merge. The changes land in the shared workspace. ### When to move to stage 3 - You want a history of changes outside Windmill (audit, backup, rollback). - You want to edit scripts and flows locally in your IDE or with [agentic coding tools](../../misc/9_guides/local_dev_with_ai/index.mdx) like Claude Code. - You want fork branches mirrored in git so PRs can be reviewed there too. - You're planning to add a separate production environment (stage 4), which requires git sync. ## Stage 3 — Add git sync Connect the workspace from stages 1 and 2 to a git repository using [git sync](../11_git_sync/index.mdx). Every deploy is committed to git, and changes pushed to the repository are synced back to Windmill via CI/CD. This gives you a full audit trail, external version control, and [local development](../4_local_development/index.mdx) — and each fork from stage 2 now automatically gets its own git branch, so the merge UI and git PRs stay in lockstep. Git sync is available on [Cloud and Enterprise Self-Hosted](/pricing), and on Community Edition for workspaces with up to 2 users. ### Architecture - The shared workspace and its forks from stage 2, unchanged - A git repository with a single branch (for example `main`) - [Git sync](../11_git_sync/index.mdx) configured on the workspace: each deploy pushes a commit - A CI/CD workflow that syncs commits back to the workspace, enabling bi-directional sync - A `wmill.yaml` file in the repository with a single entry in the [`workspaces:`](../3_cli/environment-specific-items.mdx) key - Fork branches auto-created as `wm-fork//` ![Workspace Forks + git sync](./workspace_forks.png) ### Setup #### 1. Create the git repository Create an empty repository on GitHub, GitLab or similar with a single `main` branch. #### 2. Configure git sync on the workspace 1. Go to **Workspace settings** → **Git Sync**. 2. Click **+ Add connection** and create a [`git_repository`](../../integrations/git_repository.mdx) resource pointing at your repository and the `main` branch. 3. Save. Use **Initialize Git repository** to populate the repository with the current workspace contents and a default `wmill.yaml`. See the full walkthrough in [git sync setup](../11_git_sync/index.mdx#setup---git-sync-from-windmill). #### 3. Add the CI/CD workflows Git sync only pushes *from* Windmill *to* git by default. To make changes in git flow back into the workspace, add GitHub Actions workflows (or equivalent). A working reference is the [windmill-sync-example](https://github.com/windmill-labs/windmill-sync-example) repository. You will want: - [`push-on-merge.yaml`](../11_git_sync/index.mdx#setup---cicd-from-git-repository) — runs `wmill sync push` when commits land on `main`. - [`push-on-merge-to-forks.yaml`](../11_git_sync/index.mdx#setup---cicd-from-git-repository) and [`open-pr-on-fork-commit.yaml`](../11_git_sync/index.mdx#setup---cicd-from-git-repository) — keep fork branches in sync and open PRs automatically. Without the fork workflows, forks still work exactly as in stage 2 — their git branches just won't receive external edits. You will need: - A [Windmill user token](../../core_concepts/4_webhooks/index.mdx#user-token) saved as a `WMILL_TOKEN` GitHub secret. - The `WMILL_WORKSPACE` and `WMILL_URL` variables set in the workflows. Full details in [git sync CI/CD setup](../11_git_sync/index.mdx#setup---cicd-from-git-repository). #### 4. Minimal `wmill.yaml` A single-workspace `wmill.yaml` looks like this: ```yaml defaultTs: bun includes: - f/** excludes: [] skipVariables: true skipResources: true skipSecrets: true workspaces: main: baseUrl: https://app.windmill.dev gitBranch: main ``` See [workspace-specific items](../3_cli/environment-specific-items.mdx) for the full schema. ### Developer workflow 1. Developers continue to use forks from stage 2. When they create a fork, a matching `wm-fork/main/` git branch is created automatically. 2. They can now also edit the fork locally: [work on the workspace outside of Windmill](../4_local_development/index.mdx) with `wmill sync pull`, edit in their IDE, the [VS Code extension](../../cli_local_dev/1_vscode-extension/index.mdx), or an [agentic coding tool](../../misc/9_guides/local_dev_with_ai/index.mdx) like Claude Code, then `wmill sync push` back. 3. Every deploy creates a commit in the git repository. 4. Merging a fork via the merge UI *or* via a normal git PR both work — pick whichever your team prefers for review. 5. External commits (from local development or PR merges) are replayed into the workspace by the CI/CD workflow. ### When to move to stage 4 - You need a separate production environment that is fully isolated from day-to-day development. - You want PR-based review on the *promotion* step, not just the merge-to-shared step. - You're deploying across multiple Windmill instances. ## Stage 4 — Add multi-workspace promotion Introduce one or more additional workspaces (for example `staging` and `prod`), each connected to its own git branch. Day-to-day development happens on `staging` using forks (exactly as in stage 3). When changes are ready for production, they are promoted from `staging` to `prod` through [git promotion](../9_deploy_gh_gl/index.mdx), which opens a PR on the `prod` branch. The `prod` workspace is never edited directly. Its only source of change is a merged PR on the `prod` branch. ### Architecture - Two (or more) workspaces, each on its own git branch - Each workspace has its own `git_repository` resource and its own entry under `workspaces:` in `wmill.yaml` - [Git Promotion](../11_git_sync/index.mdx#git-promotion-workflow-cross-instance-deployment-using-a-git-workflow) is configured from `staging` to `prod` - Developers edit `staging` via [forks](../20_workspace_forks/index.mdx) (stage 2 pattern) - `prod` is protected by [protection rulesets](../../core_concepts/56_protection_rulesets/index.mdx) and only the promotion CI/CD can deploy to it | Workspace | Git branch | Promotes to | |-----------|------------|-------------| | staging | staging | prod | | prod | prod | — | ![Git Promotion](./git_promotion.png 'Edits to staging use the stage 2 fork workflow') ### Setup #### 1. Create the branches and workspaces 1. In your git repository, create a `staging` branch and a `prod` branch. 2. Create a `staging` workspace and a `prod` workspace in Windmill. #### 2. Configure git sync on each workspace For each workspace, go to **Workspace settings** → **Git Sync** and create a `git_repository` resource pointing at the corresponding branch: - `staging` workspace → `git_repository` on `staging` branch - `prod` workspace → `git_repository` on `prod` branch #### 3. Configure the promotion target On the `staging` workspace: 1. Under the git sync connection, click **Add promotion target**. 2. Create a **separate** `git_repository` resource targeting the `prod` branch (this is not the same resource as the one used for git sync). 3. Save. :::caution Two resources are required The git sync resource points at the workspace's *own* branch (`staging`). The promotion target points at the *target* branch (`prod`). They must be different `git_repository` resources, even if they point at the same repository. ::: #### 4. Add the CI/CD workflows - `push-on-merge.yaml` — one copy per workspace (`WMILL_WORKSPACE=staging`, `WMILL_WORKSPACE=prod`), triggered by merges to `staging` and `prod` respectively. - `open-pr-on-promotion-commit.yaml` — opens a PR on `prod` when staging promotes a change. - `push-on-merge-to-forks.yaml` and `open-pr-on-fork-commit.yaml` — unchanged from stage 3, now scoped to the `staging` workspace. See [git sync CI/CD setup](../11_git_sync/index.mdx#setup---cicd-from-git-repository) for the workflow files. #### 5. Multi-workspace `wmill.yaml` ```yaml defaultTs: bun includes: - f/** excludes: [] workspaces: staging: baseUrl: https://app.windmill.dev gitBranch: staging specificItems: resources: - "f/config/**" settings: true prod: baseUrl: https://app.windmill.dev gitBranch: prod specificItems: resources: - "f/config/**" settings: true ``` Each workspace declares its own `gitBranch`. Items listed under `specificItems` (for example, environment-specific database credentials) are stored per workspace on disk. See [workspace-specific items](../3_cli/environment-specific-items.mdx) for the full schema. :::info Per-folder ownership defaults on deploy By default, a newly deployed script, flow, app, schedule or trigger runs as whichever user pushed it. For CLI / CI/CD deploys that's the identity behind `WMILL_TOKEN` — typically a workspace admin or a dedicated deploy user that you've added to `wm_deployers`. If you want items under a folder to run as a specific service account instead — and to differ per environment — set `default_permissioned_as` rules on the folder in the target workspace: ```yaml # f/customer_x/folder.meta.yaml — committed per workspace via specificItems summary: '' display_name: customer_x owners: [] extra_perms: {} default_permissioned_as: - path_glob: 'jobs/**' permissioned_as: u/customer_x_svc # or g/customer_x or an email - path_glob: '**' permissioned_as: u/ops ``` Rules are ordered; the first `path_glob` (relative to the folder root) that matches an item wins. Applied only at **create time** and only when the caller is admin or a member of `wm_deployers` — existing items are untouched, and non-deployer users still deploy as themselves. Set `folders: ["f/**/folder.meta.yaml"]` under `specificItems` in `wmill.yaml` if you want different rules per workspace on the same path. Edit the rules in the folder settings UI, or commit them directly in `folder.meta.yaml`. The CLI also exposes per-item overrides for already-deployed items via `wmill script|flow|app|schedule|trigger set-permissioned-as `. Folder defaults are advisory: they only fill in the default when the pushed item does not already carry an explicit `on_behalf_of`. To opt in to CLI preview of which items will be affected by rules on the next push, set `syncBehavior: v1` in `wmill.yaml`. ::: #### 6. Protect `prod` Enable [protection rulesets](../../core_concepts/56_protection_rulesets/index.mdx) on the `prod` workspace with no bypass group — the only way in is the promotion PR. ### Developer workflow 1. A developer needs to change something. They [fork](../20_workspace_forks/index.mdx) the **staging** workspace (not prod). 2. They iterate in the fork and merge back to `staging` via the merge UI. This is the stage 2 workflow plus the git sync integration from stage 3. 3. When the change needs to go live, they trigger a [git promotion](../9_deploy_gh_gl/index.mdx) from `staging` to `prod`. 4. A PR is opened on the `prod` branch. The team reviews it in their git platform. 5. Merging the PR triggers the `push-on-merge.yaml` workflow, which syncs `prod` to the new state. ### Advantages - Fully isolated production environment with a git PR as the only gate. - Dev/staging/prod across multiple Windmill instances if needed. - Per-workspace configuration via [workspace-specific items](../3_cli/environment-specific-items.mdx). - All the benefits of stages 1–3 still apply. ## Choosing a stage Most teams should start at **stage 1** and only move up when they feel the pain the next stage solves. Stage 4 is real operational overhead and should not be the default — the majority of teams are well served by stages 2 or 3. - **Stage 1** if you have a small team, no compliance requirements, and a single environment is fine. - **Stage 2** if you want enforced review on every change and isolated sandboxes, but don't need external version control. - **Stage 3** if you want git history, backup, local development, and the option of git PR-based review on top of stage 2. - **Stage 4** if you need a separate production environment with a promotion gate, or you deploy across multiple Windmill instances. Stages 1 → 2 → 3 → 4 are designed to be adopted in order, but you can skip stages: a small team on Community Edition can go stage 1 → stage 3 (skip forks) to get git sync and local development without needing fork-based review. ## Related documentation --- ## Ci tests Source: https://www.windmill.dev/docs/advanced/ci_tests # CI test scripts Add a `test:` annotation at the top of any script to turn it into a CI test. When the tested [script](../../getting_started/0_scripts_quickstart/index.mdx) or [flow](../../getting_started/6_flows_quickstart/index.mdx) is deployed, the test runs automatically. ## Writing a test script ### Single target ```typescript // test: script/u/admin/my_script if (result !== 42) { throw new Error(`Expected 42, got ${JSON.stringify(result)}`); } return result; } ``` ### Multiple targets ```typescript // test: // script/u/admin/script_a // script/u/admin/script_b // flow/u/admin/my_flow export async function main() { // test all items } ``` ### Wildcards Annotations support glob patterns so a single test script can cover a whole folder tree: - `*` matches one path segment. - `**` matches any depth. ```typescript // test: script/u/admin/* // test: flow/u/team/** ``` The first pattern matches every script directly under `u/admin/`. The second matches every flow at any depth under `u/team/`. Wildcards can be combined with multi-target annotations on separate lines. ### Branching on the triggering runnable CI test jobs receive a `WM_TESTED_RUNNABLE` environment variable containing `{kind}/{path}` of the runnable that triggered the test (for example, `script/u/admin/my_script`). This lets a single test script cover multiple targets and branch on which one fired it. ```typescript // test: script/u/admin/* export async function main() { const tested = process.env.WM_TESTED_RUNNABLE!; // e.g. "script/u/admin/my_script" const path = tested.replace(/^script\//, ""); return await wmill.runScript(path, {}); } ``` ### Python ```python # test: script/u/admin/my_script import wmill def main(): result = wmill.run_script("u/admin/my_script", {}) assert result == 42 return result ``` In Python, the triggering runnable is exposed as `os.environ["WM_TESTED_RUNNABLE"]`. The annotation uses the comment syntax of each language (`//` for TypeScript, `#` for Python, etc.). The script creation page also provides ready-made CI test templates for TypeScript and Python. ## Where results appear Test scripts show a yellow **CI test** badge in the script list and detail page. On the detail page of the tested script or flow, each test is listed with its status (pass, fail, or running) and a link to the job run. Results auto-refresh while tests are running. In [workspace forks](../20_workspace_forks/index.mdx), CI results are also visible: - The fork banner shows a summary of passing, failing, and running tests across all changed items. - On the [comparison page](../20_workspace_forks/index.mdx#merge-workspaces-from-the-ui-merge-ui), each changed item displays a CI badge, and a test summary lists all test scripts with their latest results. --- ## Cli Source: https://www.windmill.dev/docs/advanced/cli # Command-line interface (CLI) The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal. You can also use it for various automation tasks, including [syncing](./sync.mdx) folders & [GitHub repositories](./sync.mdx), or just running all you scripts and flows. See [Installation](./installation.md) for getting started. ## Unified `list` / `get` / `new` subcommands Every item type supports the same set of subcommands so the CLI can be used as a full API client in shell scripts — piping JSON output to `jq`, etc. | Subcommand | Behavior | | ---------- | -------- | | ` list [--json]` | Lists items. With `--json`, outputs machine-readable JSON. | | ` get [--json]` | Pretty-prints an item. With `--json`, outputs the full API response. | | ` new [...]` | Bootstraps a local template file for the item. `bootstrap` remains as an alias for script/flow. | The supported types are: `script`, `flow`, `app`, `resource`, `resource-type`, `variable`, `schedule`, `folder`, and `trigger`. For `trigger`, pass `--kind ` (e.g. `http`, `websocket`, `kafka`, ...) to `new` and to `get` when multiple trigger kinds share a path. ```bash wmill script list --json | jq '.[].path' wmill trigger new f/http/webhook --kind http wmill resource get f/db/prod --json | jq .value.host ``` ## `wmill docs` The `wmill docs` command searches the Windmill documentation from your terminal. It calls the backend search endpoint and prints formatted results. ```bash wmill docs [--json] ``` ### Example ```bash wmill docs "how do I create a flow?" wmill docs "what are resources?" --json | jq .answer ``` Documentation search is an [Enterprise Edition](/pricing) feature — on a Community Edition instance, the command prints a message indicating EE is required. ## Using the CLI with AI coding assistants If you drive the CLI from Claude Code, Codex, Cursor, or a similar AI coding assistant, you can give the agent up-to-date `wmill` command reference by adding the [Context7](https://context7.com) plugin and pointing it at the Windmill CLI docs index at [`context7.com/windmill-labs/windmill-cli-docs`](https://context7.com/windmill-labs/windmill-cli-docs). See [Local development with AI](../../misc/9_guides/local_dev_with_ai/index.mdx) for the full local AI workflow, including `wmill init`, the VS Code extension, and the MCP server. --- ## App Source: https://www.windmill.dev/docs/advanced/cli/app # Apps ## Listing apps The `wmill app` list command is used to list all apps in the remote workspace. ```bash wmill app ``` ## Pushing an app Pushing an app to a Windmill instance is done using the `wmill app push` command. ```bash wmill app push ``` ### Arguments | Argument | Description | | ----------- | --------------------------------- | | `file_path` | The path to the app file to push. | ### Examples 1. Push the app located at `./my_app.json`. ```bash wmill app push ./my_app.json ``` ## Full-code app commands The CLI provides additional commands for [full-code apps](/docs/full_code_apps): ### Create a new full-code app ```bash wmill app new ``` Interactive wizard to scaffold a full-code app with React, Svelte or Vue. ### Start the dev server From the app directory: ```bash wmill app dev ``` Starts a local development server with hot reload and WebSocket backend. Options: `--port`, `--host`, `--entry`, `--no-open`. ### Generate lock files Generate `.lock` files for backend runnables with dependencies using the [`wmill generate-metadata`](./generate-metadata.md) command: ```bash wmill generate-metadata ``` To only update app lockfiles, use: ```bash wmill generate-metadata --skip-scripts --skip-flows ``` Options: `--yes`, `--dry-run`, `--default-ts`. :::info Legacy command Prior to the unified command, this was done with `wmill app generate-locks`. This command is now deprecated but still works. ::: ### Generate agent documentation From the app directory: ```bash wmill app generate-agents ``` Generates `AGENTS.md` and `DATATABLES.md` for AI coding agent context. ## Remote path format ```js //... ``` --- ## Datatable Source: https://www.windmill.dev/docs/advanced/cli/datatable # Datatables The `wmill datatable` commands let you query and manage [data tables](../../core_concepts/11_persistent_storage/data_tables.mdx) from the CLI. You can list available datatables, run SQL queries, version their schema with [migrations](#managing-migrations), or start an interactive PostgreSQL session. ## Listing datatables List all datatables in the current workspace. ```bash wmill datatable list [options] ``` ### Options | Option | Description | | -------- | --------------------------------- | | `--json` | Output as JSON (for piping to jq) | ### Example ```bash wmill datatable list ``` Output: ``` +------------+---------------+---------------+ | Name | Resource Type | Resource Path | +------------+---------------+---------------+ | main | postgresql | f/db/main | | analytics | postgresql | f/db/stats | +------------+---------------+---------------+ ``` ## Running a query Execute a SQL query against a datatable and display results. ```bash wmill datatable run [options] ``` ### Arguments | Argument | Description | | -------- | ---------------------------- | | `sql` | The SQL query to execute | ### Options | Option | Parameters | Description | | ------------------- | ---------- | ----------------------------------------------------- | | `-n, --name` | `name` | Datatable name to query (default: `main`) | | `-s, --silent` | | Output only the final result as JSON (for scripting) | ### Examples 1. Basic query: ```bash wmill datatable run "SELECT * FROM users LIMIT 10" ``` 2. Query a specific datatable: ```bash wmill datatable run -n analytics "SELECT COUNT(*) as count FROM events" ``` 3. Silent mode for scripting: ```bash wmill datatable run -s "SELECT version()" | jq '.version' ``` ## Managing migrations The `wmill datatable migrate` commands let you version the schema of a datatable with SQL migrations. Each migration is a pair of `.up.sql` / `.down.sql` files stored under `migrations/datatable//` and applied in timestamp order. Applied migrations are tracked in a `_wm_migrations` table inside each datatable, so a migration is never run twice. Migrations are ordinary [workspace files](./sync.mdx): `wmill sync pull` writes them locally, and `wmill sync push` upserts (or deletes) them on the remote workspace. You can also manage the same migrations from the UI in `workspace settings` -> `Data Tables` -> `Migrations`. See the [data tables migrations](../../core_concepts/11_persistent_storage/data_tables.mdx#migrations) documentation for the overall workflow. ### Scaffolding a migration Create an empty `.up.sql` / `.down.sql` migration pair. This is a purely local operation - no network call. ```bash wmill datatable migrate new [options] ``` The name may only contain letters, digits, `_` and `-`. The files are written to `migrations/datatable//_.up.sql` (and `.down.sql`), where `` is the current UTC time as `YYYYMMDDHHMMSS`. #### Arguments | Argument | Description | | -------- | -------------------------------------- | | `name` | Name of the migration (used in the filename and DB record) | #### Options | Option | Parameters | Description | | ------------------- | ------------ | ------------------------------------------ | | `-d, --datatable` | `datatable` | Target datatable (default: `main`) | #### Example ```bash wmill datatable migrate new add_users_table ``` Then edit the generated files, for example: ```sql -- migrations/datatable/main/20260617120000_add_users_table.up.sql BEGIN; CREATE TABLE users ( id BIGINT PRIMARY KEY, name TEXT NOT NULL ); END; ``` ```sql -- migrations/datatable/main/20260617120000_add_users_table.down.sql BEGIN; DROP TABLE users; END; ``` The `.down.sql` file is optional: leave it empty (or delete it) for migrations you never intend to roll back. ### Applying migrations Apply every pending migration, in timestamp order, to a datatable. ```bash wmill datatable migrate up [options] ``` Without a `--datatable` flag, `up` targets the `main` datatable. Any migration files created or edited locally are pushed to the workspace first, so `migrate up` works even before a `wmill sync push`. #### Options | Option | Parameters | Description | | ------------------- | ------------ | ------------------------------------------ | | `-d, --datatable` | `datatable` | Target datatable (default: `main`) | #### Examples 1. Apply pending migrations to the `main` datatable: ```bash wmill datatable migrate up ``` 2. Apply pending migrations to a specific datatable: ```bash wmill datatable migrate up -d analytics ``` ### Rolling back migrations Roll back the most recently applied migration (one step), running its `.down.sql`. ```bash wmill datatable migrate down [options] ``` Without a `--datatable` flag, `down` rolls back the latest migration on the `main` datatable. #### Options | Option | Parameters | Description | | ------------------- | ------------ | ------------------------------------------ | | `-d, --datatable` | `datatable` | Target datatable (default: `main`) | #### Example ```bash wmill datatable migrate down -d analytics ``` ### Syncing migrations Migrations sync like any other workspace item: - `wmill sync pull` writes migration files under `migrations/datatable//`. - `wmill sync push` upserts changed migrations and deletes those removed locally. When the push introduces new migrations, the CLI offers to run them. Local migration sets are validated on push and rejected if two `.up` (or two `.down`) files share a timestamp, or if a `.down.sql` has no matching `.up.sql`. ## Starting a PostgreSQL proxy server Start a PostgreSQL wire-protocol proxy that serves all datatables. This allows any Postgres-compatible client (psql, DBeaver, pgAdmin, etc.) to connect and query your datatables. ```bash wmill datatable serve [options] ``` ### Options | Option | Parameters | Description | | ------------ | ---------- | ----------------------------------------------------- | | `--port` | `port` | Port to listen on (default: first available 5433-5500)| | `--host` | `host` | Bind address (default: `127.0.0.1`) | | `--password` | `password` | Connection password (default: randomly generated) | ### Behavior - Implements the PostgreSQL wire protocol (read-only queries only) - Each datatable appears as a separate database in the connection - Supports prepared statements and parameterized queries - Emulates Postgres system tables (like `pg_database`) for tool compatibility ### Example ```bash wmill datatable serve ``` Output: ``` Serving datatables on 127.0.0.1:5433 via Postgres wire protocol Available datatables: psql 'postgresql://wmill:a1b2c3d4e5f6@127.0.0.1:5433/main' psql 'postgresql://wmill:a1b2c3d4e5f6@127.0.0.1:5433/analytics' Press Ctrl+C to stop. ``` Connect with any Postgres client using the connection string shown. ## Interactive psql session Start a proxy server and launch an interactive `psql` session connected to it. ```bash wmill datatable psql [options] ``` ### Options | Option | Parameters | Description | | ------------------- | ---------- | ----------------------------------------------------- | | `-n, --name` | `name` | Datatable to connect to (default: `main`) | | `--port` | `port` | Port for the proxy (default: first available 5433-5500)| | `--host` | `host` | Bind address for the proxy (default: `127.0.0.1`) | | `--password` | `password` | Connection password (default: randomly generated) | ### Prerequisites The `psql` client must be installed: - **Linux**: `apt install postgresql-client` - **macOS**: `brew install libpq` ### Examples 1. Interactive session with the default datatable: ```bash wmill datatable psql ``` 2. Connect to a specific datatable: ```bash wmill datatable psql -n analytics ``` ## Summary | Command | Purpose | | ---------------- | ---------------------------------------------- | | `list` | List all datatables in the workspace | | `run` | Execute a single SQL query | | `migrate new` | Scaffold a new `.up.sql` / `.down.sql` migration | | `migrate up` | Apply all pending migrations | | `migrate down` | Roll back the most recent migration | | `serve` | Start a Postgres proxy for external clients | | `psql` | Launch an interactive psql session | --- ## Dev preview Source: https://www.windmill.dev/docs/advanced/cli/dev-preview # Dev & Preview The CLI can run scripts and flows against a remote workspace without deploying them, and it can serve a live-reload dev page that mirrors your local files. This is useful for iterating on local files, validating codebase scripts, or testing flow / app changes with local inline script modifications. ## `wmill dev` `wmill dev` starts a local live-reload server and opens a Windmill dev page that renders your local flow, script, or raw app. Edits on disk reload the page; edits in the page round-trip back to the local files (`flow.yaml`, inline scripts, `app.yaml`, etc.). ```bash wmill dev [--path ] [--proxy-port ] [--no-open] ``` When invoked from inside a `*__flow/` folder, `wmill dev` auto-detects the wm path. When invoked from a workspace root with no `--path`, the dev page shows a workspace picker — flows, scripts, and raw apps grouped by folder / user, with search and a kind filter. ### Run modes - **Direct** (default): a WebSocket server on a random port. The dev page lives on the remote workspace and connects back to localhost over WebSocket for live reload. This is the mode the [VS Code extension](../../cli_local_dev/1_vscode-extension/index.mdx) iframe and a regular browser tab use. - **Proxy** (`--proxy-port `): a reverse proxy at `http://localhost:/` that 302-redirects `/` to the remote dev page with workspace, auth token, and `--path` baked in, and tunnels other HTTP / WebSocket traffic to the remote workspace. This is the mode [Claude Code](../../misc/9_guides/local_dev_with_ai/index.mdx)'s port-detection preview pane uses. ### Options | Option | Description | | ----------------------- | -------------------------------------------------------------------------------------------- | | `--path ` | Open a specific flow, script, or raw app instead of the workspace picker. | | `--proxy-port ` | Run in proxy mode on the given port; needed for Claude Code's preview pane. | | `--no-open` | Don't open the browser; print the URL so the caller (e.g. an AI agent) can hand it to you. | ### Examples ```bash # Workspace picker wmill dev # Jump straight into a flow wmill dev --path f/my_flows/etl # Auto-detected when run from inside a flow folder cd f/my_flows/etl__flow && wmill dev # Proxy mode for Claude Code's preview pane wmill dev --proxy-port 4000 --path f/my_flows/etl ``` ### Round-trip editing Editing a flow or script from the dev page writes back to disk: - `flow.yaml` is rewritten only when content actually differs. - Inline scripts (`.py`, `.ts`, `.bun.ts`, `.go`, `.sh`, etc.) are kept in sync. - `PathScript` references (modules with `type: 'script'` and a `path:`) survive the round-trip. - Fixture files inside the flow folder (`README.md`, `data.json`, `.env`, etc.) are preserved. When `--path` is set, edits to other files don't push to the page, so you can keep working on unrelated files in the same workspace folder. :::tip Data pipelines [Pipelines](../../core_concepts/63_pipelines/index.mdx) have their own analog: `wmill pipeline dev` watches a folder of `// pipeline` scripts and live-previews the pipeline graph in the browser on every save. See [Pipelines (CLI)](./pipeline.md#live-preview-with-wmill-pipeline-dev). ::: ## `wmill script preview` Preview a local script file against the remote workspace. Supports both regular scripts and [codebase](../../core_concepts/33_codebases_and_bundles/index.mdx) scripts (which are bundled before running). ```bash wmill script preview [options] ``` ### Options | Option | Description | | ------------------ | ----------------------------------------------------------------- | | `-d, --data `| Inputs as a JSON string, `@filename`, or `@-` for stdin. | | `-s, --silent` | Only output the final result (no logs, useful for scripting). | ### Examples ```bash # Regular script wmill script preview u/admin/my_script.ts --data '{"x": 5}' # Codebase script (bundled before preview) wmill script preview f/codebase_test/my_script.ts --data '{"x": 7}' # Silent mode for piping wmill script preview f/scripts/hello.ts -d '{"n":3}' --silent | jq ``` ## `wmill flow preview` Preview a local flow against the remote workspace. The flow definition is read from the local `.flow` / `__flow` folder — any changes to inline scripts are picked up without deploying. ```bash wmill flow preview [options] ``` ### Options Same `-d, --data` and `-s, --silent` options as `script preview`. ### Example ```bash wmill flow preview f/my_flows/etl__flow --data '{"date":"2026-01-01"}' ``` Use `wmill dev` when you want a live UI that round-trips edits back to disk; use `wmill flow preview` when you want a one-shot run with a specific payload. --- ## Environment specific items Source: https://www.windmill.dev/docs/advanced/cli/environment-specific-items # Workspace-specific items Workspace-specific items let you store different versions of [resources](../../core_concepts/3_resources_and_types/index.mdx), [variables](../../core_concepts/2_variables_and_secrets/index.mdx), triggers, folders, and workspace settings per workspace (dev / staging / prod). The CLI transforms file paths so that each workspace has its own namespaced files locally while keeping clean base paths in the Windmill workspace itself. :::tip Multi-workspace setups are stage 4 of the collaboration guide Using `workspaces:` with multiple entries (one per environment, each with its own `gitBranch`) corresponds to **stage 4** of the [collaboration and deployment stages](../23_canonical_deployment_setups/index.mdx#stage-4--add-multi-workspace-promotion) guide. The single-workspace form of `workspaces:` is used starting at [stage 3](../23_canonical_deployment_setups/index.mdx#stage-3--add-git-sync). ::: This replaces the older split between `gitBranches` (multi-branch) and `environments` (single-branch). Both are now a single `workspaces:` key — see [Migrating from `gitBranches` / `environments`](./sync.mdx#migrating-from-gitbranches--environments). ## How it works When a file matches a pattern in your `wmill.yaml` configuration, the CLI inserts the workspace name into the local file path: - **Local files**: `database.dev.resource.yaml`, `database.prod.resource.yaml` - **Windmill workspace**: `database.resource.yaml` (clean base path) Each workspace has its own version of the same logical resource. In practice, multiple workspace-specific files usually coexist in the repository (e.g. `database.dev.resource.yaml` and `database.prod.resource.yaml` side by side) — by design when one git branch hosts multiple workspaces, or as a result of merging branches when each workspace has its own branch. The CLI only reads the files matching the current workspace and ignores the others. ## Configuration Workspace-specific items are declared inside a `workspaces:` entry, either per workspace under `specificItems`, or shared across every workspace under `commonSpecificItems`. ```yaml workspaces: dev: baseUrl: https://dev.windmill.dev overrides: skipSecrets: true specificItems: resources: - "u/alex/config/**" variables: - "u/alex/env_*" triggers: - "u/alex/kafka_*" folders: - "f/env_*" settings: true prod: baseUrl: https://app.windmill.dev gitBranch: main # workspace "prod" lives on git branch "main" overrides: skipSecrets: false specificItems: resources: - "u/alex/config/**" variables: - "u/alex/env_*" folders: - "f/env_*" settings: true # Items that are workspace-specific across ALL workspaces commonSpecificItems: resources: - "u/alex/config/**" variables: - "u/alex/database_*" folders: - "f/env_*" settings: true ``` The `workspaces` key is the map key (`dev`, `prod`, …) and is also the suffix inserted into file names on disk. `gitBranch` and `workspaceId` default to that key — only set them when they differ. ## Resolving the current workspace `wmill sync pull` and `wmill sync push` pick the workspace in this order: 1. `--workspace ` — explicit override, looks up the entry by key. 2. **Current git branch** — the first entry whose effective `gitBranch` matches `git rev-parse --abbrev-ref HEAD`. 3. No match — top-level sync options are used as-is, no workspace-specific transform. ### One git branch per workspace Each workspace has its own git branch. The CLI detects the current branch automatically and applies the matching entry. ```bash git checkout main wmill sync pull # Uses workspace "prod" (gitBranch: main) # Creates: u/alex/config/database.prod.resource.yaml git checkout dev wmill sync pull # Uses workspace "dev" # Creates: u/alex/config/database.dev.resource.yaml ``` ### Multiple workspaces on one git branch All workspaces live on the same branch and you pick the target with `--workspace`. ```bash wmill sync pull --workspace dev # Creates: u/alex/config/database.dev.resource.yaml wmill sync pull --workspace prod # Creates: u/alex/config/database.prod.resource.yaml wmill sync push --workspace staging ``` No `git checkout` is needed — all workspace-specific files coexist on the same branch. :::info Migrating from `--env` and `--branch` `--branch` and `--env` are still accepted on sync commands but are deprecated — they print a one-time warning and resolve the same way `--workspace` does. Update any scripts or CI jobs to use `--workspace` instead. ::: ## File path transformation ### Transform logic **Pull** (Windmill workspace → local): - `u/alex/database.resource.yaml` → `u/alex/database.prod.resource.yaml` - `u/alex/orders.kafka_trigger.yaml` → `u/alex/orders.prod.kafka_trigger.yaml` - `f/env_staging/folder.meta.yaml` → `f/env_staging/folder.prod.meta.yaml` - `settings.yaml` → `settings.prod.yaml` **Push** (local → Windmill workspace): - `u/alex/database.dev.resource.yaml` → `u/alex/database.resource.yaml` - `u/alex/orders.dev.kafka_trigger.yaml` → `u/alex/orders.kafka_trigger.yaml` - `f/env_staging/folder.dev.meta.yaml` → `f/env_staging/folder.meta.yaml` - `settings.dev.yaml` → `settings.yaml` The suffix is the **workspace name** (the key in `workspaces:`). In migrated configs where the workspace name equals the old branch name, existing files keep working unchanged. ### Resource files Resources can include associated files (certificates, config files, etc.) alongside their YAML definitions. These follow the same transformation: - Base file: `u/alex/certificate.resource.file.pem` - Workspace-specific: `u/alex/certificate.dev.resource.file.pem` When a resource YAML file is marked as workspace-specific, all associated resource files are automatically treated the same way. ### Supported file types - **Variables**: `*.variable.yaml` - **Resources**: `*.resource.yaml` and resource files (`*.resource.file.*`) - **Triggers**: `*.kafka_trigger.yaml`, `*.http_trigger.yaml`, `*.websocket_trigger.yaml`, `*.nats_trigger.yaml`, `*.postgres_trigger.yaml`, `*.mqtt_trigger.yaml`, `*.sqs_trigger.yaml`, `*.gcp_trigger.yaml` - **Schedules**: `*.schedule.yaml` - **Folders**: `folder.meta.yaml` files within folder directories - **Settings**: `settings.yaml` (workspace settings file) ## Pattern matching Patterns support standard glob syntax: - `*` matches any characters within a path segment - `**` matches any characters across path segments - `u/alex/database_*` matches `u/alex/database_config`, `u/alex/database_url`, etc. - `f/environments/**` matches all files under `f/environments/` recursively ## Common specific items Items that should be workspace-specific across all workspaces can be declared once under `commonSpecificItems`: ```yaml workspaces: commonSpecificItems: variables: - "u/alex/database_*" - "f/config/**" resources: - "u/alex/api_keys/**" - "f/environments/**" triggers: - "u/alex/kafka_*" - "f/streaming/**" folders: - "f/env_*" settings: true ``` ## Per-workspace specific items If some resources only exist in certain workspaces, define different patterns per workspace instead of using `commonSpecificItems`: ```yaml workspaces: prod: specificItems: variables: - "u/alex/prod_*" resources: - "u/alex/production/**" triggers: - "u/alex/prod_kafka_*" folders: - "f/prod_config" settings: true dev: specificItems: variables: - "u/alex/dev_*" resources: - "u/alex/development/**" triggers: - "u/alex/dev_kafka_*" folders: - "f/dev_config" settings: true ``` Here, `prod_kafka_*` triggers only get the workspace suffix on `prod` — they don't exist in `dev`, so there's no need to mark them there. Most setups use `commonSpecificItems` since the same resources typically exist across all workspaces. ## Name safety Workspace names containing certain characters are automatically sanitized for filesystem safety: - **Unsafe characters**: `/ \ : * ? " < > | .` - **Replacement**: All unsafe characters are replaced with `_` - **Warning**: The CLI shows a clear warning when sanitization occurs ```bash Warning: Branch name "feature/api-v2.1" contains filesystem-unsafe characters (/ \ : * ? " < > | .) and was sanitized to "feature_api-v2_1". This may cause collisions with other similarly named branches. ``` Prefer simple workspace keys (`dev`, `staging`, `prod`) to avoid collisions. ## Best practices - **Start broad**: Use `commonSpecificItems` for widely-used resources - **Be specific**: Target exact paths rather than overly broad patterns - **Group logically**: Organize patterns by team, workspace, or function - **Define patterns early**: Establish patterns before team members start using them - **Test locally**: Verify workspace-specific behavior works correctly before CI/CD integration ## Troubleshooting ### Files not being detected as workspace-specific 1. **Check patterns**: Ensure your glob patterns match the file paths exactly 2. **Verify file types**: Only the types listed above are supported 3. **Pattern testing**: Use tools like [globster.xyz](https://globster.xyz/) to test your patterns ### Files syncing to the wrong workspace 1. **Check config**: Ensure your `workspaces:` entry is correct 2. **Verify current context**: Make sure you're on the expected git branch or using the right `--workspace` flag 3. **Configuration precedence**: CLI flags override configuration file settings ## Git sync integration When [Git sync](../11_git_sync/index.mdx) is configured, workspace-specific items work seamlessly with your Git repository setup. On each Windmill-triggered sync, the backend invokes the CLI with `--base-url` and `--workspace` set to the current instance and workspace id. The CLI matches these against the `workspaces:` entries in `wmill.yaml` (on `baseUrl` + `workspaceId`) and applies the corresponding entry — no extra configuration on the Windmill side. - **One git branch per workspace**: each Windmill workspace's `git_repository` resource points to a different branch, and the `wmill.yaml` entry for that workspace sets `gitBranch` accordingly. - **Production workspace**: `git_repository` pointing to `repoX` on `main` branch - Changes to `database.resource.yaml` → pushed as `database.prod.resource.yaml` - **Development workspace**: `git_repository` pointing to `repoX` on `dev` branch - Changes to `database.resource.yaml` → pushed as `database.dev.resource.yaml` - **Multiple workspaces on one git branch** (mono-branch): all Windmill workspaces point to the same branch. List each workspace under `workspaces:` with a distinct `workspaceId` (and the same `gitBranch`); the CLI selects the right entry from the `--workspace` flag the backend passes, so no UI toggle is needed. ## Related documentation - [CLI sync](./sync.mdx) - Main sync operations and configuration - [Git sync settings](./gitsync-settings.mdx) - Managing sync configuration - [Git sync](../11_git_sync/index.mdx) - Backend Git integration --- ## Flow Source: https://www.windmill.dev/docs/advanced/cli/flow # Flows ## Listing flows The `wmill flow` list command is used to list all flows in the remote workspace. ```bash wmill flow ``` ## Pushing a flow Pushing a flow to a Windmill instance is done using the `wmill flow push` command. ```bash wmill flow push ``` ### Arguments | Argument | Description | | ------------- | -------------------------------------------------------------- | | `file_path` | The path to the flow file to push. | | `remote_path` | The remote path where the flow specification should be pushed. | ### Examples 1. Push the flow located at `path/to/local/flow.yaml` to the remote path `f/flows/test`. ```bash wmill flow push path/to/local/flow.yaml f/flows/test ``` ## Creating a new flow The wmill flow bootstrap command is used to create a new flow locally. ```bash wmill flow bootstrap [--summary ] [--description ] ``` ### Arguments | Argument | Description | | ---------- | ------------------------------------ | | `path` | The path of the flow to be created. | ### Examples 1. Create a new flow `f/flows/flashy_flow` ```bash wmill flow bootstrap f/flows/flashy_flow ``` ## Running a flow Running a flow by its path is done using the `wmill flow run` command. Logs are streamed step-by-step with labeled headers showing each module's ID and summary. For-loop iterations are tracked individually as they complete. ```bash wmill flow run [options] ``` ### Arguments | Argument | Description | | ------------- | ------------------------------- | | `remote_path` | The path of the flow to be run. | ### Options | Option | Parameters | Description | | -------------- | ---------- | ----------------------------------------------------------------------------- | | `-d, --data` | `data` | Inputs specified as a JSON string or a file using @filename or stdin using @- . Resources and variables must be passed using "$res:..." or "$var:..." For example `wmill flow run u/henri/message_to_slack -d '{"slack":"$res:u/henri/henri_slack_perso","channel":"general","text":"hello dear team"}'` | | `-s, --silent` | | Do not output anything other then the final output. Useful for scripting. | ![CLI arguments](../../assets/cli/cli_arguments.png "CLI arguments") :::tip Inspecting flow runs after completion Use `wmill job get ` to see a hierarchical step tree with status and durations, or `wmill job logs ` to see aggregated logs from all steps. See [Jobs](./job.md) for details. ::: ## Update flow inline scripts lockfile Flows inline script [lockfiles](../6_imports/index.mdx) can be updated locally using the [`wmill generate-metadata`](./generate-metadata.md) command: ```bash wmill generate-metadata ``` This command handles scripts, flows and apps in a single pass. To only update flow lockfiles, use: ```bash wmill generate-metadata --skip-scripts --skip-apps ``` :::info Legacy command Prior to the unified command, this was done with `wmill flow generate-locks`. This command is now deprecated but still works. ::: ## Flow specification You can find the definition of the flow file structure [here](../../openflow/index.mdx). ## Remote path format ```js //... ``` --- ## Folder Source: https://www.windmill.dev/docs/advanced/cli/folder # Folder ## Listing folders The `wmill folder` list command is used to list all folders in the remote workspace. ```bash wmill folder ``` ## Push The `wmill folder push` command is used to push a local folder specification to a remote location, overriding any existing remote versions. ```bash wmill folder push ``` ### Arguments | Argument | Description | | ------------- | ---------------------------------------------------------------- | | `folder_path` | The path to the local folder. | | `remote_path` | The path to the remote location where the folder will be pushed. | ## Adding missing folders When you create scripts, flows, or apps under `f//` without a corresponding `f//folder.meta.yaml` file, `wmill sync push` will either warn (admins) or fail (non-admins, due to RLS) because the remote folder does not exist. The `wmill folder add-missing` command scans every `f//` subdirectory and creates a default `folder.meta.yaml` for any that are missing one. ```bash wmill folder add-missing [-y] ``` ### Options | Option | Description | | ------ | ---------------------------------- | | `-y` | Skip the confirmation prompt. | ### Example ```bash # Dry scan, then create the missing files after confirmation wmill folder add-missing # Non-interactive (useful in scripts) wmill folder add-missing -y ``` `wmill sync push` runs the same detection automatically and suggests `wmill folder add-missing` when folders are missing. ## Stale script detection `wmill generate-metadata` builds a dependency tree across scripts, flows, apps, and [workspace dependencies](../../core_concepts/55_workspace_dependencies/index.mdx), then propagates staleness along that graph. This means if script `C` changes, any scripts `A` and `B` that transitively import `C` (including via relative imports like `./helper` or `../shared/utils`) are correctly detected as stale and regenerated. When you run `wmill generate-metadata f/lib`, scripts **outside** `f/lib` that import from it are included in the run by default. Use `--strict-folder-boundaries` to restrict the run to items physically inside the folder — excluded items that would otherwise have been regenerated are printed as warnings. See [`wmill generate-metadata`](./generate-metadata.md) for the full option list. --- ## Generate metadata Source: https://www.windmill.dev/docs/advanced/cli/generate-metadata # Generate metadata The `wmill generate-metadata` command generates metadata (locks, schemas) for all scripts, flows and apps. It replaces the previous separate commands (`wmill script generate-metadata`, `wmill flow generate-locks`, `wmill app generate-locks`). ## Usage ```bash wmill generate-metadata [folder] [options] ``` ## Options | Option | Description | | ------ | ----------- | | `--yes` | Skip confirmation prompt | | `--dry-run` | Show what would be updated without making changes | | `--lock-only` | Regenerate only lock files | | `--schema-only` | Regenerate only script schemas (skips flows and apps) | | `--skip-scripts` | Skip processing scripts | | `--skip-flows` | Skip processing flows | | `--skip-apps` | Skip processing apps | | `--strict-folder-boundaries` | Only update items inside the specified folder (requires folder argument) | | `-i, --includes ` | Comma-separated patterns to specify which files to include | | `-e, --excludes ` | Comma-separated patterns to specify which files to exclude | The `rehash` subcommand (see [Rehash](#rehash) below) rewrites lockfile hashes from on-disk content without any backend round-trip. ## Arguments | Argument | Description | | -------- | ----------- | | `folder` | Optional folder path to filter metadata generation | ## Examples ### Generate metadata for entire workspace ```bash wmill generate-metadata ``` ### Preview changes without applying ```bash wmill generate-metadata --dry-run ``` ### Auto-confirm all updates ```bash wmill generate-metadata --yes ``` ### Generate only for a specific folder ```bash wmill generate-metadata f/my_folder ``` ### Strict folder boundaries ```bash wmill generate-metadata f/my_folder --strict-folder-boundaries ``` ### Only update lockfiles ```bash wmill generate-metadata --lock-only ``` ### Only update schemas ```bash wmill generate-metadata --schema-only ``` ### Include specific patterns ```bash wmill generate-metadata -i "f/production/*,f/shared/*" ``` ### Exclude specific patterns ```bash wmill generate-metadata -e "f/test/*,f/drafts/*" ``` ## Rehash `wmill generate-metadata rehash [folder]` rewrites `wmill-lock.yaml` hashes directly from on-disk content. It does not contact the backend, does not regenerate scripts/flows/apps, and does not rewrite YAML files — it only recomputes the staleness hashes. Use it when: - **`wmill generate-metadata` flags items as stale that you didn't change.** Common causes: an older CLI wrote the lockfile with a different YAML library or unsorted hash keys, leaving entries that no longer match what the current CLI computes. `rehash` rebuilds those entries from disk so they line up again. - **Your `wmill-lock.yaml` has duplicate `./`-prefixed entries** (e.g. `./f/foo+__script_hash` next to `f/foo+__script_hash`). These come from past `wmill generate-metadata ./folder` invocations on older CLIs. `rehash` deduplicates them by writing canonical (non-prefixed) keys. - **You bootstrapped a long-lived `wmill sync`-managed repo** and want to fill in lockfile entries for items that pre-date the lockfile, without redeploying everything. ### Options | Option | Description | | ------ | ----------- | | `--skip-scripts` | Skip processing scripts | | `--skip-flows` | Skip processing flows | | `--skip-apps` | Skip processing apps | | `-i, --includes ` | Comma-separated patterns to specify which files to include | | `-e, --excludes ` | Comma-separated patterns to specify which files to exclude | ### Examples ```bash # Rebuild every lockfile entry from disk wmill generate-metadata rehash # Scope to a single folder wmill generate-metadata rehash f/billing # Rehash scripts only wmill generate-metadata rehash --skip-flows --skip-apps ``` `rehash` is non-destructive: it never touches your scripts, flows, apps, or YAML files. It only writes to `wmill-lock.yaml`. :::caution Trust, don't verify `rehash` accepts the on-disk content as canonical without checking it against the backend. Don't use it as a substitute for a regular `wmill generate-metadata` run when you actually want to validate that locks/schemas are up to date. Use it only when you've already confirmed the disk content is what you want and just need the lockfile to stop reporting it as stale. ::: `wmill sync pull` runs an automatic, lighter version of this for items that exist on disk but have no lockfile entry — so for typical pull-then-push workflows you don't need to invoke `rehash` manually. The auto-fill is skipped under `--json` and `--dry-run` so non-interactive scripts and previews don't mutate `wmill-lock.yaml`. ## Migration from legacy commands The following commands are now deprecated: | Deprecated command | Replacement | | ------------------ | ----------- | | `wmill script generate-metadata` | `wmill generate-metadata --skip-flows --skip-apps` | | `wmill flow generate-locks` | `wmill generate-metadata --skip-scripts --skip-apps` | | `wmill app generate-locks` | `wmill generate-metadata --skip-scripts --skip-flows` | The legacy commands will show deprecation warnings but continue to work. For centralized dependency management, see [workspace dependencies](../../core_concepts/55_workspace_dependencies/index.mdx). --- ## Gitsync settings Source: https://www.windmill.dev/docs/advanced/cli/gitsync-settings # Git sync settings Managing git-sync configuration between your local `wmill.yaml` file and the Windmill workspace backend is made easy using the wmill CLI. Git-sync settings operations are behind the `wmill gitsync-settings` subcommand. The `gitsync-settings` command allows you to synchronize filter settings, path configurations, and branch-specific overrides without affecting the actual workspace content (scripts, flows, apps, etc.). Git-sync settings management is done using `wmill gitsync-settings pull` & `wmill gitsync-settings push`. Settings operations are configuration-only and work with your `wmill.yaml` file structure. The command will show you the list of changes and ask for confirmation before applying them. When pulling, the source is the remote workspace git-sync configuration and the target is your local `wmill.yaml` file, and when pushing, the source is your local `wmill.yaml` and the target is the remote workspace configuration. ## Pull API The `wmill gitsync-settings pull` command is used to pull remote git-sync settings and apply them to your local `wmill.yaml` file. It synchronizes your local configuration with the workspace git-sync settings. ```bash wmill gitsync-settings pull [options] ``` ### Options | Option | Parameter | Description | | ----------------------------- | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `-h, --help` | None | Show help options. | | `--repository` | `` | Specify repository path (e.g., u/user/repo). If not specified, will auto-select if only one repository exists or prompt for selection. | | `--default` | None | Write settings to top-level defaults instead of branch-specific overrides. | | `--replace` | None | Replace existing settings (non-interactive mode). Overwrites top-level wmill.yaml settings. | | `--override` | None | Add workspace-specific override (non-interactive mode). Creates or updates an entry under the `workspaces:` section. | | `--diff` | None | Show differences without applying changes. Preview what would be modified in your local configuration. | | `--json-output` | None | Output in JSON format. Useful for scripting and automation. | | `--with-backend-settings` | `` | Use provided JSON settings instead of querying backend (primarily for testing purposes). | | `--yes` | None | Skip interactive prompts and use default behavior. The command proceeds automatically without user intervention. | | `--promotion` | `` | Use promotionOverrides from the specified branch instead of regular overrides. | | `--workspace` | `` | Specify the target workspace. This overrides the default workspace. | | `--debug`, `--verbose` | None | Show debug/verbose logs. | | `--token` | `` | Specify an API token. This will override any stored token. | | `--base-url` | `` | Specify the base URL of the API. If used, `--token` and `--workspace` are required and no local remote/workspace will be used. | ## Push API The `wmill gitsync-settings push` command is used to push local git-sync settings and apply them to the remote workspace configuration. It synchronizes the workspace git-sync settings with your local `wmill.yaml`. ```bash wmill gitsync-settings push [options] ``` ### Options | Option | Parameter | Description | | ----------------------------- | --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `-h, --help` | None | Show help options. | | `--repository` | `` | Specify repository path (e.g., u/user/repo). If not specified, will auto-select if only one repository exists or prompt for selection. | | `--diff` | None | Show what would be pushed without applying changes. Preview the modifications that would be made to the workspace configuration. | | `--json-output` | None | Output in JSON format. Useful for scripting and automation. | | `--with-backend-settings` | `` | Use provided JSON settings instead of querying backend (primarily for testing purposes). | | `--yes` | None | Skip interactive prompts and use default behavior. The command proceeds automatically without user intervention. | | `--promotion` | `` | Use promotionOverrides from the specified branch instead of regular overrides. | | `--workspace` | `` | Specify the target workspace. This overrides the default workspace. | | `--debug`, `--verbose` | None | Show debug/verbose logs. | | `--token` | `` | Specify an API token. This will override any stored token. | | `--base-url` | `` | Specify the base URL of the API. If used, `--token` and `--workspace` are required and no local remote/workspace will be used. | ## Configuration management The `gitsync-settings` command manages git-sync filter settings and configuration options defined in your `wmill.yaml` file and the workspace backend. ### Settings managed The command handles these git-sync configuration fields (see [wmill.yaml](./sync.mdx#wmillyaml) for complete details): - **Path filters**: `includes`, `excludes`, `extraIncludes` - control which files are synced based on path patterns - **Type filters**: Control which types of resources are synced: - `skipScripts`, `skipFlows`, `skipApps`, `skipFolders` - resource type filters - `skipVariables`, `skipResources`, `skipResourceTypes`, `skipSecrets` - additional type filters - `includeSchedules`, `includeTriggers`, `includeUsers`, `includeGroups` - optional inclusions - `includeSettings`, `includeKey` - system resource inclusions ### Configuration modes The pull command supports different write modes when conflicts exist: - **Replace mode** (`--replace`): Overwrites top-level wmill.yaml settings - **Override mode** (`--override`): Creates workspace-specific overrides in the `workspaces:` section - **Default mode** (`--default`): Writes to top-level defaults - **Interactive mode** (default): Prompts user to choose between replace/override when conflicts exist ## Usage examples ### Basic operations ```bash # Pull git-sync settings from workspace to local wmill.yaml wmill gitsync-settings pull # Push local wmill.yaml git-sync settings to workspace wmill gitsync-settings push # Work with specific repository wmill gitsync-settings pull --repository u/user/my-repo wmill gitsync-settings push --repository u/user/my-repo ``` ### Preview changes ```bash # Show what would change in local configuration wmill gitsync-settings pull --diff # Show what would be pushed to workspace wmill gitsync-settings push --diff # JSON output for scripting wmill gitsync-settings pull --diff --json-output ``` ### Configuration modes ```bash # Replace top-level settings (non-interactive) wmill gitsync-settings pull --replace # Add workspace-specific override (non-interactive) wmill gitsync-settings pull --override # Write to defaults section wmill gitsync-settings pull --default # Skip prompts, use default behavior wmill gitsync-settings pull --yes ``` ### Branch-specific operations ```bash # Use promotion-specific overrides from main branch wmill gitsync-settings pull --promotion main # Push promotion settings to workspace wmill gitsync-settings push --promotion production ``` When using the `--promotion` flag, the command will use `promotionOverrides` instead of regular `overrides` from the specified branch in your `wmill.yaml`. This should be used for [Git Promotion](../9_deploy_gh_gl/index.mdx) connections. ## Differences from sync command The `gitsync-settings` command is specifically for managing configuration, not workspace content: | Feature | `gitsync-settings` | `sync` | |---------|-------------------|---------| | **Purpose** | Manages git-sync configuration settings | Synchronizes actual workspace content (scripts, flows, etc.) | | **Scope** | Configuration metadata only | Workspace resources and files | | **Target** | `wmill.yaml` git-sync settings | Workspace scripts, flows, apps, resources | | **File Operations** | Modifies `wmill.yaml` structure | Creates/updates resource files in sync directory | | **Safety** | Configuration changes only | Can create/delete workspace resources | For more details on the `wmill.yaml` configuration structure, see [wmill.yaml](./sync.mdx#wmillyaml). --- ## Installation Source: https://www.windmill.dev/docs/advanced/cli/installation # Installation To install the wmill CLI: ```bash npm install -g windmill-cli ``` Node version must greater than v20. Also, to punch through some networking layers like Cloudflare Tunnel, you might need some custom headers. You just need to use the HEADERS env variable: ``` export HEADERS=header_key:header_value,header_key2:header_value2 ``` Verify that the installation was successful by running the following command: ```bash wmill --version ``` If the installation was successful, you should see the version of wmill that you just installed. ## Upgrade wmill To upgrade your wmill installation to the latest version, run the following command: ```bash wmill upgrade ``` ## Completion The CLI comes with built-in completions for various shells. Use the following instructions to enable completions for your preferred shell. ### Bash To enable bash completions, add the following line to your ~/.bashrc: ```bash source <(wmill completions bash) ``` ### Zsh To enable zsh completions, add the following line to your ~/.zshrc: ```bash source <(wmill completions zsh) ``` ### Fish To enable fish completions, add the following line to your ~/.config/fish/config.fish: ```bash source (wmill completions fish | psub) ``` --- ## Job Source: https://www.windmill.dev/docs/advanced/cli/job # Jobs The `wmill job` commands let you list, inspect, and manage jobs from the CLI. For flow jobs, `job get` shows a hierarchical step tree and `job logs` aggregates logs from all steps. ## Listing jobs List recent jobs in the workspace. ```bash wmill job list [options] ``` ### Options | Option | Parameters | Description | | ------------------- | ------------- | --------------------------------------------------------------------------- | | `--json` | | Output as JSON (for piping to jq). | | `--script-path` | `path` | Filter by exact script or flow path. | | `--created-by` | `username` | Filter by creator username. | | `--running` | | Show only running jobs. | | `--failed` | | Show only failed jobs. | | `--limit` | `number` | Number of jobs to return (default 30, max 100). | | `--all` | | Include sub-jobs (flow steps). By default only top-level jobs are shown. | | `--parent` | `id` | Show only sub-jobs of a specific flow job. | | `--is-flow-step` | | Show only flow step jobs. | ### Examples 1. List recent failed jobs: ```bash wmill job list --failed ``` 2. List jobs for a specific flow: ```bash wmill job list --script-path f/production/etl_pipeline ``` 3. List sub-jobs of a flow run: ```bash wmill job list --parent 019d447b-114f-a018-0b72-9e541fb77c02 ``` ## Getting job details Get details about a specific job. For flow jobs, this displays a hierarchical step tree showing each module's status, label, duration, and sub-job ID. ```bash wmill job get [options] ``` ### Options | Option | Description | | -------- | -------------------------------- | | `--json` | Output as JSON (for piping to jq). | ### Example ```bash wmill job get 019d447b-114f-a018-0b72-9e541fb77c02 ``` For a flow job, the output includes a step tree: ``` ID: 019d447b-114f-a018-0b72-9e541fb77c02 Type: flow Status: success ... Steps: ✓ a: Generate list (019d447b-2a3f-...) 1.2s ✓ b: Process items (019d447b-3b4c-...) 3.4s ✓ iteration 0 (019d447b-4c5d-...) 1.1s ✓ iteration 1 (019d447b-5d6e-...) 1.2s ✓ iteration 2 (019d447b-6e7f-...) 1.1s ✓ c: Aggregate (019d447b-7f80-...) 0.8s ``` ## Getting job results Get the result of a completed job as JSON. Useful for scripting and piping to other commands. ```bash wmill job result ``` ## Getting job logs Get logs for a job. For flow jobs, this aggregates logs from all steps with labeled headers. For-loop iterations are shown individually. ```bash wmill job logs ``` ### Example For a flow job: ```bash wmill job logs 019d447b-114f-a018-0b72-9e541fb77c02 ``` ``` ====== a: Generate list ====== generating 3 items... ====== b: Process items (iteration 0) ====== processing item 1... ====== b: Process items (iteration 1) ====== processing item 2... ====== c: Aggregate ====== aggregating results... ``` For a specific step, use the sub-job ID from `job get`: ```bash wmill job logs 019d447b-2a3f-... ``` ## Cancelling a job Cancel a running or queued job. ```bash wmill job cancel [options] ``` ### Options | Option | Parameters | Description | | ---------- | ---------- | ------------------------ | | `--reason` | `reason` | Reason for cancellation. | ## Flow debugging workflow A typical workflow for debugging a failed flow run: ```bash # 1. Find the flow job wmill job list --script-path f/production/etl_pipeline --failed # 2. Inspect the step tree to see which step failed wmill job get # 3. See all step logs at once wmill job logs # 4. Or dive into a specific step's logs wmill job logs ``` --- ## Lint Source: https://www.windmill.dev/docs/advanced/cli/lint # Lint The `wmill lint` command validates Windmill YAML files (flows, schedules, and triggers) against their schemas. It scans a directory tree, infers each file's type from its filename (`.flow.yaml`, `.schedule.yaml`, `.http_trigger.yaml`, etc.), and reports missing required fields, unknown properties, or invalid values. ## Usage ```bash wmill lint [directory] [options] ``` If no directory is provided, the current directory is used. The command respects the `includes`/`excludes` patterns from your `wmill.yaml`. ## Options | Option | Description | | ------------------- | ------------------------------------------------------------------------------------------------ | | `--json` | Output machine-readable JSON instead of formatted text. | | `--fail-on-warn` | Exit non-zero on warnings (e.g. skipped native triggers). | | `--locks-required` | Fail if any script or inline script that needs a lock is missing one. | Native triggers (triggers whose schema depends on a dynamic `service_config`) are skipped with a warning — no static schema is available for them. ## Examples Lint every YAML file under the current directory: ```bash wmill lint ``` Lint a specific folder and emit JSON for CI tooling: ```bash wmill lint f/my_project --json ``` Fail the build if any script/flow/app inline script is missing its lockfile: ```bash wmill lint --locks-required ``` ## `--locks-required` `--locks-required` checks standalone scripts, flow inline scripts, normal app inline scripts, and raw app backend scripts. It applies to languages that need a lock: `bun`, `python3`, `php`, `go`, `deno`, `rust`, and `ansible`. The flag can also be enabled globally in `wmill.yaml`: ```yaml locksRequired: true ``` When set, `wmill sync push` runs the same verification before pushing, failing fast if anything is missing. ## CI integration A typical GitHub Actions step: ```yaml - name: Validate Windmill YAML files run: | npm install -g windmill-cli wmill lint --fail-on-warn --locks-required ``` Pair `wmill lint` with [`wmill generate-metadata`](./generate-metadata.md) to (re)generate any missing lockfiles before linting. --- ## Pipeline Source: https://www.windmill.dev/docs/advanced/cli/pipeline # Pipelines The `wmill pipeline` commands inspect and run [pipelines](../../core_concepts/63_pipelines/index.mdx): folders of scripts marked `// pipeline`, wired together by `// on ` annotations. All commands work against the deployed workspace by default; `show`, `run` and `docs` also accept `--local` to work from your working-tree files instead, and `wmill pipeline dev` live-previews local files in the browser (see [local development with --local](#local-development-with---local)). :::caution Alpha Pipelines are in alpha; the annotation syntax and behavior are still evolving. See the [Pipelines](../../core_concepts/63_pipelines/index.mdx) page for the full annotation reference. ::: ## Listing pipelines ```bash wmill pipeline list [--json] ``` Lists the pipeline folders in the workspace. `--json` outputs the list as JSON for piping to `jq`. ## Showing the pipeline graph ```bash wmill pipeline show [--json] [--local] ``` Renders the folder's pipeline DAG in the terminal: sources, asset lineage and subscriptions. | Option | Description | | --- | --- | | `--json` | Output the raw asset graph as JSON. | | `--local` | Build the graph from local working-tree files instead of the deployed workspace. Fully offline, no deploy needed. | ## Running a pipeline ```bash wmill pipeline run [options] ``` Runs a cascade: starting from `--from`, every downstream step runs in topological order, stopping at the `--to` end node(s) if given. `--from` may be any runnable node - a schedule/manual root or a mid-DAG model (asset subscriber or pure reader). A mid-DAG start runs that node plus its transitive downstream and never re-runs upstream, matching dbt's `--select model+`. This is the CLI counterpart of the graph's Run + downstream and [selective execution](../../core_concepts/63_pipelines/index.mdx#selective-execution-run-up-to-here). | Option | Description | | --- | --- | | `--from {#if loading}
Loading...
{:else} {#each users as user} {/each}
NameEmail
{user.name}{user.email}
{/if} ``` ![Full-code apps](../full_code_apps.png 'Full-code apps') ## Type safety During development, `wmill app dev` generates a `wmill.d.ts` file with typed signatures for each backend runnable. For example, given `backend/get_users.ts`: ```typescript export async function main(limit: number = 10): Promise { ... } ``` The generated types will be: ```typescript }; ``` This gives you autocomplete and type checking when calling runnables from your frontend. --- ## Project structure Source: https://www.windmill.dev/docs/full_code_apps/project_structure # Project structure A full-code app lives in a directory with the `.raw_app` suffix (e.g. `my_app.raw_app/`). If your filesystem or tooling hides dotfiles, you can use the `__raw_app` suffix instead (e.g. `my_app__raw_app/`) by setting `nonDottedPaths: true` in `wmill.yaml`. All examples below use the default `.raw_app` suffix. ## Directory layout ``` f/folder/my_app.raw_app/ ├── raw_app.yaml # App configuration (required) ├── package.json # Frontend dependencies ├── index.tsx # Frontend entry point ├── App.tsx # Main component (React example) ├── index.css # Styles ├── wmill.d.ts # Auto-generated TypeScript definitions ├── AGENTS.md # Auto-generated AI agent instructions ├── DATATABLES.md # Auto-generated data table documentation ├── backend/ # Backend runnables │ ├── get_users.ts # Runnable code │ ├── get_users.yaml # Runnable config (optional) │ ├── get_users.lock # Generated lock file │ └── query.pg.sql # SQL runnable (language inferred from extension) └── sql_to_apply/ # SQL migrations (dev only, not deployed) └── 001_create_table.sql ``` ## raw_app.yaml The `raw_app.yaml` file is the main configuration file for a full-code app. It defines metadata, execution policy and data table access. ```yaml summary: "My dashboard app" # Optional: custom URL path (admin only) custom_path: "my-dashboard" # Optional: make the app publicly accessible public: false # Optional: data table access configuration data: datatable: "main" # Datatable name tables: # Tables to whitelist - "users" - "orders" schema: "app1" # Schema within the datatable ``` The execution policy (including `execution_mode`, `triggerables` and `triggerables_v2`) is auto-generated at deployment time and does not need to be specified in `raw_app.yaml`. ### Fields | Field | Type | Required | Description | |-------|------|----------|-------------| | `summary` | string | Yes | Short description of the app (max 1000 chars) | | `custom_path` | string | No | Custom URL path (admin only) | | `public` | boolean | No | Whether the app is publicly accessible | | `data.datatable` | string | No | [Datatable](../../core_concepts/11_persistent_storage/data_tables.mdx) name | | `data.tables` | string[] | No | List of table names to whitelist | | `data.schema` | string | No | Schema name within the datatable | ## Auto-generated files These files are [generated by the CLI](../5_cli_workflow/index.mdx) and should not be edited manually: - **`wmill.d.ts`**: TypeScript definitions for the `wmill.ts` module, with typed signatures for each backend runnable. Regenerated when runnables change during `wmill app dev`. - **`AGENTS.md`**: Instructions for AI coding agents (Claude, Copilot, etc.) describing the project structure and available APIs. Generated by `wmill app generate-agents`. - **`DATATABLES.md`**: Documentation of available data tables and their schemas. Generated by `wmill app generate-agents`. ## Files excluded from deployment The following are not included when pushing to Windmill: - `node_modules/` - `dist/` - `.claude/`, `.git/` - `wmill.d.ts`, `package-lock.json` - `raw_app.yaml` (metadata is handled separately) - `sql_to_apply/` (dev-only migrations) - `DATATABLES.md`, `AGENTS.md` - `backend/` (runnables are extracted and uploaded separately) ## App path format App paths follow the standard Windmill path convention: ``` // ``` For example: `f/dashboard/my_app`, `u/admin/my_tool`, `g/engineering/ops_panel`. --- ## Ui editor Source: https://www.windmill.dev/docs/full_code_apps/ui_editor # UI editor Full-code apps can be created and edited directly in the Windmill web interface, without using the [Windmill CLI](../5_cli_workflow/index.mdx). ![Full-code apps](../full_code_apps.png 'Full-code apps') ## Overview The in-browser editor provides: - **File tree**: navigate and manage frontend files - **Code editor**: edit frontend and backend code with syntax highlighting - **Runnables panel**: create, edit and configure backend runnables from a sidebar - **Data table configuration**: whitelist data tables from the UI - **Preview**: test your app without deploying ## Creating a full-code app From your Windmill home page: 1. Click **New** and select **App (full-code)** to start from scratch, then choose your framework (React or Svelte). 2. To start from an existing project instead, hover **App (full-code)** in the **New** popover and click **Import full-code app**, then paste your YAML or JSON export in the drawer. ## Editing frontend files The file tree on the left shows all frontend files. Click a file to open it in the code editor. You can: - Create new files and folders - Rename and delete files - Edit code with syntax highlighting and autocompletion ## Managing backend runnables The runnables panel on the right lists all backend runnables. From here you can: - Add new inline scripts in any supported language - Reference existing workspace scripts or flows - Configure static input fields - View and edit runnable code ## Data table configuration Configure which [data tables](../4_data_tables/index.mdx) your app can access: 1. Open the data configuration panel 2. Select a datatable from your workspace 3. Whitelist specific tables 4. Optionally set a schema name ## Preview and testing Click the preview button to test your app in the browser. The preview runs your bundled frontend code and connects to Windmill's backend for runnable execution. ## When to use the UI editor vs CLI | Scenario | Recommended | |----------|-------------| | Quick edits to an existing app | UI editor | | New app from scratch | CLI (`wmill app new`) | | Complex frontend with many dependencies | CLI (`wmill app dev`) | | Team collaboration with git | CLI + [git sync](../../advanced/11_git_sync/index.mdx) | --- ## Apps quickstart Source: https://www.windmill.dev/docs/getting_started/apps_quickstart # Apps quickstart (low-code, legacy) :::info Legacy The low-code app editor is legacy. For new apps, we recommend [full-code apps](../../full_code_apps/index.mdx) with React or Svelte. See the [full-code apps quickstart](../9_full_code_apps_quickstart/index.mdx) to get started. ::: Welcome to the low-code apps quickstart! This page will provide you with the necessary knowledge to build your first applications in a matter of minutes using the drag-and-drop app editor. If you're more into videos, you can check out our tutorial on the App editor: Although Windmill provides [auto-generated UIs to scripts and flows](../../core_concepts/6_auto_generated_uis/index.mdx), you can build your own internal applications designed to your needs. Either with the [low-code app editor](../../apps/0_app_editor/index.mdx), or by building [full-code apps](../../full_code_apps/index.mdx) with React or Svelte. Windmill applications are customized UIs to interact with datasources (web, internal, data providers, etc). They are a great way to have non-technical users interact with custom-made workflows. In short, what you need to remember about apps: - They work on a what-you-see-is-what-you-get basis. - You can connect apps and components to [datasources](../../integrations/0_integrations_on_windmill.mdx). - Components can be empowered by Windmill [scripts](../../getting_started/0_scripts_quickstart/index.mdx) and [flows](../6_flows_quickstart/index.mdx). :::tip Follow our [detailed section](../../apps/0_app_editor/index.mdx) on the App editor for more information. ::: To create your first app, you could pick one from our [Hub](https://hub.windmill.dev/apps) and fork it. Here, we're going to build our own app from scratch, step by step. From the Windmill home page, click **New** and select **App (low-code)**, and let's get started! ### Toolbar Before jumping in, you should decide if you want to build a Desktop or a Mobile app. Just click on the appropriated icon on top. Switch from Editor to Preview mode to take a step back on what you're building. ![Toolbar](./toolbar.png.webp) ### Components The App editor works on a drag-and-drop basis. From the right-side menu, click on a component to create it on the [canvas](../../apps/1_canvas.mdx), move it maintaining a click on it and delete it clicking on `Delete component` at the bottom of the `Settings` tab. You can resize your components by dragging the resize handler on the bottom-right corner. If you have numerous components you might need to Anchor some of them to have them unmovable by other components. Check out all of our components at [this page](../../apps/4_app_configuration_settings/1_app_component_library.mdx) or (better) directly from the app editor. Know that we can quickly add components to the library, just [reach out to us](../../misc/6_getting_help/index.mdx). ### Empower components with scripts The beauty of Windmill App editor is the integrations of scripts & workflows to components. On the bottom of the app editor, you can find the [Runnable editor](../../apps/3_app-runnable-panel.mdx). It allows you to create, edit or manage the scripts or flows linked to components (Runnables), and [background runnables](../../apps/3_app-runnable-panel.mdx#background-runnables). From a component, click on `Create an inline script` or `Select a script or flow` (from workspace or hub) and you're good! ![App Runnables panel](../../assets/apps/0_app_editor/app-sections.png) The [Outputs](../../apps/2_outputs.mdx) of each component can be found on the left side menu. Each of them is associated with a component (see the component id). ![App Outputs](../../assets/apps/0_app_editor/app-outputs.png.webp) [Inputs can be connected to any output](../../apps/2_connecting_components/index.mdx): on a component click on `Connect` and associate inputs to outputs. At any time, have a look at your app's Inputs associated with each component on the dedicated `App inputs` tab found in the `⋮` menu of the toolbar. ![Apps inputs](./apps_inputs.png.webp) ### Tailor the look of your app At the [component level](../../apps/4_app_configuration_settings/4_app_styling.mdx#component-level), on the `Settings` tab, configure the style of each component (color, size, label, etc.) either directly in the boxes or using Custom CSS (on the Component Settings - Styling tab). ![Customize components](./customize_component.png.webp) At the [app level](../../apps/4_app_configuration_settings/4_app_styling.mdx#global-styling), you can give a harmonized style to your app with CSS. On the `Global Styling` tab, give details in boxes or in JSON, for the whole app and per class of component. ![App styling](./customize_app.png.webp) ### Time to test While building your app, you can try each component clicking on refresh button. You can do a refresh all for the whole app with the refresh button on top of the canvas, or schedule automatic refresh. ![Refresh app](./refresh_app.png.webp) Most convenient solution to test your app might just be to [preview](../../apps/0_toolbar.mdx#preview-mode) it: ![App previewed](./app_previewed.png 'App previewed') For possible bugs, there is a `Debug runs` tab to review past runs with details for each component. ![Debug runs](./debug_runs.png.webp) ### Then what? Your app is automatically saved as a [draft](../../core_concepts/0_draft_and_deploy/index.mdx#draft) as you edit. When you're ready, [Deploy](../../core_concepts/0_draft_and_deploy/index.mdx#deployed-version) the current version to make it available to users with a proper and explicit name. All additional changes can be seen and reversed in the [Diff Viewer](../../apps/0_toolbar.mdx#diff). Once it's saved, it's ready to use! You can also `Publish` it, or even `Publish to Hub` and export it in JSON or Hub compatible JSON from the `⋮` tab. Follow our [detailed section](../../apps/0_app_editor/index.mdx) on the App editor for more information. --- ## Flows quickstart Source: https://www.windmill.dev/docs/getting_started/flows_quickstart # Flows quickstart The present document will introduce you to [Flows](../../flows/1_flow_editor.mdx) and how to build your first one. > [Here](https://hub.windmill.dev/flows/43/) is an example of a simple flow built with Windmill. Have in mind that in Windmill, Scripts are at the basis of Flows and Apps. To sum up roughly, workflows are state machines [represented as DAGs](../../flows/16_architecture.mdx) (Directed Acyclic Graphs) to compose scripts together. To learn more about scripts, check the [Script quickstart](../0_scripts_quickstart/index.mdx). You will not necessarily have to re-build each script as you can reuse them from your workspace or from the [Hub](https://hub.windmill.dev/). Those workflows can run for-loops, branches (parallelizable), suspend themselves until a timeout or receiving events such as webhooks or approvals. They can be scheduled very frequently and check for new external items to process (what we call "Trigger" script). The result of a flow is the result of the last step executed, unless [error](../../flows/8_error_handling.mdx) was returned before or [Early return](../../flows/19_early_return.mdx) is set. The overhead and coldstart between each step is about 20ms, which is [faster than any other orchestration engine](/blog/launch-week-1/fastest-workflow-engine), by a large margin. To create your first workflow, you could also pick one from our [Hub](https://hub.windmill.dev/flows) and fork it. Here, we're going to build our own flow from scratch, step by step. From the [Windmill](../00_how_to_use_windmill/index.mdx) home page, click **New** and select **Flow**, and let's get started! :::tip Follow our [detailed section](../../flows/1_flow_editor.mdx) on the Flow editor for more information. ::: ## Settings ### Metadata The first thing you'll see is the [Settings](../../flows/3_editor_components.mdx#settings) menu. From there, you can set the [permissions](../../core_concepts/16_roles_and_permissions/index.mdx) of the workflow: User (by default, you), and [Folder](../../core_concepts/8_groups_and_folders/index.mdx) (referring to read and/or write groups). Also, you can give succinctly a Name, a Summary and a Description to your flow. Those are supposed to be explicit, we recommend you to give context and make them as self-explanatory as possible. ![Flows metadata](./flows_metadata.png.webp) ### Schedule On another tab, you can configure a [Schedule](../../core_concepts/1_scheduling/index.mdx) to trigger your flow. Flows can be [triggered](../../triggers/index.mdx) by any schedules, their [webhooks](../../core_concepts/4_webhooks/index.mdx) or their UI but they only have only one primary schedule with which they share the same path. This menu is where you set the primary schedule with CRON. The default schedule is none. ![Flows schedule](./flows_schedule.png.webp) ### Shared directory Last tab of the settings menu is the [Shared Directory](../../core_concepts/11_persistent_storage/states_resources_shared_directory.mdx#shared-directory). By default, flows on Windmill are based on a [result basis](#how-data-is-exchanged-between-steps). A step will take as inputs the results of previous steps. And this works fine for lightweight automation. For heavier ETLs and any output that is not suitable for JSON, you might want to use the `Shared Directory` to share data between steps. Steps share a folder at `./shared` in which they can store heavier data and pass them to the next step. Get more details on the [Persistent storage & databases dedicated page](../../core_concepts/11_persistent_storage/index.mdx). ![Flows shared directory](./flows_shared_directory.png.webp) ### Worker group When a [worker group](../../core_concepts/9_worker_groups/index.mdx) is defined at the flow level, any steps inside the flow will run on that worker group, regardless of the steps' worker group. If no worker group is defined, the flow controls will be executed by the default worker group 'flow' and the steps will be executed in their respective worker group. You can always go back to this menu by clicking on `Settings` on the top lef, or on the name of the flow on the [toolbar](../../flows/3_editor_components.mdx#toolbar). ## How data is exchanged between steps Flows on Windmill are generic and reusable, they therefore expose inputs. Input and outputs are piped together. Inputs are either: - Static: fixed values set directly in the step input fields (strings, numbers, JSON, etc.). These are constants that do not change between executions. - [Flow env variables](../../flows/3_editor_components.mdx#flow-env-variables): flow-level constants accessible from any step using `flow_env.VARIABLE_NAME`. They support strings, JSON and [resources](../../core_concepts/3_resources_and_types/index.mdx). - [Dynamically linked to others](../../flows/16_architecture.mdx): with [JSON objects](../../core_concepts/13_json_schema_and_parsing/index.mdx) as result that allow to refer to the output of any step. You can refer to the result of any step: - using the id associated with the step - clicking on the plug logo that will let you pick flow inputs or previous steps' results (after testing flow or step). ## Flow editor On the left of the editor, you'll find a graphical view of the flow. From there you can architecture your flow and take action at each step. ![Flow editor menu](./flow_editor_menu.png.webp) :::tip Pro tips Keep your flows organized and documented with [sticky notes](../../flows/24_sticky_notes.mdx) for free-floating comments and TODOs, and with [flow groups](../../flows/1_flow_editor.mdx#flow-groups) to visually cluster related steps and document complex workflow sections. ::: There are five kinds of scripts: [Action](../../flows/3_editor_components.mdx#flow-actions), [Trigger](../../flows/10_flow_trigger.mdx), [Approval](../../flows/11_flow_approval.mdx), [Error handler](../../flows/7_flow_error_handler.md) and [Preprocessor](../../core_concepts/43_preprocessors/index.mdx). You can sequence them how you want. Action is the default script type. Each script can be called from Workspace or [Hub](https://hub.windmill.dev/), you can also decide to write them inline. ![Import or write scripts](./import_or_write_scripts.png.webp) Your flow can be deepened with [additional features](../../flows/1_flow_editor.mdx), below are some major ones. ### For loops [For loops](../../flows/12_flow_loops.md) are a special type of steps that allows you to iterate over a list of items, given by an iterator expression. ![Flows For loops](./for_loops.png.webp) ### While loops While loops execute a sequence of code indefinitely until the user cancels or a step set to [Early stop](../../flows/2_early_stop.md) stops. ### Branching [Branches](../../flows/13_flow_branches.md) build branching logic to create and manage complex workflows based on conditions. There are two of them: - [Branch one](../../flows/13_flow_branches.md#branch-one): allows you to execute a branch if a condition is true. - [Branch all](../../flows/13_flow_branches.md#branch-all): allows you to execute all the branches in parallel, as if each branch is a flow. ![Flow branching](flow_branches.png.webp) ### Retries At each step, Windmill allows you to [customize the number of retries](../../flows/14_retries.md) by going on the `Advanced` tabs of the individual script. If defined, upon error this step will be retried with a delay and a maximum number of attempts. ![Flows retries](./flows_retries.png.webp) ### Suspend/Approval Step At each step you can add [Approval scripts](../../flows/11_flow_approval.mdx) to manage security and control over your flows. Request approvals can be sent by email, Slack, anything. Then you can automatically resume workflows with secret webhooks after the approval steps. ![Approval step diagram](../../assets/flows/approval_diagram.png 'Approval step diagram') You can find all the flows' features in their [dedicated section](../../flows/1_flow_editor.mdx). ## Triggers There are several ways to trigger a flow with Windmill. 1. The most direct one is from the [autogenerated UI provided by Windmill](../../core_concepts/6_auto_generated_uis/index.mdx). It is the one you will see from the flow editor. 2. A similar but more customized way is to use Windmill Apps using the [App editor](../7_apps_quickstart/index.mdx). 3. We saw above that you can trigger flows using [schedules](../../core_concepts/1_scheduling/index.mdx) that you can check from the [Runs](../../core_concepts/5_monitor_past_and_future_runs/index.mdx) page. One special way to use scheduling is to combine it with [trigger scripts](../../flows/10_flow_trigger.mdx). 4. [Execute flows from the CLI](../../advanced/3_cli/index.mdx) to trigger your flows from your terminal. 5. [Trigger the flow from another flow](../../triggers/index.mdx#trigger-from-flows). 6. Using [trigger scripts](../../flows/10_flow_trigger.mdx) to trigger only if a condition has been met. 7. [Webhooks](../../core_concepts/4_webhooks/index.mdx). Each Flow created in the app gets autogenerated webhooks. You can see them once you flow is saved. You can even [trigger flows without leaving Slack](/blog/handler-slack-commands)! You can test your triggers in test mode: ## Test your flow You don't have to explore all Flow editor possibilities at once. At each step, test what you're building to keep control on your wonder. You can also test up to a certain step by clicking on an action (x) and then on `Test up to x`. When you're done, [deploy](../../core_concepts/0_draft_and_deploy/index.mdx) your flow, schedule it, [create and app from it](../../core_concepts/6_auto_generated_uis/index.mdx), or even [publish it to Hub](../../misc/1_share_on_hub/index.md). Follow our [detailed section](../../flows/1_flow_editor.mdx) on the Flow editor for more information. ## Flow as Code Flows are not the only way to write distributed programs that execute distinct jobs. Another approach is to write a program that defines the jobs and their dependencies, and then execute that program within a [Python](../0_scripts_quickstart/2_python_quickstart/index.mdx) or [TypeScript](../0_scripts_quickstart/1_typescript_quickstart/index.mdx) script. This is known as workflows as code. ![Flow as code](../../core_concepts/31_workflows_as_code/wac-editor-1.png) --- ## Full code apps quickstart Source: https://www.windmill.dev/docs/getting_started/full_code_apps_quickstart # Full-code apps quickstart This guide walks you through building your first [full-code app](../../full_code_apps/index.mdx). We'll create a React app from the Windmill UI, explore the scaffolded code, add a second backend runnable, and wire both into a polished frontend. ![Full code apps](./full_code_app_demo.png 'Full code apps') Full-code apps give you complete control over your UI with React or Svelte, while Windmill handles backend execution, permissions and deployment. | | Full-code apps | Low-code apps | |---|---|---| | **UI** | Custom React/Svelte components | Drag-and-drop component library | | **Frontend logic** | Full framework features (hooks, stores, routing) | Connecting components + inline scripts | | **Backend** | Scripts in `backend/` folder, any language | Runnables panel, inline or workspace scripts | | **Local dev** | `wmill app dev` with hot reload | Web-based editor only | | **Best for** | Custom UIs, complex interactions, existing codebases | Quick dashboards, forms, CRUD interfaces | ## Step 1: Create the app ### From the platform From your [Windmill](../00_how_to_use_windmill/index.mdx) home page, click **New** and select **App (full-code)**. ![Pick full-code app](./pick_raw_app.png 'Pick full-code app') In the setup dialog: 1. Pick a framework (React or Svelte 5), for the example we'll use React 19 2. Choose a Data configuration. Here we'll use a new datatable. It's not required for apps to have a datatable, it will be used only in a [dedicated section](#step-5-use-a-data-table-for-persistence) of the quickstart. 3. Start the app 'without AI'. Or just enter a prompt and start with AI, and you're done for the quickstart :) ![Pick React framework](./pick_react.png 'Pick React framework') The [UI editor](../../full_code_apps/6_ui_editor/index.mdx) opens with a scaffolded project. ### From the CLI You can do the same from the [Windmill CLI](../../advanced/3_cli/index.mdx): ```bash wmill app new ``` The wizard prompts for the same choices. Then install dependencies and start the dev server: ```bash cd f/folder/my_app.raw_app npm install wmill app dev ``` This starts a local server with hot reload at `http://localhost:4000`. ## Step 2: Explore the scaffolded project The created app has this structure: ``` f/folder/my_app.raw_app/ ├── raw_app.yaml # App metadata and configuration ├── package.json # Frontend dependencies ├── index.tsx # Entry point (renders App) ├── App.tsx # Main React component ├── index.css # Styles └── backend/ ├── a.yaml # Sample backend runnable config └── a.ts # Sample backend runnable code ``` ![Default App.tsx](./default_tsx.png 'Default App.tsx') ### The default App.tsx The scaffolded `App.tsx` looks like this: ```tsx import './index.css' const App = () => { const [value, setValue] = useState(undefined as string | undefined) const [loading, setLoading] = useState(false) async function runA() { setLoading(true) try { setValue(await backend.a({ x: 42 })) } catch (e) { console.error(e) } setLoading(false) } return

hello world

{loading ? 'Loading ...' : value ?? 'Click button to see value here'}
} export default App ``` It imports `backend` from `./wmill` - this is an auto-generated module that provides typed functions to call your [backend runnables](../../full_code_apps/2_backend_runnables/index.mdx). Here, clicking the button calls `backend.a()` which runs the sample runnable `a` in the `backend/` folder. ### The default backend runnable The sample runnable `backend/a.ts` is a simple TypeScript function: ```typescript // import * as wmill from "windmill-client" export async function main(x: string) { return x } ``` You can preview your UI by selecting 'App.tsx' to see it in the right pane, or by clicking 'Preview' in the UI editor for a fullscreen view. If you're using the CLI, open `http://localhost:4000` in your browser to access the app. When you click the button, it sends a request to the backend and displays the returned result. ![Default backend runnable](./default_backend.png 'Default backend runnable') ### How it works The key concept: `backend.a({ x: 42 })` sends the call to a Windmill worker that executes `backend/a.ts` and returns the result. Your frontend never runs the backend code directly - it goes through Windmill's execution engine via WebSocket, which means you get logging, permissions and error handling for free. ## Step 3: Edit and add backend runnables The scaffolded app comes with one runnable (`a`). Let's update it and add a second one. ### From the UI editor Click on the `a` runnable in the runnables panel. Give it the summary "Multiply" and replace the code with: ```typescript // backend/a.ts return `Result: ${x} × 2 = ${x * 2}`; } ``` ![Multiply runnable](./multiply.png 'Multiply runnable') Now add a second runnable - this time in [Python](../../getting_started/0_scripts_quickstart/2_python_quickstart/index.mdx) as you can mix languages within the same app: 1. In the runnables panel on the right, click the `+` button 2. Select **Python** as the language 3. Name it `b` and give it the summary "Get timestamp" ![Choose language](./choose_language.png 'Choose language') Paste this code: ```python # backend/b.py from datetime import datetime def main(format: str): now = datetime.now() if format == "iso": return now.isoformat() elif format == "locale": return now.strftime("%c") else: return str(now) ``` ![Get timestamp runnable](./get_timestamp.png 'Get timestamp runnable') As you can see, the [auto-generated UI](../../core_concepts/6_auto_generated_uis/index.mdx) updated with the new input name (`format`). You now have two runnables in different languages: `a` (TypeScript) doubles a number, `b` (Python) returns a formatted date. The frontend calls them the exact same way - it doesn't need to know which language runs behind the scenes. ### From local files Alternatively, work directly in `backend/`: - Edit `backend/a.ts` with the multiply code above - Create `backend/b.py` with the Python code above The language is auto-detected from the file extension (`.ts` for TypeScript, `.py` for Python) and the runnable ID is derived from the filename (`a`, `b`). ## Step 4: Build the frontend Now let's update `App.tsx` to call both runnables. The auto-generated `wmill` module automatically picks up the new `b` runnable, so we can call `backend.a()` and `backend.b()` right away. Replace the content of `App.tsx` with: ```tsx import './index.css' const App = () => { const [valueA, setValueA] = useState(undefined) const [valueB, setValueB] = useState(undefined) const [loadingA, setLoadingA] = useState(false) const [loadingB, setLoadingB] = useState(false) const [inputNumber, setInputNumber] = useState(42) async function runA() { setLoadingA(true) try { setValueA(await backend.a({ x: inputNumber })) } catch (e) { console.error('Error running a:', e) } setLoadingA(false) } async function runB() { setLoadingB(true) try { setValueB(await backend.b({ format: 'locale' })) } catch (e) { console.error('Error running b:', e) } setLoadingB(false) } async function runBoth() { setLoadingA(true) setLoadingB(true) try { const [resultA, resultB] = await Promise.all([ backend.a({ x: inputNumber }), backend.b({ format: 'iso' }) ]) setValueA(resultA) setValueB(resultB) } catch (e) { console.error('Error running both:', e) } setLoadingA(false) setLoadingB(false) } return (

Full-code app demo

Calling 2 backend runnables

Multiply (TypeScript)

{loadingA ? 'Loading...' : valueA ?? 'Click a button to see result'}

Timestamp (Python)

{loadingB ? 'Loading...' : valueB ?? 'Click a button to see result'}
) } export default App ``` A few things to notice: - Each button calls a different backend runnable (`backend.a()` or `backend.b()`) - **Run both** uses `Promise.all` to call both runnables in parallel - each one runs as a separate Windmill job - The `format` parameter on `backend.b()` is passed as an argument, just like `x` on `backend.a()` - `a` runs TypeScript on a Bun worker, `b` runs Python - the frontend doesn't need to care ### Update the styles Replace `index.css` to give the app a cleaner look: ```css .container { max-width: 600px; margin: 0 auto; padding: 20px; font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; } h1 { margin-bottom: 5px; color: #333; } .subtitle { color: #666; margin-top: 0; margin-bottom: 24px; } .input-section { margin-bottom: 20px; } .input-section label { display: flex; align-items: center; gap: 10px; font-weight: 500; } .input-section input { padding: 8px 12px; border: 1px solid #ddd; border-radius: 6px; font-size: 16px; width: 100px; } .buttons { display: flex; gap: 10px; flex-wrap: wrap; margin-bottom: 24px; } button { padding: 10px 18px; border: 1px solid #ddd; border-radius: 6px; background: white; cursor: pointer; font-size: 14px; transition: all 0.2s; } button:hover:not(:disabled) { background: #f5f5f5; border-color: #ccc; } button:disabled { opacity: 0.6; cursor: not-allowed; } button.primary { background: #3b82f6; color: white; border-color: #3b82f6; } button.primary:hover:not(:disabled) { background: #2563eb; } .results { display: grid; gap: 16px; } .result-card { border: 1px solid #e5e7eb; border-radius: 8px; padding: 16px; background: #fafafa; } .result-card h3 { margin: 0 0 10px 0; font-size: 14px; color: #666; text-transform: uppercase; letter-spacing: 0.5px; } .result-value { font-size: 16px; color: #333; font-family: 'Monaco', 'Menlo', monospace; word-break: break-all; } ``` See the preview in the right part of the screen of App.tsx (or click 'Preview' in the UI editor to see it in full screen, or check `http://localhost:4000` if using the CLI) to see the result. Try clicking each button individually, then "Run both" to see parallel execution. ![Updated App.tsx](./updated_tsx.png 'Updated App.tsx') ![Index CSS](./index_css.png 'Index CSS') :::info Svelte If you chose Svelte 5 instead of React, the same pattern applies. Import `backend` from `./wmill` and use Svelte's `$state` and `$effect` for reactivity. See the [frontend reference](../../full_code_apps/3_frontend/index.mdx) for Svelte examples. ::: ### How backend calls work Every call to `backend.a()` or `backend.b()` is a real Windmill job execution: - `backend.xxx(args)` - calls a runnable and waits for the result (synchronous) - `backendAsync.xxx(args)` - starts a runnable and returns a job ID immediately (for long-running tasks) - `waitJob(jobId)` - waits for an async job to complete ## Step 5: Use a data table for persistence So far our runnables compute values on the fly. Full-code apps can also read and write to Windmill [data tables](../../core_concepts/11_persistent_storage/data_tables.mdx) - a built-in storage layer. ### Set up a database First, make sure a database is configured in your workspace. Go to **Workspace settings** > **Data Tables** and set up a database connection (or use the [Custom instance database](../../core_concepts/53_custom_instance_database/index.mdx) if available). ![Create table](./create_table.png 'Create table') ### Add a table to your app In the raw app editor, open the **Data** section on the left panel and click the `+` button. You can either pick an existing table (public or from other apps) or create a new one for your app. Let's create a new table: 1. Click `+` in the Data section 2. Select **Create new table** 3. Name it `computation_logs` 4. Define the columns: - `id` — `BIGSERIAL` (primary key, added by default) - `input` — `INT` - `result` — `TEXT` - `created_at` — `TIMESTAMP`, default `now()` 5. Click **Create table** ![Create table columns](./create_table_2.png 'Create table columns') The table is now whitelisted for your app. You can view its schema and data from the Data section. ### Add a SQL runnable Now create a backend runnable that queries this table. In the runnables panel, click `+` and select **PostgreSQL** as the language. Name it `get_logs`. For the database resource, pick the same resource as the one configured in your workspace Data Tables settings. ```sql -- backend/get_logs.pg.sql SELECT * FROM app_demo.computation_logs ORDER BY created_at DESC LIMIT 10; ``` ![Get logs runnable](./get_logs.png 'Get logs runnable') From the frontend, call it like any other runnable: ```typescript const logs = await backend.get_logs(); ``` The frontend code doesn't need to know whether a runnable is TypeScript, Python or SQL - the `wmill` module handles them all the same way. :::tip CLI From local files, the `.pg.sql` extension tells Windmill to run the script as a PostgreSQL query. Other SQL dialects are supported too (`.my.sql` for MySQL, `.bq.sql` for BigQuery, etc.). See the [backend runnables](../../full_code_apps/2_backend_runnables/index.mdx) reference for the full list. ::: ## Step 6: Deploy ### From the UI editor Click the **Deploy** button in the toolbar. Each deployment creates a new version of your app. ![Deploy](./deploy.png 'Deploy') ![Deployed app](./deployed_app.png 'Deployed app') ### From the CLI Generate lock files for your runnables and push: ```bash wmill generate-metadata wmill sync push ``` ### Make it public To make the app accessible without login, add `public: true` to `raw_app.yaml`: ```yaml summary: "Full-code app demo" public: true ``` Admins can also set a custom URL path: ```yaml custom_path: "my-demo" ``` The app is then accessible at `https:///apps/custom/my-demo`. ## Runnable configuration So far we've used code-only runnables (just a file in `backend/`). For more control, you can add a `.yaml` config file alongside the code to pre-fill inputs: ```yaml # backend/a.yaml type: inline fields: x: type: static value: 100 ``` This pre-fills the `x` parameter so the frontend doesn't need to pass it. You can also reference existing workspace [scripts](../../script_editor/index.mdx) or [flows](../../flows/1_flow_editor.mdx) instead of writing inline code: ```yaml # backend/send_notification.yaml type: script path: f/production/send_slack_notification ``` ## Next steps You now have a working full-code app with a custom React frontend calling multiple backend runnables. From here you can: - Add [backend runnables](../../full_code_apps/2_backend_runnables/index.mdx) in any language (Python, SQL, Go, etc.) - Style your app with CSS, Tailwind or any React library - Set up [CI/CD with git sync](../../advanced/11_git_sync/index.mdx) for team workflows - Use [Windmill AI](../../core_concepts/22_ai_generation/index.mdx) to generate apps from prompts --- ## How to use windmill Source: https://www.windmill.dev/docs/getting_started/how_to_use_windmill # Getting started with Windmill Windmill is a fast, **open-source** workflow engine and developer platform to build endpoints, workflows and UIs, coding in TypeScript, Python, Go, and [many other languages](../0_scripts_quickstart/index.mdx). For the full pitch and how Windmill compares to its alternatives, see [What is Windmill?](../../intro.mdx). ## Choose your setup ### Windmill cloud Quickly get started with our [Cloud App](https://app.windmill.dev/), no credit card required. Sign up using GitHub, GitLab, Google, or Microsoft SSO. Start with 1,000 monthly executions on our Community Plan, and easily upgrade for more. [Start with Windmill cloud](https://app.windmill.dev/). Windmill cloud is hosted in the US. We offer dedicated cloud instances in the EU for [Cloud Enterprise](/pricing) customers. ### Self-host Windmill For full control over your infrastructure, self-host Windmill using our [helm charts](https://github.com/windmill-labs/windmill-helm-charts) for Kubernetes or docker-compose for simpler setups. [Learn how to self-host Windmill](../../advanced/1_self_host/index.mdx). ## Development options ### Integrated Development Environment (IDE) Windmill supports development directly within its [built-in IDE](../../code_editor/index.mdx), tailored for creating scripts, workflows, [low-code apps](../../apps/0_app_editor/index.mdx) and [full-code apps](../../full_code_apps/index.mdx) efficiently. ### Local development Prefer your own setup? No problem. Windmill integrates smoothly with local environments, including a [VS Code extension](../../cli_local_dev/1_vscode-extension/index.mdx) and tools for [Git-based deployment to production](../../advanced/12_deploy_to_prod/index.mdx). [Explore local development options](../../advanced/4_local_development/index.mdx). ### Ready to deploy? Move from staging to production seamlessly with Windmill's deployment guides, ensuring your projects are production-ready. [Deploy to production](../../advanced/12_deploy_to_prod/index.mdx). ## Creating resources from the home page The workspace home page lists everything in your workspace. To create something new, use the **New** button in the top-right corner of the header. Hovering (or clicking) it opens a two-pane popover: the option list on the right, a description of the highlighted option on the left. Clicking **New** directly creates a script, the default option. The available options are: - [Script](../0_scripts_quickstart/index.mdx) - a single standalone script in [any supported language](../0_scripts_quickstart/index.mdx). - [Flow](../../flows/1_flow_editor.mdx) - compose scripts into a workflow with branches, loops, approvals and retries. - [App (full-code)](../9_full_code_apps_quickstart/index.mdx) - build a UI with React or Svelte. - [Workflow-as-Code](../../core_concepts/31_workflows_as_code/index.mdx) - badged _Advanced_. Express a whole workflow as a single script; picking it offers a **Python** or **TypeScript** choice. - [Data pipelines](../10_pipeline_quickstart/index.mdx) - badged _Alpha_. Visual editor to chain ingestion, transformation and materialization steps. - [App (low-code)](../7_apps_quickstart/index.mdx) - badged _Legacy_. Drag-and-drop UI builder. For new apps, prefer full-code apps. Some options also expose secondary import actions in their detail panel (for example _Import flow_, _Import Workflow-as-Code_, _Import full-code app_, _Import low-code app_), which open a drawer where you paste a YAML or JSON export to pre-fill the editor. The header also has a **Hub** button that opens the [Windmill Hub](https://hub.windmill.dev/) in a new tab, and a **CLI / MCP** button to connect external tooling. --- ## Pipeline quickstart Source: https://www.windmill.dev/docs/getting_started/pipeline_quickstart # Pipelines quickstart :::caution Alpha Pipelines are in alpha. The entry point is deliberately tucked away while we develop the feature, and the annotation syntax and behavior described in this guide are still evolving and may change in future releases. We would love your feedback - share it on [Discord](https://discord.com/invite/V7PM2YHsPB) or [GitHub](https://github.com/windmill-labs/windmill). ::: This guide builds your first [pipeline](../../core_concepts/63_pipelines/index.mdx): a set of scripts in a folder wired together automatically by the data they read and write, no manual orchestration. It uses the recommended [DuckDB](../../integrations/duckdb.md) + [DuckLake](../../core_concepts/11_persistent_storage/ducklake.mdx) path, where each step materializes a table (this is what makes the writes idempotent, versioned and testable). It takes about five minutes. DuckLake is not required, though. Pipelines wire scripts in any language (Python, TypeScript, ...) by the plain [assets](../../core_concepts/52_assets/index.mdx) they exchange - S3 objects, resources, data tables or volumes - with the same `-- on` model and no materialization. See [inputs and outputs](../../core_concepts/63_pipelines/index.mdx#inputs-and-outputs-assets) for that path; this guide shows DuckLake because it is the most powerful default. :::info Prerequisite Because this guide uses DuckLake, you need a [workspace storage and a DuckLake](../../core_concepts/11_persistent_storage/ducklake.mdx) configured. The default DuckLake is named `main`; this guide uses it. (A plain S3-based pipeline needs only the [workspace storage](../../core_concepts/38_object_storage_in_windmill/index.mdx#workspace-object-storage).) On a fresh workspace the pipelines page shows a **setup checklist** whenever either prerequisite is missing, with a link straight to the [workspace object storage](../../core_concepts/38_object_storage_in_windmill/index.mdx#workspace-object-storage) and [DuckLake](../../core_concepts/11_persistent_storage/ducklake.mdx) settings that fix it. Nothing materializes until both are set. ::: ## 1. Create a pipeline On the home page, click **New** and select **Data pipelines** (badged _Alpha_) in the popover. ![Create a pipeline from the New popover](./create_pipeline_menu.png 'Create a pipeline from the New popover') You can also open the pipelines index page at `/pipeline`: it lists existing pipelines with their script counts and lets you pick or create a folder. When you edit a SQL or DuckDB script that is not yet part of a pipeline, a dismissible hint below the toolbar links there too. ![The pipelines index page](./pipeline_index_page.png 'The pipelines index page') Pipelines live in a [folder](../../core_concepts/8_groups_and_folders/index.mdx) (for example `f/demo`); every script you add to it and mark with `-- pipeline` becomes part of the same pipeline graph. ## 2. Add a producer script Create a DuckDB script `f/demo/ingest`. The `-- pipeline` line places it in the folder's pipeline, and `-- materialize` tells Windmill to own the write: it creates the `ducklake://main/events` table from the trailing `SELECT` and records a snapshot and row count. ```sql -- pipeline -- materialize ducklake://main/events SELECT * FROM (VALUES (1, 'click', TIMESTAMP '2026-01-01 10:00'), (2, 'view', TIMESTAMP '2026-01-01 10:05'), (3, 'click', TIMESTAMP '2026-01-01 11:00') ) AS t(id, kind, ts); ``` ## 3. Add a consumer script Create a DuckDB script `f/demo/rollup`. The `-- on ducklake://main/events` annotation declares the table it reads, which becomes an incoming edge: this script runs automatically whenever `events` is materialized. It materializes its own aggregate table. ```sql -- pipeline -- on ducklake://main/events -- materialize ducklake://main/events_by_kind ATTACH 'ducklake://main' AS dl; SELECT kind, count(*) AS n FROM dl.events GROUP BY kind; ``` ## 4. Open the pipeline graph Open the folder and select the pipeline view. You will see the lineage: `ingest` → `ducklake://main/events` → `rollup` → `ducklake://main/events_by_kind` ## 5. Run it On the `ingest` node, choose "Run + downstream". `ingest` materializes `events`, and the asset cascade automatically fires `rollup`, which materializes `events_by_kind`. Each node shows live status, its DuckLake snapshot and row count as the run progresses, and the result panel previews the materialized table. That is the whole model: mark scripts with `-- pipeline`, declare inputs with `-- on`, and Windmill infers and runs the graph from asset lineage. Adding `-- materialize` (the optional DuckLake layer used here) is what also gives you idempotent re-runs, time-travel, [data tests](../../core_concepts/63_pipelines/materialization.mdx#data-tests) and [backfill](../../core_concepts/63_pipelines/materialization.mdx#partition-status-and-backfill) with no extra work. ## Next steps Add materialization strategies (merge, append, SCD2 history), partitions, schedules, AND/OR joins, data tests and debounce. Every annotation and option is documented, with examples, on the concept page. --- ## Scripts quickstart Source: https://www.windmill.dev/docs/getting_started/scripts_quickstart # Scripts quickstart Windmill supports scripts in TypeScript, Python, Go, PHP, Bash, C#, SQL and more. --- ## Ansible quickstart Source: https://www.windmill.dev/docs/getting_started/scripts_quickstart/ansible # Ansible quickstart In this quickstart guide, we will write our first script/playbook with [Ansible](https://www.ansible.com/). This tutorial covers how to create a simple Ansible script through Windmill web IDE. See the dedicated page to [develop scripts locally](../../../advanced/4_local_development/index.mdx). Scripts are the basic building blocks in Windmill. They can be [run and scheduled](../../../triggers/index.mdx) as standalone, chained together to create [Flows](../../../flows/1_flow_editor.mdx) or displayed with a personalized user interface as [Apps](../../7_apps_quickstart/index.mdx). Scripts consist of 2 parts: - [Code](#code-playbook): for Ansible this is a playbook file written in yaml. - [Settings](#settings): settings & metadata about the Script such as its path, summary, description, [JSON Schema](../../../core_concepts/13_json_schema_and_parsing/index.mdx) of its inputs (inferred from its signature). When stored in a code repository, these 2 parts are stored separately at `.playbook.yml` and `.script.yaml` ![Ansible in Windmill](./create_ansible_script.png) ## Settings ![New script](../../../../static/images/script_languages.png "New script") As part of the [settings](../../../script_editor/settings.mdx) menu, each script has metadata associated with it, enabling it to be defined and configured in depth. - **Summary** (optional) is a short, human-readable summary of the Script. It will be displayed as a title across Windmill. If omitted, the UI will use the `path` by default. - **Path** is the Script's unique identifier that consists of the [script's owner](../../../core_concepts/16_roles_and_permissions/index.mdx), and the script's name. The owner can be either a user, or a group ([folder](../../../core_concepts/8_groups_and_folders/index.mdx#folders)). - **Description** is where you can give instructions through the [auto-generated UI](../../../core_concepts/6_auto_generated_uis/index.mdx) to users on how to run your Script. It supports markdown. - **Language** of the script. - **Script kind**: Action (by default), [Trigger](../../../flows/10_flow_trigger.mdx), [Approval](../../../flows/11_flow_approval.mdx), [Error handler](../../../flows/7_flow_error_handler.md) or [Preprocessor](../../../core_concepts/43_preprocessors/index.mdx). This acts as a tag to filter appropriate scripts from the [flow editor](../../6_flows_quickstart/index.mdx). This menu also has additional settings on [Runtime](../../../script_editor/settings.mdx#runtime), [Generated UI](#generated-ui) and [Triggers](../../../script_editor/settings.mdx#triggers). Now click on the code editor on the left side. ## Code (Playbook) In order to make Ansible playbooks compatible with the Windmill environment and script model, there is some extra information preceding the start of the playbook that can be entered. Because of this, an Ansible playbook in Windmill will typically look like this: ![Ansible in Windmill](./ansible_script_ide.png) ```yml --- inventory: - resource_type: ansible_inventory # You can pin an inventory to this script: # resource: u/user/your_resource # Additional inventories available as script arguments additional_inventories: - "delegate_git_repository/hosts/inventory.ini" # File resources will be written in the relative \`target\` location before # running the playbook files: - resource: u/user/fabulous_jinja_template target: ./config_template.j2 # Define the arguments of the Windmill script extra_vars: world_qualifier: type: string dependencies: galaxy: collections: - name: community.general - name: community.vmware python: - jmespath --- - name: Echo hosts: 127.0.0.1 connection: local vars: my_result: a: 2 b: true c: "Hello" tasks: - name: Print debug message debug: msg: "Hello, {{world_qualifier}} world!" - name: Write variable my_result to result.json delegate_to: localhost copy: content: "{{ my_result | to_json }}" dest: result.json ``` There are two YAML documents in series, the second being the Ansible playbook. The first one is only used by Windmill, and will not be visible to Ansible when executing the playbook. It contains different sections that declare some metadata about the script. We will now go through each of these sections. ### Arguments (extra-args) Windmill scripts can take [arguments](../../../core_concepts/13_json_schema_and_parsing/index.mdx), and in order to define the names and types of the arguments you can use this section. These definitions will be parsed allowing the frontend to interactively display dynamic inputs for the script. ```yaml extra_vars: world_qualifier: type: string nested_object: type: object properties: a: type: string b: type: number some_arr: type: array objects: type: string ``` ![Parsing Yaml and generating UI](./extra_vars_ui.png) The type definition is inspired and tries to follow the [OpenAPI Data Types standard](https://swagger.io/docs/specification/data-models/data-types/). Note that not all features / types are supported, the best way to know what is supported is to test it out in the Web IDE. :::tip Argument defaults You can set a default value for your arguments by using a `default:` field, for example: ```yml extra_vars: my_string: type: string default: 'Fascinating String of Words' ``` ::: To use Windmill [resources](../../../core_concepts/3_resources_and_types/index.mdx) as types you can use the following type definition: ```yaml extra_vars: my_resource: type: windmill_resource resource_type: postgresql ``` ![Postgres Resource UI](./postgres_ui.png) Under the hood, Windmill will pass these variables using the `--extra-vars` flag to Ansible, so you can expect the according behavior. ### Static resources and variables Resources and [variables](../../../core_concepts/2_variables_and_secrets/index.mdx) that are hardcoded to a particular script can also be defined in the `extra-vars` section. This is because they are similarly passed through the `--extra-vars` flag in the Ansible playbook. This is what the syntax looks like: ```yml extra_vars: my_variable: type: windmill_variable variable: u/user/my_variable my_resource: type: windmill_resource resource: u/user/my_resource ``` Under `resource` or `variable` you can statically link the path to the resource/variable. As you do, you will notice the UI update and hide the resource input as it is now static. :::tip About static and non-static variables Note that Variables defined this way can only be static. If you want to use non-static Variables, define a normal argument with `type: string` and from the UI fill it with one of your Variables or Secrets. ::: ### Return values In Windmill scripts usually have a return value, which allows scripts to be chained in flows and run conditionally on the result of a previous operation. For Ansible playbooks you can achieve the same result by having one of the tasks (preferably the last one for coherence of results/errors) write a file named `result.json` with the JSON object you want to return: ```yaml --- tasks: [...] - name: Write variable my_result to result.json delegate_to: localhost copy: content: "{{ my_result | to_json }}" dest: result.json ``` Note that valid json must be written to the file or else the job will fail. Also, this should be done by the control node i.e. your worker, so it's important to use the `delegate_to: localhost` directive. ### Inventories When using ansbile playbooks, you would usually run a command such as `ansible-playbook playbook.yml -i inventory.ini`. The ways to pass inventories to Ansible in Windmill is by filling the following section: ```yaml inventory: - resource_type: ansible_inventory ``` To create similar resource type, refer to [creating plain text resources](../../../core_concepts/3_resources_and_types/index.mdx#plain-text-file-resources). Otherwise `ansible_inventory` should be available after syncing resource types from the hub. After adding this in the Web IDE, you will see a new `inventory.ini` argument pop up. You can then select or create a new ansible_inventory resource. ![inventory ui](./inventory_ui.png) If you don't want one of the inputs of the script be the inventory, you can pin a specific resource to the script by specifying its path. In this case you don't need to specify the resource_type anymore: ```yaml inventory: - resource: u/user/my_ansible_inventory ``` Then the UI will not prompt you for the inventory but will use this resource at every run of the script. If otherwise you wish to not specify any inventory, you can remove the section altogether By default, the inventory will be named `inventory.ini`, but if your inventory needs to have a different extension (e.g. dynamic invetories) you can specify the name of the inventory file like this: ```yaml inventory: - resource_type: c_dynamic_ansible_inventory name: hcloud.yml ``` Additionally, if you need to pass multiple inventories, you just need to continue the yaml array with your other invetories, they will all be passed to the `ansible-playbook` command. ```yaml # Declaring three different inventories to be passed to the playbook inventory: - resource: u/user/my_base_inventory name: base.ini - resource_type: ansible_inventory - resource_type: c_dynamic_ansible_inventory name: hcloud.yml ``` ### Additional inventories You can also declare additional inventories that will be made available as script arguments without specifying their source. This allows users to dynamically select inventories when running the script. ```yaml additional_inventories: - name: "Extra inventories" options: - "delegate_git_repository/hosts/inventory1.ini" - "delegate_git_repository/hosts/inventory2.ini" - "delegate_git_repository/hosts/inventory3.ini" ``` They can also be defined statically to always be passed in for this script: ```yaml additional_inventories: - "delegate_git_repository/hosts/permanent_inventory.ini" ``` Note that this only declares the inventory, but you still need to make it available by either having it in a git repo or using [file resources](#other-non-inventory-file-resources). Otherwise ansible will fail saying it couldn't find your inventory. ### Other non-inventory file resources It sometimes happens that your Ansible playbook depends on some text file existing at a relative path to the playbook. This can be a configuration file, a template, some other file that you can't inline or otherwise is simpler to keep as a separate file. In this case, Windmill's [plain text file resources](../../../core_concepts/3_resources_and_types/index.mdx#plain-text-file-resources) can be used to create these files at the specified path before running the playbook. The syntax will be the following: ```yaml files: - resource: u/user/fabulous_jinja_template target: ./config_template.j2 ``` In the example above, the resource `u/user/faboulous_jinja_template` is a special plain text file resource. The target `./config_template.j2` is the path relative to the playbook where the file will be created and where the playbook can access it. Now you can write your playbook assuming that this file will exist at the time of execution. #### Variable inside files If you want to achieve a similar effect with a variable or a secret, you can use a similar syntax: ```yaml files: - variable: u/user/my_ssh_key target: ./id_rsa ``` And the content of the variable will be written to the file. This is useful when you want to store the data in a secret for example, like you would do for SSH keys. #### Ansible and SSH To successfully have the playbook SSH, you might need to follow these tips: 1) Write the SSH key into a *secret* variable, and **make sure it has an ending newline**, otherwise you might get an error. ``` -----BEGIN OPENSSH PRIVATE KEY----- MHgCAQEEIQDWlK/Rk2h4WGKCxRs2SwplFVTSyqouwTQKIXrJ/L2clqAKBggqhkjO PQMBB6FEA0IABErMvG2Fa1jjG7DjEQuwRGCEDnVQc1G0ibU/HI1BjkIyf4d+sh 91GhwKDvHGbPaEQFWeTBQ+KbYwjtomLfmZM[...] -----END OPENSSH PRIVATE KEY----- ``` 2) Make a file for the script that will contain this SSH key. Make sure to add the `mode: '0600'` or you might get another error. ```yaml files: - variable: u/user/my_ssh_key target: ./ssh_key mode: '0600' ``` 3) In your inventory file, you'll want to add these : ```ini ... [your_host:vars] ansible_host=your_host ansible_user=john # The SSH user ansible_ssh_private_key_file=ssh_key # The file we declared where the SSH key can be found. ansible_ssh_common_args='-o StrictHostKeyChecking=no' # This skips host key verification, avoiding the error. Alternatively, you can add the host to known_hosts, either as an init script or a task in your playbook ... ``` ### Dependencies Ansible playbooks often depend on Python packages or Ansible Galaxy Collections. In Windmill you can specify these dependencies in the `dependencies` section and Windmill will take care of satisfying them before running the playbook. ```yaml dependencies: galaxy: collections: - name: community.general - name: community.vmware roles: - name: geerlingguy.apache python: - jmespath ``` The syntax is similar to `ansible-builder` and Execution Environments, however all is installed locally using the same technology as for managing [Python dependencies](../../../advanced/15_dependencies_in_python/index.mdx) in Python scripts, meaning no extra container is created. :::info Ansible vs Ansible-core Currently the Windmill image supporting Ansible runs the full `ansible` and not `ansible-core`. You can expect the respective collections to be preinstalled. ::: ### Git repo dependencies Outside of galaxy dependencies, a role or collection can exist on a git repo and be imported as such. The only caveat is that the repo needs to be a valid role or collection at its root. Check the [ansible documentation](https://docs.ansible.com/ansible/latest/collections_guide/collections_installing.html#install-multiple-collections-with-a-requirements-file) for more information. ```yaml collections: - name: git+https://github.com/organization/collection.git type: git version: main ``` To enable more flexibility however, it is possible to declare a git repo to be cloned at a specified location before the playbook is run. You can do this as follows: ```yaml git_repos: - url: git@github.com:some_user/your_git_repo.git target: ./git_repo1 commit: a34ac4fa branch: prod # An https or ssh url can be used: - url: https://github.com/some_user/your_other_git_repo.git target: ./git_repo2 ``` :::info Specifying a commit for your repo If you do not specify the commit to be used, the latest commit hash will be stored in the script lockfile on deployment, and all subsequent executions will use that commit. This is done to ensure reproducibility. If you need to update this, you can simply redeploy the script ::: If you want to clone a private repo, you can add the ssh private key like so: ```yaml git_ssh_identity: - u/user/ssh_id_priv git_repos: - url: git@github.com:some_user/your_private_repo.git target: ./my_roles_and_collections ``` ### Ansible Vault If you have files that are encrypted by ansible vault, you need to pass a password to decrypt them. This can be easily done by storing the password as a Windmill secret, and specifying the path to the secret in the metadata section of your playbook: ```yaml vault_password: u/user/ansible_vault_password ``` If you are using multiple vault password with Vault IDs, the setup is slightly different. You need to define your password files, and also add them as [file resources](#other-non-inventory-file-resources): ```yaml vault_id: - label1@password_filename1 - label2@password_filename2 - label3@password_filename3 files: - variable: u/user/password_for_label1 target: ./password_filename1 - variable: u/user/password_for_label2 target: ./password_filename2 - variable: u/user/password_for_label3 target: ./password_filename3 ``` ### Playbook options Pass `ansible-playbook` flags through the `options` block in the script metadata: ```yaml options: - vvv - forks: 10 - timeout: 30 - flush_cache - force_handlers - limit: webservers:!db1.example.com ``` Supported entries: | Entry | `ansible-playbook` flag | | --- | --- | | `vv` / `vvv` / `vvvv` (1 to 6 `v`s) | `-v` (verbosity) | | `verbosity: vvv` | same as above | | `forks: ` | `--forks` | | `timeout: ` | `--timeout` | | `flush_cache` | `--flush-cache` | | `force_handlers` | `--force-handlers` | | `limit: ` | `--limit` (target a subset of hosts) | The `limit` value may reference a script argument with `{{ argname }}` so a flow can pick the target host at runtime: ```yaml options: - limit: "{{ target_host }}" extra_vars: target_host: type: string ``` ### Delegate the environment setup to a git repo (EE) :::info EE feature Parts of this feature depend on instance-wide blob storage, which is only available in [Enterprise Edition](/pricing). ::: You can choose to set a git repository that contains all your inventories, custom roles, and playbooks as an alternate way to run your ansible script. When declaring this you will get an additional UI that lets you explore the repository, and some helpers to help you define the inventories. You can do this by either declaring this section on the metadata part of the script: ```yaml delegate_to_git_repo: resource: u/user/git_repo_resource ``` Or using the utility button that will help you pick a git repo resource to be used. You will need to first create a git_repository resource that points to the repository you're trying to use. You will then see your editor split in two and a Hovering popup indicating the alternate execution mode is detected. The first time you do this the repo viewer will show a button to load the git repository. This will clone and cache the contents of your repository in blob storage, for you to explore the files from within windmill. If you click on the top-right floating pop-up, you will access a screen letting you manage the definition of the git repo, and will contain some utils for ease of use. You can for example use the inventories section to define a subfolder containing your inventories and quickly import the filenames into the script. If you want to set a path to the playbook you want executed, you can do so like so: ```yaml delegate_to_git_repo: resource: u/user/git_repo_resource playbook: playbooks/your_playbook.yml ``` If this is undefined, the worker will default to using the second YAML section like normal. #### Inventory from the cloned repo Point ansible at an inventory file (or directory) that lives inside the cloned repository: ```yaml delegate_to_git_repo: resource: u/user/git_repo_resource playbook: playbooks/site.yml inventories_location: inventories/dev.ini ``` The path is relative to the repository root. It is added to the `ansible-playbook` invocation as `-i`, alongside any inventories declared in the script metadata. #### Installing requirements.yml from the cloned repo If your repository ships its own `requirements.yml`, set `install_requirements: true` to have Windmill install the listed roles and collections after cloning, before the playbook runs: ```yaml delegate_to_git_repo: resource: u/user/git_repo_resource playbook: playbooks/site.yml install_requirements: true ``` The worker looks for `requirements.yml`, `requirements.yaml`, `collections/requirements.yml`, and `roles/requirements.yml` at the root of the cloned repo and runs `ansible-galaxy role install -r` and `ansible-galaxy collection install -r` on every match. If none are found, the step is skipped. This is independent of the `dependencies.galaxy` block in the script metadata: use whichever fits your workflow, or both. #### Using the repo's ansible.cfg Ansible loads a single configuration file, the first one found among `ANSIBLE_CONFIG`, the working directory, `~/.ansible.cfg` and `/etc/ansible/ansible.cfg`, without merging. Windmill runs `ansible-playbook` from the job directory, where it writes its own generated `ansible.cfg`, so a config file checked into the repository is normally ignored. Set `ansible_cfg` to a repo-relative path to make it the effective configuration instead: ```yaml delegate_to_git_repo: resource: u/user/git_repo_resource playbook: playbooks/site.yml ansible_cfg: ansible.cfg ``` When set, Windmill points `ANSIBLE_CONFIG` at that file so the repository's settings (roles paths, inventory plugins, callbacks, `host_key_checking`, ssh arguments, etc.) apply as they would outside Windmill. Only the settings that depend on runtime state are layered back on top, through environment variables: - Temp and home directories (`ANSIBLE_HOME`, `ANSIBLE_LOCAL_TEMP`, `ANSIBLE_REMOTE_TEMP`) point to the ephemeral job directory. - Vault settings declared in the script metadata ([`vault_password`, `vault_id`](#ansible-vault)) keep taking precedence over the repo config. - The directories where Windmill installs galaxy roles and collections are prepended to the `roles_path` and `collections_path` declared in your config, so dependencies installed by Windmill (via `dependencies.galaxy` or `install_requirements`) and those shipped in the repo both resolve. The job fails if the file does not exist in the cloned repository. Without `ansible_cfg`, Windmill's generated config takes precedence as before. Like git repo delegation in general, this requires workers running with `DISABLE_NSJAIL=true`. #### Dynamic field values The `playbook`, `commit`, `inventories_location`, and `ansible_cfg` fields accept `{{ argname }}` placeholders that are substituted from the script's arguments before the repo is cloned and the playbook runs. This lets a flow pick the playbook, branch, or inventory at runtime: ```yaml delegate_to_git_repo: resource: u/user/git_repo_resource playbook: "playbooks/{{ playbook_name }}.yml" inventories_location: "inventories/{{ env }}.ini" commit: "{{ git_sha }}" extra_vars: playbook_name: type: string env: type: string git_sha: type: string ``` Only string, number, and boolean argument values can be substituted. The path fields reject absolute paths and `..` segments to prevent escaping the cloned repo. ## Instant preview & testing Look at the UI preview on the right: it was updated to match the input signature. Run a test (`Ctrl` + `Enter`) to verify everything works. Now let's go to the last step: the "Generated UI" settings. ## Generated UI From the Settings menu, the "Generated UI" tab lets you customize the script's arguments. The UI is generated from the Script's main function signature, but you can add additional constraints here. For example, we could use the `Customize property`: add a regex by clicking on `Pattern` to make sure users are providing a name with only alphanumeric characters: `^[A-Za-z0-9]+$`. Let's still allow numbers in case you are some tech billionaire's kid. ## Workflows as code One way to write distributed programs that execute distinct jobs is to use [flows](../../../flows/1_flow_editor.mdx) that chain scripts together. Another approach is to write a program that defines the jobs and their dependencies, and then execute that program directly in your script. This is known as [workflows as code](../../../core_concepts/31_workflows_as_code/index.mdx). ![Flow as code in Python](../../../core_concepts/31_workflows_as_code/wac-editor-1.png "Flow as code in Python") All details at: ## Run! We're done! Now let's look at what users of the script will do. Click on the [Deploy](../../../core_concepts/0_draft_and_deploy/index.mdx) button to load the script. You'll see the user input form we defined earlier. Note that Scripts are [versioned](../../../core_concepts/34_versioning/index.mdx#script-versioning) in Windmill, and each script version is uniquely identified by a hash. Fill in the input field, then hit "Run". You should see a run view, as well as your logs. All script runs are also available in the [Runs](../../../core_concepts/5_monitor_past_and_future_runs/index.mdx) menu on the left. You can also choose to [run the script from the CLI](../../../advanced/3_cli/index.mdx) with the pre-made Command-Line Interface call. ## Caching Every dependency on Python is cached on disk by default. Furthermore if you use the [Distributed cache storage](../../../core_concepts/38_object_storage_in_windmill/index.mdx#instance-object-storage-distributed-cache-for-python-rust-go), it will be available to every other worker, allowing fast startup for every worker. ## What's next? This script is a minimal working example, but there's a few more steps that can be useful in a real-world use case: - Pass [variables and secrets](../../../core_concepts/2_variables_and_secrets/index.mdx) to a script. - Connect to [resources](../../../core_concepts/3_resources_and_types/index.mdx). - [Trigger that script](../../../triggers/index.mdx) in many ways. - Compose scripts in [Flows](../../../flows/1_flow_editor.mdx), [low-code apps](../../../apps/0_app_editor/index.mdx) or [full-code apps](../../../full_code_apps/index.mdx). - You can [share your scripts](../../../misc/1_share_on_hub/index.md) with the community on [Windmill Hub](https://hub.windmill.dev). Once submitted, they will be verified by moderators before becoming available to everyone right within Windmill. Scripts are immutable and there is an hash for each deployment of a given script. Scripts are never overwritten and referring to a script by path is referring to the latest deployed hash at that path. For each script, a UI is autogenerated from the JSON schema inferred from the script signature, and can be customized further as standalone or embedded into rich UIs using the [App builder](../../7_apps_quickstart/index.mdx). In addition to the UI, sync and async [webhooks](../../../core_concepts/4_webhooks/index.mdx) are generated for each deployment. --- ## Bash / PowerShell / Nu quickstart Source: https://www.windmill.dev/docs/getting_started/scripts_quickstart/bash # Bash / PowerShell / Nu quickstart In this quick start guide, we will write our first script in Bash, PowerShell or Nu. This tutorial covers how to create a simple script through Windmill web IDE. See the dedicated page to [develop scripts locally](../../../advanced/4_local_development/index.mdx). Scripts are the basic building blocks in Windmill. They can be [run and scheduled](../../../triggers/index.mdx) as standalone, chained together to create [Flows](../../../flows/1_flow_editor.mdx) or displayed with a personalized User Interface as [Apps](../../7_apps_quickstart/index.mdx). Scripts consist of 2 parts: - [Code](#code). - [Settings](#settings): settings & metadata about the Script such as its path, summary, description, [JSON Schema](../../../core_concepts/13_json_schema_and_parsing/index.mdx) of its inputs (inferred from its signature). When stored in a code repository, those 2 parts are stored separately at `.sh` and `.script.yaml`. Below is a simple example of a script built in Bash/Powershell/Nu with Windmill: ```bash # shellcheck shell=bash # arguments of the form X="$I" are parsed as parameters X of type string url="${1:-default value}" status_code=$(curl -s -o /dev/null -w "%{http_code}" $url) if [[ $status_code == 2* ]] || [[ $status_code == 3* ]]; then echo "The URL is reachable!" else echo "The URL is not reachable." fi ``` ```powershell param($url = "default value") $status_code = (Invoke-WebRequest -Uri $url -Method Get).StatusCode if ($status_code -like "2*" -or $status_code -like "3*") { Write-Host "The URL is reachable!" } else { Write-Host "The URL is not reachable." } ``` ```python def main [ url: string = "default value" ] { try { # Nu will throw an error automatically if request fails http get $url echo "The URL is reachable!" } catch { echo "The URL is not reachable." } } ``` In this quick start guide, we'll create a script that greets the operator running it. From the Home page, click **New** and select **Script**. This will take you to the first step of script creation: [Metadata](../../../script_editor/settings.mdx#metadata). ## Settings ![New script](../../../../static/images/script_languages.png "New script") As part of the [settings](../../../script_editor/settings.mdx) menu, each script has metadata associated with it, enabling it to be defined and configured in depth. - **Summary** (optional) is a short, human-readable summary of the Script. It will be displayed as a title across Windmill. If omitted, the UI will use the `path` by default. - **Path** is the Script's unique identifier that consists of the [script's owner](../../../core_concepts/16_roles_and_permissions/index.mdx), and the script's name. The owner can be either a user, or a group ([folder](../../../core_concepts/8_groups_and_folders/index.mdx#folders)). - **Description** is where you can give instructions through the [auto-generated UI](../../../core_concepts/6_auto_generated_uis/index.mdx) to users on how to run your Script. It supports markdown. - **Language** of the script. - **Script kind**: Action (by default), [Trigger](../../../flows/10_flow_trigger.mdx), [Approval](../../../flows/11_flow_approval.mdx), [Error handler](../../../flows/7_flow_error_handler.md) or [Preprocessor](../../../core_concepts/43_preprocessors/index.mdx). This acts as a tag to filter appropriate scripts from the [flow editor](../../6_flows_quickstart/index.mdx). This menu also has additional settings on [Runtime](../../../script_editor/settings.mdx#runtime), [Generated UI](#generated-ui) and [Triggers](../../../script_editor/settings.mdx#triggers). Now click on the code editor on the left side. ## Code Windmill provides an online editor to work on your Scripts. The left-side is the editor itself. The right-side [previews the UI](../../../core_concepts/6_auto_generated_uis/index.mdx) that Windmill will generate from the Script's signature - this will be visible to the users of the Script. You can preview that UI, provide input values, and [test your script](#instant-preview--testing) there. ![Editor for Bash](./editor_bash.png.webp) ### Bash As we picked `Bash` for this example, Windmill provided some Bash boilerplate. Let's take a look: ```bash # shellcheck shell=bash # arguments of the form X="$I" are parsed as parameters X of type string msg="$1" dflt="${2:-default value}" # the last line of the stdout is the return value echo "Hello $msg" ``` In Bash, the arguments are inferred from the arguments requiring a \$1, \$2, \$3. Default arguments can be specified using the syntax above: `dflt="${2:-default value}"`. The last line of the output, here `echo "Hello $msg"`, is the return value, which might be useful if the script is used in a [flow](../../../flows/1_flow_editor.mdx) or [app](../../../full_code_apps/index.mdx) to pass its result on. ### PowerShell As we picked `PowerShell` for this example, Windmill provided some PowerShell boilerplate. Let's take a look: ```powershell param($Msg, [string[]]$Names, [PSCustomObject]$Obj, $Dflt = "default value", [int]$Nb = 3) # Import-Module MyModule # the last line of the stdout is the return value Write-Output "Hello $Msg" ``` In PowerShell, the arguments are inferred from the param instruction. It has to be first in the script. Arguments can be of type `string`, `int`/`long`/`double`/`decimal`/`single`, `PSCustomObject` (parsed from JSON), `datetime`, `bool` and array of these types. Default arguments can be specified using the following syntax: `$argument_name = "Its default value"`. The last line of the output, here `Write-Output "Hello $Msg"`, is the return value, which might be useful if the script is used in a [flow](../../../flows/1_flow_editor.mdx) or [app](../../../full_code_apps/index.mdx) to pass its result on. ### Nu Unlike `Bash` and `PowerShell`, `Nu` requires main function and all arguments should defined in signature. It supports typed, optional and default arguments. ```python def main [ msg, dflt = "default value", nb: number = 3 ] { echo $"Hello ($msg)" } ``` ```python use std assert # Nushell # A new type of shell def main [ no_default: string, name = "Nicolas Bourbaki", age: int = 42, date_of_birth?: datetime, obj: record = {"records": "included"}, l: list = ["or", "lists!"], tables?: table, enable_kill_mode?: bool = true, ] { # Test # https://www.nushell.sh/book/testing.html assert ($age == 42) print $"Hello World and a warm welcome especially to ($name)" print "and its acolytes.." $age $obj $l print $tables let secret = try { get_variable f/examples/secret } catch { 'No secret yet at f/examples/secret !' }; print $"The variable at \`f/examples/secret\`: ($secret)" # fetch context variables let user = $env.WM_USERNAME # Nu pipelines ls | where size > 1kb | sort-by modified | print "ls:" $in # Nu works with existing data # Nu speaks JSON, YAML, SQLite, Excel, and more out of the box. # It's easy to bring data into a Nu pipeline whether it's in a file, a database, or a web API: let nu_license = http get https://api.github.com/repos/nushell/nushell | get license return { split: ($name | split words), user: $user, nu_license: $nu_license} # Interested in learning more? # https://www.nushell.sh/book/getting_started.html ``` One of the strong sides of `Nu` is that it is cross-platform. If you have linux workers and [windows workers](../../../misc/17_windows_workers/index.mdx) Nushell scripts will be able to run on both! If you are interested in `Nu` you can read their [official documentation](https://www.nushell.sh/book/getting_started.html) ## Instant preview & testing Look at the UI preview on the right: it was updated to match the input signature. Run a test (`Ctrl` + `Enter`) to verify everything works. You can change how the UI behaves by changing the main signature. For example, if you add a default for the `name` argument, the UI won't consider this field as required anymore. ```bash argument_name="${1:-Its default value}" ``` ```bash $argument_name = "Its default value" ``` ```python def main [ argument_name = "Its default value" ] { } ``` Now let's go to the last step: the "Generated UI" settings. ## Generated UI From the Settings menu, the "Generated UI" tab lets you customize the script's arguments. The UI is generated from the Script's main function signature, but you can add additional constraints here. For example, we could use the `Customize property`: add a regex by clicking on `Pattern` to make sure users are providing a name with only alphanumeric characters: `^[A-Za-z0-9]+$`. Let's still allow numbers in case you are some tech billionaire's kid. ![Advanced settings for Bash](./customize_bash.png.webp) ## Run! We're done! Now let's look at what users of the script will do. Click on the [Deploy](../../../core_concepts/0_draft_and_deploy/index.mdx) button to load the script. You'll see the user input form we defined earlier. Note that Scripts are [versioned](../../../core_concepts/34_versioning/index.mdx#script-versioning) in Windmill, and each script version is uniquely identified by a hash. Fill in the input field, then hit "Run". You should see a run view, as well as your logs. All script runs are also available in the [Runs](../../../core_concepts/5_monitor_past_and_future_runs/index.mdx) menu on the left. ![Run Hello in Bash](./run_bash.png.webp) You can also choose to [run the script from the CLI](../../../advanced/3_cli/index.mdx) with the pre-made Command-line interface call. ## JSON result The last line returned by the script will be the string result. To use a json result instead, output your result in `./result.json` and it will be automatically picked-up and considered as the JSON result for Bash and Powershell scripts. For Nu first returned data from main function will be used as a result. ## Run Docker containers In some cases where your task requires a complex set of dependencies or is implemented in a non-supported language, you can still include it as a flow step or individual script. Windmill supports running any docker container through its Bash support with the `# sandbox ` annotation. The image runs inside the job's own nsjail sandbox, daemonless: no Docker socket or Docker-in-Docker sidecar required. ## Run on a remote SSH host A bash script starting with a `#ssh ` directive runs on the remote host described by the referenced `ssh_target` resource instead of the worker, with the same typed arguments, result collection and live logs ([Enterprise Edition](/pricing), off by default). ## What's next? This script is a minimal working example, but there's a few more steps that can be useful in a real-world use case: - Pass [variables and secrets](../../../core_concepts/2_variables_and_secrets/index.mdx) to a script. - Connect to [resources](../../../core_concepts/3_resources_and_types/index.mdx). - [Trigger that script](../../../triggers/index.mdx) in many ways. - Compose scripts in [Flows](../../../flows/1_flow_editor.mdx), [low-code apps](../../../apps/0_app_editor/index.mdx) or [full-code apps](../../../full_code_apps/index.mdx). - You can [share your scripts](../../../misc/1_share_on_hub/index.md) with the community on [Windmill Hub](https://hub.windmill.dev). Once submitted, they will be verified by moderators before becoming available to everyone right within Windmill. Scripts are immutable and there is an hash for each deployment of a given script. Scripts are never overwritten and referring to a script by path is referring to the latest deployed hash at that path. For each script, a UI is autogenerated from the JSON schema inferred from the script signature, and can be customized further as standalone or embedded into rich UIs using the [App builder](../../7_apps_quickstart/index.mdx). In addition to the UI, sync and async [webhooks](../../../core_concepts/4_webhooks/index.mdx) are generated for each deployment. --- ## C# quickstart Source: https://www.windmill.dev/docs/getting_started/scripts_quickstart/csharp # C# quickstart In this quick start guide, we will write our first script in [C#](https://learn.microsoft.com/en-us/dotnet/csharp/tour-of-csharp/overview). ![Editor for C#](./editor_csharp.png "Script in C#") This tutorial covers how to create a simple script through Windmill web IDE. See the dedicated page to [develop scripts locally](../../../advanced/4_local_development/index.mdx). Scripts are the basic building blocks in Windmill. They can be [run and scheduled](../../../triggers/index.mdx) as standalone, chained together to create [Flows](../../../flows/1_flow_editor.mdx) or displayed with a personalized User Interface as [Apps](../../7_apps_quickstart/index.mdx). Scripts consist of 2 parts: - [Code](#code): for C# scripts, it must have at least a **public** static Main method inside a class (the name of the class is irrelevant). - [Settings](#settings): settings & metadata about the Script such as its path, summary, description, [JSON Schema](../../../core_concepts/13_json_schema_and_parsing/index.mdx) of its inputs (inferred from its signature). When stored in a code repository, these 2 parts are stored separately at `.cs` and `.script.yaml` Windmill automatically manages [dependencies](../../../advanced/6_imports/index.mdx) for you. When you import libraries in your C# script, Windmill parses these imports upon saving the script and automatically generates a list of dependencies. It then spawns a dependency job to associate these NuGet packages with a lockfile, ensuring that the same version of the script is always executed with the same versions of its dependencies. From the Home page, click **New** and select **Script**. This will take you to the first step of script creation: [Metadata](../../../script_editor/settings.mdx#metadata). ## Settings ![New script](./select_csharp.png "New script") As part of the [settings](../../../script_editor/settings.mdx) menu, each script has metadata associated with it, enabling it to be defined and configured in depth. - **Summary** (optional) is a short, human-readable summary of the Script. It will be displayed as a title across Windmill. If omitted, the UI will use the `path` by default. - **Path** is the Script's unique identifier that consists of the [script's owner](../../../core_concepts/16_roles_and_permissions/index.mdx), and the script's name. The owner can be either a user, or a group ([folder](../../../core_concepts/8_groups_and_folders/index.mdx#folders)). - **Description** is where you can give instructions through the [auto-generated UI](../../../core_concepts/6_auto_generated_uis/index.mdx) to users on how to run your Script. It supports markdown. - **Language** of the script. - **Script kind**: Action (by default), [Trigger](../../../flows/10_flow_trigger.mdx), [Approval](../../../flows/11_flow_approval.mdx), [Error handler](../../../flows/7_flow_error_handler.md) or [Preprocessor](../../../core_concepts/43_preprocessors/index.mdx). This acts as a tag to filter appropriate scripts from the [flow editor](../../6_flows_quickstart/index.mdx). This menu also has additional settings on [Runtime](../../../script_editor/settings.mdx#runtime), [Generated UI](#generated-ui) and [Triggers](../../../script_editor/settings.mdx#triggers). Now click on the code editor on the left side. ## Code Windmill provides an online editor to work on your Scripts. The left-side is the editor itself. The right-side [previews the UI](../../../core_concepts/6_auto_generated_uis/index.mdx) that Windmill will generate from the Script's signature - this will be visible to the users of the Script. You can preview that UI, provide input values, and [test your script](#instant-preview--testing) there. ![Editor for C#](./editor_csharp.png "Editor for C#") As we picked `C#` for this example, Windmill provided some boilerplate. Let's take a look: ```cs #r "nuget: Humanizer, 2.14.1" using System; using System.Linq; using Humanizer; class Script { public static DateTime Main(string[] extraWords, string word = "clue", int highNumberThreshold = 50) { Console.WriteLine("Hello, World!"); Console.WriteLine("Your chosen words are pluralized here:"); string[] newWordArray = extraWords.Concat(new[] { word }).ToArray(); foreach (var s in newWordArray) { Console.WriteLine($" {s.Pluralize()}"); } var random = new Random(); int randomNumber = random.Next(1, 101); Console.WriteLine($"Random number: {randomNumber}"); string greeting = randomNumber > highNumberThreshold ? "High number!" : "Low number!"; greeting += " (according to the threshold parameter)"; Console.WriteLine(greeting); // Humanize a timespan var timespan = TimeSpan.FromMinutes(90); Console.WriteLine($"Timespan: {timespan.Humanize()}"); // Humanize numbers into words int number = 123; Console.WriteLine($"Number: {number.ToWords()}"); // Pluralize words string singular = "apple"; // Humanize date difference var date = DateTime.UtcNow.AddDays(-3); Console.WriteLine($"Date: {date.Humanize()}"); return date; } } ``` In Windmill, scripts need to have a main function that will be the script's entrypoint. There are a few important things to note about the `Main`. - The arguments are used for generating 1. the [input spec](../../../core_concepts/13_json_schema_and_parsing/index.mdx) of the Script 2. the [frontend](../../../core_concepts/6_auto_generated_uis/index.mdx) that you see when running the Script as a standalone app. - Type annotations are used to generate the UI form, and help pre-validate inputs. While not mandatory, they are highly recommended. You can customize the UI in later steps (but not change the input type!). Packages can be installed through NuGet. Just add the dependencies you need at the top of the file, using the following format: ```cs #r "nuget: Humanizer, 2.14.1" #r "nuget: AutoMapper, 6.1.0" ``` :::caution Note that only the lines at the very top will be taken into account. ::: ## Instant preview & testing Look at the UI preview on the right: it was updated to match the input signature. Run a test (`Ctrl` + `Enter`) to verify everything works. ## Generated UI From the Settings menu, the "Generated UI" tab lets you customize the script's arguments. The UI is generated from the Script's main function signature, but you can add additional constraints here. For example, we could use the `Customize property`: add a regex by clicking on `Pattern` to make sure users are providing a name with only alphanumeric characters: `^[A-Za-z0-9]+$`. Let's still allow numbers in case you are some tech billionaire's kid. ![Advanced settings for C#](./customize_csharp.png "Advanced settings for C#") ## Run! We're done! Now let's look at what users of the script will do. Click on the [Deploy](../../../core_concepts/0_draft_and_deploy/index.mdx) button to load the script. You'll see the user input form we defined earlier. Note that Scripts are [versioned](../../../core_concepts/34_versioning/index.mdx#script-versioning) in Windmill, and each script version is uniquely identified by a hash. Fill in the input field, then hit "Run". You should see a run view, as well as your logs. All script runs are also available in the [Runs](../../../core_concepts/5_monitor_past_and_future_runs/index.mdx) menu on the left. ![Run in C#](./run_csharp.png "Run in C#") You can also choose to [run the script from the CLI](../../../advanced/3_cli/index.mdx) with the pre-made Command-line interface call. ## Caching Every binary on C# is cached on disk by default. Furthermore if you use the [Distributed cache storage](../../../core_concepts/38_object_storage_in_windmill/index.mdx#instance-object-storage-distributed-cache-for-python-rust-go), it will be available to every other worker, allowing fast startup for every worker. ## What's next? This script is a minimal working example, but there's a few more steps that can be useful in a real-world use case: - Pass [variables and secrets](../../../core_concepts/2_variables_and_secrets/index.mdx) to a script. - Connect to [resources](../../../core_concepts/3_resources_and_types/index.mdx). - [Trigger that script](../../../triggers/index.mdx) in many ways. - Compose scripts in [Flows](../../../flows/1_flow_editor.mdx), [low-code apps](../../../apps/0_app_editor/index.mdx) or [full-code apps](../../../full_code_apps/index.mdx). - You can [share your scripts](../../../misc/1_share_on_hub/index.md) with the community on [Windmill Hub](https://hub.windmill.dev). Once submitted, they will be verified by moderators before becoming available to everyone right within Windmill. Scripts are immutable and there is an hash for each deployment of a given script. Scripts are never overwritten and referring to a script by path is referring to the latest deployed hash at that path. For each script, a UI is autogenerated from the JSON schema inferred from the script signature, and can be customized further as standalone or embedded into rich UIs using the [App builder](../../7_apps_quickstart/index.mdx). In addition to the UI, sync and async [webhooks](../../../core_concepts/4_webhooks/index.mdx) are generated for each deployment. --- ## Docker quickstart Source: https://www.windmill.dev/docs/getting_started/scripts_quickstart/docker # Docker quickstart In this quick start guide, we will write our first script ran from a [Docker](https://www.docker.com/) container. Windmill natively supports Python, TypeScript, Go, PHP, Bash or SQL. In some cases where your task requires a complex set of dependencies or is implemented in a non-supported language, Windmill allows running any Docker container through its [Bash](../4_bash_quickstart/index.mdx) support. The recommended way is the sandboxed `# sandbox ` runtime: it is daemonless (no Docker socket or Docker-in-Docker sidecar) and runs the image inside the job's own nsjail sandbox, so it is safe to run untrusted code and is available on [Windmill Cloud](/pricing). See [Run Docker containers](../../../advanced/7_docker/index.mdx) for the full reference. ![script 1](../../../advanced/7_docker/as_script.png.webp) This tutorial covers how to create a simple script through Windmill web IDE. See the dedicated page to [develop scripts locally](../../../advanced/4_local_development/index.mdx). Scripts are the basic building blocks in Windmill. They can be [run and scheduled](../../../triggers/index.mdx) as standalone, chained together to create [Flows](../../../flows/1_flow_editor.mdx) or displayed with a personalized User Interface as [Apps](../../7_apps_quickstart/index.mdx). Scripts consist of 2 parts: - [Code](#code). - [Settings](#settings): settings & metadata about the Script such as its path, summary, description, [JSON Schema](../../../core_concepts/13_json_schema_and_parsing/index.mdx) of its inputs (inferred from its signature). When stored in a code repository, those 2 parts are stored separately at `.docker` and `.script.yaml`. Below is a simple example of a script built using Bash to run a Docker container from Windmill: ```bash # shellcheck shell=bash # sandbox alpine:latest # The "# sandbox " annotation runs this script INSIDE the image above, # sandboxed via nsjail. The body runs with the image's /bin/sh and windmill args # bind positionally as $1, $2, ... msg="${1:-world}" echo "Hello $msg" cat /etc/os-release | head -1 ``` To see more details about the sandboxed runtime, see [Run docker containers](../../../advanced/7_docker/index.mdx). :::note A bare `# docker` annotation selects a separate legacy daemon-based runtime that requires a mounted Docker socket and is intended for trusted setups only. New scripts should use `# sandbox `. ::: ## Settings ![New script](../../../../static/images/script_languages.png "New script") As part of the [settings](../../../script_editor/settings.mdx) menu, each script has metadata associated with it, enabling it to be defined and configured in depth. - **Summary** (optional) is a short, human-readable summary of the Script. It will be displayed as a title across Windmill. If omitted, the UI will use the `path` by default. - **Path** is the Script's unique identifier that consists of the [script's owner](../../../core_concepts/16_roles_and_permissions/index.mdx), and the script's name. The owner can be either a user, or a group ([folder](../../../core_concepts/8_groups_and_folders/index.mdx#folders)). - **Description** is where you can give instructions through the [auto-generated UI](../../../core_concepts/6_auto_generated_uis/index.mdx) to users on how to run your Script. It supports markdown. - **Language** of the script. - **Script kind**: Action (by default), [Trigger](../../../flows/10_flow_trigger.mdx), [Approval](../../../flows/11_flow_approval.mdx), [Error handler](../../../flows/7_flow_error_handler.md) or [Preprocessor](../../../core_concepts/43_preprocessors/index.mdx). This acts as a tag to filter appropriate scripts from the [flow editor](../../6_flows_quickstart/index.mdx). This menu also has additional settings on [Runtime](../../../script_editor/settings.mdx#runtime), [Generated UI](#generated-ui) and [Triggers](../../../script_editor/settings.mdx#triggers). Now click on the code editor on the left side. ### Code Windmill provides an online editor to work on your Scripts. The left-side is the editor itself. The right-side [previews the UI](../../../core_concepts/6_auto_generated_uis/index.mdx) that Windmill will generate from the Script's signature - this will be visible to the users of the Script. You can preview that UI, provide input values, and [test your script](#instant-preview--testing) there. ![Editor for Bash](../4_bash_quickstart/editor_bash.png.webp) As we picked `Docker` for this example, Windmill provided some Bash boilerplate. Let's take a look: ```bash # shellcheck shell=bash # sandbox alpine:latest # The "# sandbox " annotation runs this script INSIDE the image above, # sandboxed via nsjail. The body runs with the image's /bin/sh and windmill args # bind positionally as $1, $2, ... msg="${1:-world}" echo "Hello $msg" cat /etc/os-release | head -1 ``` `msg` is just a normal Bash variable. It can be used to pass arguments to the script. This syntax is the standard Bash one to assign default values to parameters. With the `# sandbox ` annotation, the rest of the script runs **inside** that image, sandboxed by nsjail: the image rootfs is pulled and the body runs chrooted in it via the image's `/bin/sh`, inheriting the job's confinement. Windmill arguments bind positionally as `$1`, `$2`, … It is daemonless, so there is no Docker socket to mount and no `docker run` to manage. ### Instant preview & testing Look at the UI preview on the right: it was updated to match the input signature. Run a test (`Ctrl` + `Enter`) to verify everything works. Now let's go to the last step: the "Generated UI" settings. ## Generated UI From the Settings menu, the "Generated UI" tab lets you customize the script's arguments. The UI is generated from the Script's main function signature, but you can add additional constraints here. For example, we could use the `Customize property`: add a regex by clicking on `Pattern` to make sure users are providing a name with only alphanumeric characters: `^[A-Za-z0-9]+$`. Let's still allow numbers in case you are some tech billionaire's kid. ![Advanced settings for Bash](../4_bash_quickstart/customize_bash.png.webp) We're done! Save your script. Note that Scripts are [versioned](../../../core_concepts/34_versioning/index.mdx#script-versioning) in Windmill, and each script version is uniquely identified by a hash. ### Run! Now let's look at what users of the script will do. Click on the [Deploy](../../../core_concepts/0_draft_and_deploy/index.mdx) button to load the script. You'll see the user input form we defined earlier. Fill in the input field, then hit "Run". You should see a run view, as well as your logs. All script runs are also available in the [Runs](../../../core_concepts/5_monitor_past_and_future_runs/index.mdx) menu on the left. ![Run Hello in Bash](../4_bash_quickstart/run_bash.png.webp) You can also choose to [run the script from the CLI](../../../advanced/3_cli/index.mdx) with the pre-made Command-line interface call. ## JSON result The last line returned by the script will be the string result. To use a json result instead, output your result in `./result.json` and it will be automatically picked-up and considered as the JSON result for Bash and Powershell scripts. ## What's next? This script is a minimal working example, but there's a few more steps that can be useful in a real-world use case: - Pass [variables and secrets](../../../core_concepts/2_variables_and_secrets/index.mdx) to a script. - Connect to [resources](../../../core_concepts/3_resources_and_types/index.mdx). - [Trigger that script](../../../triggers/index.mdx) in many ways. - Compose scripts in [Flows](../../../flows/1_flow_editor.mdx), [low-code apps](../../../apps/0_app_editor/index.mdx) or [full-code apps](../../../full_code_apps/index.mdx). - You can [share your scripts](../../../misc/1_share_on_hub/index.md) with the community on [Windmill Hub](https://hub.windmill.dev). Once submitted, they will be verified by moderators before becoming available to everyone right within Windmill. Scripts are immutable and there is an hash for each deployment of a given script. Scripts are never overwritten and referring to a script by path is referring to the latest deployed hash at that path. For each script, a UI is autogenerated from the JSON schema inferred from the script signature, and can be customized further as standalone or embedded into rich UIs using the [App builder](../../7_apps_quickstart/index.mdx). In addition to the UI, sync and async [webhooks](../../../core_concepts/4_webhooks/index.mdx) are generated for each deployment. --- ## Go quickstart Source: https://www.windmill.dev/docs/getting_started/scripts_quickstart/go # Go quickstart In this quick start guide, we will write our first script in Go. This tutorial covers how to create a simple script through Windmill web IDE. See the dedicated page to [develop scripts locally](../../../advanced/4_local_development/index.mdx). Scripts are the basic building blocks in Windmill. They can be [run and scheduled](../../../triggers/index.mdx) as standalone, chained together to create [Flows](../../../flows/1_flow_editor.mdx) or displayed with a personalized User Interface as [Apps](../../7_apps_quickstart/index.mdx). Scripts consist of 2 parts: - [Code](#code): for Go scripts, it must have at least a main function. - [Settings](#settings): settings & metadata about the Script such as its path, summary, description, [JSON Schema](../../../core_concepts/13_json_schema_and_parsing/index.mdx) of its inputs (inferred from its signature). When stored in a code repository, those 2 parts are stored separately at `.go` and `.script.yaml`. Below is a simple example of a script built in Go with Windmill: ```go package inner import ( "net/http" ) func main(url string) (bool, error) { resp, err := http.Get(url) // send a GET request to the provided URL if err != nil { return false, err // if there is an error, return false and the error } defer resp.Body.Close() // make sure to close the response body when the function returns // if the status code is between 200 and 299, the page exists if resp.StatusCode >= 200 && resp.StatusCode <= 299 { return true, nil } // if the status code is not between 200 and 299, the page does not exist return false, nil } ``` In this quick start guide, we'll create a script that greets the operator running it. From the Home page, click **New** and select **Script**. This will take you to the first step of script creation: [Metadata](../../../script_editor/settings.mdx#metadata). ## Settings ![New script](../../../../static/images/script_languages.png "New script") As part of the [settings](../../../script_editor/settings.mdx) menu, each script has metadata associated with it, enabling it to be defined and configured in depth. - **Summary** (optional) is a short, human-readable summary of the Script. It will be displayed as a title across Windmill. If omitted, the UI will use the `path` by default. - **Path** is the Script's unique identifier that consists of the [script's owner](../../../core_concepts/16_roles_and_permissions/index.mdx), and the script's name. The owner can be either a user, or a group ([folder](../../../core_concepts/8_groups_and_folders/index.mdx#folders)). - **Description** is where you can give instructions through the [auto-generated UI](../../../core_concepts/6_auto_generated_uis/index.mdx) to users on how to run your Script. It supports markdown. - **Language** of the script. - **Script kind**: Action (by default), [Trigger](../../../flows/10_flow_trigger.mdx), [Approval](../../../flows/11_flow_approval.mdx), [Error handler](../../../flows/7_flow_error_handler.md) or [Preprocessor](../../../core_concepts/43_preprocessors/index.mdx). This acts as a tag to filter appropriate scripts from the [flow editor](../../6_flows_quickstart/index.mdx). This menu also has additional settings on [Runtime](../../../script_editor/settings.mdx#runtime), [Generated UI](#generated-ui) and [Triggers](../../../script_editor/settings.mdx#triggers). Now click on the code editor on the left side. ## Code Windmill provides an online editor to work on your Scripts. The left-side is the editor itself. The right-side [previews the UI](../../../core_concepts/6_auto_generated_uis/index.mdx) that Windmill will generate from the Script's signature - this will be visible to the users of the Script. You can preview that UI, provide input values, and [test your script](#instant-preview--testing) there. ![Editor for python](./editor_go.png.webp) As we picked `Go` for this example, Windmill provided some Go boilerplate. Let's take a look: ```go package inner import ( "fmt" "rsc.io/quote" // wmill "github.com/windmill-labs/windmill-go-client" ) // the main must return (interface{}, error) func main(x string, nested struct { Foo string `json:"foo"` }) (interface{}, error) { fmt.Println("Hello, World") fmt.Println(nested.Foo) fmt.Println(quote.Opt()) // v, _ := wmill.GetVariable("f/examples/secret") return x, nil } ``` In Windmill, scripts need to have a `main` function that will be the script's entrypoint. There are a few important things to note about the `main`. - The main arguments are used for generating 1. the [input spec](../../../core_concepts/13_json_schema_and_parsing/index.mdx) of the Script 2. the [frontend](../../../core_concepts/6_auto_generated_uis/index.mdx) that you see when running the Script as a standalone app. - Type annotations are used to generate the UI form, and help pre-validate inputs. While not mandatory, they are highly recommended. You can customize the UI in later steps (but not change the input type!). - In Go, the main function must return (interface{}, error). The last import line imports the Windmill client, which is needed for example to access [variables](../../../core_concepts/2_variables_and_secrets/index.mdx) or [resources](../../../core_concepts/3_resources_and_types/index.mdx). In Go, the dependencies and their versions are contained in the script and hence there is no need for any additional steps. Back to our Hello World. We can clean up unused import statements, change the main to take in the user's name. Let's also return the `name`, maybe we can use this later if we use this Script within a [flow](../../../flows/1_flow_editor.mdx) or [app](../../../full_code_apps/index.mdx) and need to pass its result on. ```go package inner import ( "fmt" ) func main(name string) (string, error) { return fmt.Sprintf("Hello %s", name), nil } ``` ## Instant preview & testing Look at the UI preview on the right: it was updated to match the input signature. Run a test (`Ctrl` + `Enter`) to verify everything works. Now let's go to the last step: the "Generated UI" settings. ## Generated UI From the Settings menu, the "Generated UI" tab lets you customize the script's arguments. The UI is generated from the Script's main function signature, but you can add additional constraints here. For example, we could use the `Customize property`: add a regex by clicking on `Pattern` to make sure users are providing a name with only alphanumeric characters: `^[A-Za-z0-9]+$`. Let's still allow numbers in case you are some tech billionaire's kid. ![Advanced settings for Python](./customize_go.png.webp) ## Run! We're done! Now let's look at what users of the script will do. Click on the [Deploy](../../../core_concepts/0_draft_and_deploy/index.mdx) button to load the script. You'll see the user input form we defined earlier. Note that Scripts are [versioned](../../../core_concepts/34_versioning/index.mdx#script-versioning) in Windmill, and each script version is uniquely identified by a hash. Fill in the input field, then hit "Run". You should see a run view, as well as your logs. All script runs are also available in the [Runs](../../../core_concepts/5_monitor_past_and_future_runs/index.mdx) menu on the left. ![Run hello world in Python](./run_go.png.webp) You can also choose to [run the script from the CLI](../../../advanced/3_cli/index.mdx) with the pre-made Command-line interface call. ## Caching Every bundle on Go is cached on disk by default. Furthermore if you use the [Distributed cache storage](../../../core_concepts/38_object_storage_in_windmill/index.mdx#instance-object-storage-distributed-cache-for-python-rust-go), it will be available to every other worker, allowing fast startup for every worker. ## What's next? This script is a minimal working example, but there's a few more steps that can be useful in a real-world use case: - Pass [variables and secrets](../../../core_concepts/2_variables_and_secrets/index.mdx) to a script. - Connect to [resources](../../../core_concepts/3_resources_and_types/index.mdx). - [Trigger that script](../../../triggers/index.mdx) in many ways. - Compose scripts in [Flows](../../../flows/1_flow_editor.mdx), [low-code apps](../../../apps/0_app_editor/index.mdx) or [full-code apps](../../../full_code_apps/index.mdx). - You can [share your scripts](../../../misc/1_share_on_hub/index.md) with the community on [Windmill Hub](https://hub.windmill.dev). Once submitted, they will be verified by moderators before becoming available to everyone right within Windmill. Scripts are immutable and there is an hash for each deployment of a given script. Scripts are never overwritten and referring to a script by path is referring to the latest deployed hash at that path. For each script, a UI is autogenerated from the JSON schema inferred from the script signature, and can be customized further as standalone or embedded into rich UIs using the [App builder](../../7_apps_quickstart/index.mdx). In addition to the UI, sync and async [webhooks](../../../core_concepts/4_webhooks/index.mdx) are generated for each deployment. --- ## Java quickstart Source: https://www.windmill.dev/docs/getting_started/scripts_quickstart/java # Java quickstart In this quick start guide, we will write our first script in [Java](https://www.java.com/). ![Editor for Java](./java_exec.png "Script in Java") This tutorial covers how to create a simple script through Windmill web IDE. See the dedicated page to [develop scripts locally](../../../advanced/4_local_development/index.mdx). Scripts are the basic building blocks in Windmill. They can be [run and scheduled](../../../triggers/index.mdx) as standalone, chained together to create [Flows](../../../flows/1_flow_editor.mdx) or displayed with a personalized User Interface as [Apps](../../7_apps_quickstart/index.mdx). Scripts consist of 2 parts: - [Code](#code): for Java scripts, it must have at least a **public** static main method inside a Main class. - [Settings](#settings): settings & metadata about the Script such as its path, summary, description, [JSON Schema](../../../core_concepts/13_json_schema_and_parsing/index.mdx) of its inputs (inferred from its signature). From the Home page, click **New** and select **Script**. This will take you to the first step of script creation: [Metadata](../../../script_editor/settings.mdx#metadata). ## Settings ![New script](./java_settings.png "New script") As part of the [settings](../../../script_editor/settings.mdx) menu, each script has metadata associated with it, enabling it to be defined and configured in depth. - **Summary** (optional) is a short, human-readable summary of the Script. It will be displayed as a title across Windmill. If omitted, the UI will use the `path` by default. - **Path** is the Script's unique identifier that consists of the [script's owner](../../../core_concepts/16_roles_and_permissions/index.mdx), and the script's name. The owner can be either a user, or a group ([folder](../../../core_concepts/8_groups_and_folders/index.mdx#folders)). - **Description** is where you can give instructions through the [auto-generated UI](../../../core_concepts/6_auto_generated_uis/index.mdx) to users on how to run your Script. It supports markdown. - **Language** of the script. - **Script kind**: Action (by default), [Trigger](../../../flows/10_flow_trigger.mdx), [Approval](../../../flows/11_flow_approval.mdx), [Error handler](../../../flows/7_flow_error_handler.md) or [Preprocessor](../../../core_concepts/43_preprocessors/index.mdx). This acts as a tag to filter appropriate scripts from the [flow editor](../../6_flows_quickstart/index.mdx). This menu also has additional settings on [Runtime](../../../script_editor/settings.mdx#runtime), [Generated UI](#generated-ui) and [Triggers](../../../script_editor/settings.mdx#triggers). Now click on the code editor on the left side. ## Code Windmill provides an online editor to work on your Scripts. The left-side is the editor itself. The right-side [previews the UI](../../../core_concepts/6_auto_generated_uis/index.mdx) that Windmill will generate from the Script's signature - this will be visible to the users of the Script. You can preview that UI, provide input values, and [test your script](#instant-preview--testing) there. ![Editor for Java](./java_exec.png "Editor for Java") As we picked `Java` for this example, Windmill provided some boilerplate. Let's take a look: ```java //requirements: //com.google.code.gson:gson:2.8.9 //com.github.ricksbrown:cowsay:1.1.0 public class Main { public static class Person { private String name; private int age; // Constructor public Person(String name, int age) { this.name = name; this.age = age; } } public static Object main( // Primitive int a, float b, // Objects Integer age, Float d, Object e, String name, // Lists String[] f // No trailing commas! ){ Gson gson = new Gson(); // Get resources var theme = Wmill.getResource("f/app_themes/theme_0"); System.out.println("Theme: " + theme); // Create a Person object Person person = new Person( (name == "") ? "Alice" : name, (age == null) ? 30 : age); // Serialize the Person object to JSON String json = gson.toJson(person); System.out.println("Serialized JSON: " + json); // Use cowsay String[] args = new String[]{"-f", "dragon", json }; String result = Cowsay.say(args); return result; } } ``` In Java you need `Main` public class and public static `main` function. Return type can either be an `Object` or `void`. Any primitive java type can be automatically converted to `Object`. There are a few important things to note about the `Main`. - The arguments are used for generating 1. the [input spec](../../../core_concepts/13_json_schema_and_parsing/index.mdx) of the Script 2. the [frontend](../../../core_concepts/6_auto_generated_uis/index.mdx) that you see when running the Script as a standalone app. - Type annotations are used to generate the UI form, and help pre-validate inputs. While not mandatory, they are highly recommended. You can customize the UI in later steps (but not change the input type!). Packages can be installed through [Coursier](https://get-coursier.io/). Just add the dependencies you need at the top of the file, one per line in the format `groupId:artifactId:version`: ```java //requirements: //com.google.code.gson:gson:2.8.9 //com.github.ricksbrown:cowsay:1.1.0 ``` It supports [Maven](https://maven.apache.org/what-is-maven.html) and [Ivy](https://ant.apache.org/ivy/) repositories. ### Private Maven registries On [Enterprise Edition](/pricing), you can configure private Maven repositories from [Instance settings](../../../advanced/18_instance_settings/index.mdx#registries) -> Registries -> Maven `settings.xml`. Provide the full content of a Maven `settings.xml` file. Windmill writes this file to the Java home `.m2/settings.xml` directory, allowing Coursier to use your configured servers, mirrors, and repository URLs when resolving dependencies. If the setting is cleared, the `settings.xml` file is removed. Example `settings.xml` content: ```xml my-private-repo deploy-user my-secret-token private my-private-repo https://maven.example.com/releases private ``` ## Instant preview & testing Look at the UI preview on the right: it was updated to match the input signature. Run a test (`Ctrl` + `Enter`) to verify everything works. ## Generated UI From the Settings menu, the "Generated UI" tab lets you customize the script's arguments. The UI is generated from the Script's main function signature, but you can add additional constraints here. For example, we could use the `Customize property`: add a regex by clicking on `Pattern` to make sure users are providing a name with only alphanumeric characters: `^[A-Za-z0-9]+$`. Let's still allow numbers in case you are some tech billionaire's kid. ![Advanced settings for Java](./ui_java.png "Advanced settings for Java") ## Run! We're done! Now let's look at what users of the script will do. Click on the [Deploy](../../../core_concepts/0_draft_and_deploy/index.mdx) button to load the script. You'll see the user input form we defined earlier. Note that Scripts are [versioned](../../../core_concepts/34_versioning/index.mdx#script-versioning) in Windmill, and each script version is uniquely identified by a hash. Fill in the input field, then hit "Run". You should see a run view, as well as your logs. All script runs are also available in the [Runs](../../../core_concepts/5_monitor_past_and_future_runs/index.mdx) menu on the left. ![Run in Java](./run_java.png "Run in Java") You can also choose to [run the script from the CLI](../../../advanced/3_cli/index.mdx) with the pre-made Command-line interface call. ## Caching Every binary and dependency on Java is cached on disk by default. Furthermore if you use the [Distributed cache storage](../../../core_concepts/38_object_storage_in_windmill/index.mdx#instance-object-storage-distributed-cache-for-python-rust-go), it will be available to every other worker, allowing fast startup for every worker. ## What's next? This script is a minimal working example, but there's a few more steps that can be useful in a real-world use case: - Pass [variables and secrets](../../../core_concepts/2_variables_and_secrets/index.mdx) to a script. - Connect to [resources](../../../core_concepts/3_resources_and_types/index.mdx). - [Trigger that script](../../../triggers/index.mdx) in many ways. - Compose scripts in [Flows](../../../flows/1_flow_editor.mdx), [low-code apps](../../../apps/0_app_editor/index.mdx) or [full-code apps](../../../full_code_apps/index.mdx). - You can [share your scripts](../../../misc/1_share_on_hub/index.md) with the community on [Windmill Hub](https://hub.windmill.dev). Once submitted, they will be verified by moderators before becoming available to everyone right within Windmill. Scripts are immutable and there is an hash for each deployment of a given script. Scripts are never overwritten and referring to a script by path is referring to the latest deployed hash at that path. For each script, a UI is autogenerated from the JSON schema inferred from the script signature, and can be customized further as standalone or embedded into rich UIs using the [App builder](../../7_apps_quickstart/index.mdx). In addition to the UI, sync and async [webhooks](../../../core_concepts/4_webhooks/index.mdx) are generated for each deployment. --- ## PHP quickstart Source: https://www.windmill.dev/docs/getting_started/scripts_quickstart/php # PHP quickstart In this quick start guide, we will write our first script in php. This tutorial covers how to create a simple script through Windmill web IDE. See the dedicated page to [develop scripts locally](../../../advanced/4_local_development/index.mdx). Scripts are the basic building blocks in Windmill. They can be [run and scheduled](../../../triggers/index.mdx) as standalone, chained together to create [Flows](../../../flows/1_flow_editor.mdx) or displayed with a personalized User Interface as [Apps](../../7_apps_quickstart/index.mdx). Scripts consist of 2 parts: - [Code](#code): for php scripts, it must have at least a main function. - [Settings](#settings): settings & metadata about the Script such as its path, summary, description, [JSON Schema](../../../core_concepts/13_json_schema_and_parsing/index.mdx) of its inputs (inferred from its signature). When stored in a code repository, these 2 parts are stored separately at `.php` and `.script.yaml` Windmill automatically manages [dependencies](../../../advanced/6_imports/index.mdx) for you. When you import libraries in your php script, Windmill parses these imports upon saving the script and automatically generates a list of dependencies. It then spawns a dependency job to associate these Composer packages with a lockfile, ensuring that the same version of the script is always executed with the same versions of its dependencies. This is a simple example of a script built in php with Windmill: ``` .py` and `.script.yaml` Windmill automatically manages [dependencies](../../../advanced/15_dependencies_in_python/index.mdx) for you. When you import libraries in your Python script, Windmill parses these top-level imports upon saving the script and automatically generates a list of dependencies. For automatic dependency installation, Windmill will only consider these top-level imports. It then spawns a dependency job to associate these PyPI packages with a lockfile, ensuring that the same version of the script is always executed with the same versions of its dependencies [This](https://hub.windmill.dev/scripts/%22%22/1530/do-sentiment-analysis-with-nltk-%22%22) is a simple example of a script built in Python with Windmill: ```py #import wmill import nltk from nltk.sentiment import SentimentIntensityAnalyzer nltk.download("vader_lexicon") def main(text: str = "Wow, NLTK is really powerful!"): return SentimentIntensityAnalyzer().polarity_scores(text) ``` In this quick start guide, we'll create a script that greets the operator running it. From the Home page, click **New** and select **Script**. This will take you to the first step of script creation: [Metadata](../../../script_editor/settings.mdx#metadata). ## Settings ![New script](../../../../static/images/script_languages.png "New script") As part of the [settings](../../../script_editor/settings.mdx) menu, each script has metadata associated with it, enabling it to be defined and configured in depth. - **Summary** (optional) is a short, human-readable summary of the Script. It will be displayed as a title across Windmill. If omitted, the UI will use the `path` by default. - **Path** is the Script's unique identifier that consists of the [script's owner](../../../core_concepts/16_roles_and_permissions/index.mdx), and the script's name. The owner can be either a user, or a group ([folder](../../../core_concepts/8_groups_and_folders/index.mdx#folders)). - **Description** is where you can give instructions through the [auto-generated UI](../../../core_concepts/6_auto_generated_uis/index.mdx) to users on how to run your Script. It supports markdown. - **Language** of the script. - **Script kind**: Action (by default), [Trigger](../../../flows/10_flow_trigger.mdx), [Approval](../../../flows/11_flow_approval.mdx), [Error handler](../../../flows/7_flow_error_handler.md) or [Preprocessor](../../../core_concepts/43_preprocessors/index.mdx). This acts as a tag to filter appropriate scripts from the [flow editor](../../6_flows_quickstart/index.mdx). This menu also has additional settings on [Runtime](../../../script_editor/settings.mdx#runtime), [Generated UI](#generated-ui) and [Triggers](../../../script_editor/settings.mdx#triggers). Now click on the code editor on the left side, and let's build our Hello World! ## Code Windmill provides an online editor to work on your Scripts. The left-side is the editor itself. The right-side [previews the UI](../../../core_concepts/6_auto_generated_uis/index.mdx) that Windmill will generate from the Script's signature - this will be visible to the users of the Script. You can preview that UI, provide input values, and [test your script](#instant-preview--testing) there. ![Editor for python](./editor_python.png.webp) As we picked `python` for this example, Windmill provided some python boilerplate. Let's take a look: ```python import os import wmill # You can import any PyPI package. # See here for more info: https://www.windmill.dev/docs/advanced/dependencies_in_python # you can use typed resources by doing a type alias to dict #postgresql = dict def main( no_default: str, #db: postgresql, name="Nicolas Bourbaki", age=42, obj: dict = {"even": "dicts"}, l: list = ["or", "lists!"], file_: bytes = bytes(0), ): print(f"Hello World and a warm welcome especially to {name}") print("and its acolytes..", age, obj, l, len(file_)) # retrieve variables, resources, states using the wmill client try: secret = wmill.get_variable("f/examples/secret") except: secret = "No secret yet at f/examples/secret !" print(f"The variable at `f/examples/secret`: {secret}") # Get last state of this script execution by the same trigger/user last_state = wmill.get_state() new_state = {"foo": 42} if last_state is None else last_state new_state["foo"] += 1 wmill.set_state(new_state) # fetch context variables user = os.environ.get("WM_USERNAME") # return value is converted to JSON return {"split": name.split(), "user": user, "state": new_state} ``` In Windmill, scripts need to have a `main` function that will be the script's entrypoint. There are a few important things to note about the `main`. - The main arguments are used for generating 1. the [input spec](../../../core_concepts/13_json_schema_and_parsing/index.mdx) of the Script 2. the [frontend](../../../core_concepts/6_auto_generated_uis/index.mdx) that you see when running the Script as a standalone app. - Type annotations are used to generate the UI form, and help pre-validate inputs. While not mandatory, they are highly recommended. You can customize the UI in later steps (but not change the input type!). - Windmill supports [Pydantic `BaseModel`](https://docs.pydantic.dev/) and [`@dataclass`](https://docs.python.org/3/library/dataclasses.html) classes as parameter types. When a parameter is typed with a Pydantic model or dataclass, Windmill infers the JSON schema from the class fields and generates a structured input form. Nested models, `Optional`, `List`, and `Dict` types are supported. ### Pydantic and dataclass support You can use Pydantic `BaseModel` or Python `@dataclass` as parameter types for your main function. Windmill will parse the class definition and generate a structured input form with proper field types. ```python from pydantic import BaseModel from typing import Optional, List class Address(BaseModel): street: str city: str zip_code: Optional[str] = None class User(BaseModel): name: str age: int addresses: List[Address] = [] def main(user: User): return f"Hello {user.name}, age {user.age}" ``` Dataclasses work the same way: ```python from dataclasses import dataclass @dataclass class Config: host: str port: int = 8080 debug: bool = False def main(config: Config): return f"Connecting to {config.host}:{config.port}" ``` The last import line imports the [Windmill client](../../../advanced/2_clients/python_client.md), which is needed for example to access [variables](../../../core_concepts/2_variables_and_secrets/index.mdx) or [resources](../../../core_concepts/3_resources_and_types/index.mdx). Back to our Hello World. We can clean up unused import statements, change the main to take in the user's name. Let's also return the `name`, maybe we can use this later if we use this Script within a [flow](../../../flows/1_flow_editor.mdx) or [app](../../../full_code_apps/index.mdx) and need to pass its result on. ```py def main(name: str): print("Hello world. Oh, it's you {}? Greetings!".format(name)) return name ``` ## Instant preview & testing Look at the UI preview on the right: it was updated to match the input signature. Run a test (`Ctrl` + `Enter`) to verify everything works. You can change how the UI behaves by changing the main signature. For example, if you add a default for the `name` argument, the UI won't consider this field as required anymore. ```py def main(name: str = "you"): ``` Now let's go to the last step: the "Generated UI" settings. ## Generated UI From the Settings menu, the "Generated UI" tab lets you customize the script's arguments. The UI is generated from the Script's main function signature, but you can add additional constraints here. For example, we could use the `Customize property`: add a regex by clicking on `Pattern` to make sure users are providing a name with only alphanumeric characters: `^[A-Za-z0-9]+$`. Let's still allow numbers in case you are some tech billionaire's kid. ![Advanced settings for Python](./customize_python.png.webp) ## Workflows as code One way to write distributed programs that execute distinct jobs is to use [flows](../../../flows/1_flow_editor.mdx) that chain scripts together. Another approach is to write a program that defines the jobs and their dependencies, and then execute that program directly in your script. This is known as [workflows as code](../../../core_concepts/31_workflows_as_code/index.mdx). Use the `@workflow` decorator on your orchestration function and `@task` on task functions. Each task runs as a separate job with its own logs and timeline entry, while the workflow suspends between tasks (releasing its worker slot). ![Flow as code in Python](../../../core_concepts/31_workflows_as_code/wac-editor-1.png "Flow as code in Python") All details at: ## Run! We're done! Now let's look at what users of the script will do. Click on the [Deploy](../../../core_concepts/0_draft_and_deploy/index.mdx) button to load the script. You'll see the user input form we defined earlier. Note that Scripts are [versioned](../../../core_concepts/34_versioning/index.mdx#script-versioning) in Windmill, and each script version is uniquely identified by a hash. Fill in the input field, then hit "Run". You should see a run view, as well as your logs. All script runs are also available in the [Runs](../../../core_concepts/5_monitor_past_and_future_runs/index.mdx) menu on the left. ![Run hello world in Python](./run_python.png.webp) You can also choose to [run the script from the CLI](../../../advanced/3_cli/index.mdx) with the pre-made Command-line interface call. ## Select Python version You can annotate the version of Python you would like to use for a script using the following annotations: py310, py311, py312, or py313: ```python # py312 type Foo = str def main(): foo: Foo = "Foo" return foo ``` ### Python version specifiers You can also use more advanced version specifiers with the shortcut format to specify exact version requirements: ```python # py: >=3.12 def main(): return "Hello from Python 3.12+" ``` ```python # py: ==3.12.* def main(): return "Hello from any Python 3.12 version" ``` ```python # py: >=3.11,<3.14 def main(): return "Hello from Python 3.11 to 3.13" ``` These version specifiers use [PEP 440](https://peps.python.org/pep-0440/) syntax and provide more precise control over Python version requirements than the simple annotations. This is especially useful when you need to ensure compatibility with specific Python features or avoid known issues in certain versions. Alternatively, you can set a global version by configuring the INSTANCE_PYTHON_VERSION [environment variable](../../../core_concepts/47_environment_variables/index.mdx) to one of the mentioned versions or unset it to use "Latest Stable". If you leave `INSTANCE_PYTHON_VERSION` empty it will inherit "Latest Stable" version, which depends on Windmill. For newly deployed scripts, the annotated or instance version will be assigned to the lockfile, and all future executions will adhere to that specified version. For scripts that are already deployed and have no version specified in lockfile, Python 3.11 will be used by default, even if the instance version is changed to a different one. During test runs or deployments, if there are imported scripts, Windmill will search through all of them to find an annotated version, which will be used as the final version. If no annotated version is found, the instance version will be used instead. For [Enterprise Edition](/pricing) (EE) customers, [S3 cache](../../../core_concepts/38_object_storage_in_windmill/index.mdx#instance-object-storage-distributed-cache-for-python-rust-go) tarballs will be organized and separated by Python version. ## Caching Every dependency on Python is cached on disk by default. Furthermore if you use the [Distributed cache storage](../../../core_concepts/38_object_storage_in_windmill/index.mdx#instance-object-storage-distributed-cache-for-python-rust-go), it will be available to every other worker, allowing fast startup for every worker. ## What's next? This script is a minimal working example, but there's a few more steps that can be useful in a real-world use case: - Pass [variables and secrets](../../../core_concepts/2_variables_and_secrets/index.mdx) to a script. - Connect to [resources](../../../core_concepts/3_resources_and_types/index.mdx). - [Trigger that script](../../../triggers/index.mdx) in many ways. - Compose scripts in [Flows](../../../flows/1_flow_editor.mdx), [low-code apps](../../../apps/0_app_editor/index.mdx) or [full-code apps](../../../full_code_apps/index.mdx). - You can [share your scripts](../../../misc/1_share_on_hub/index.md) with the community on [Windmill Hub](https://hub.windmill.dev). Once submitted, they will be verified by moderators before becoming available to everyone right within Windmill. Scripts are immutable and there is an hash for each deployment of a given script. Scripts are never overwritten and referring to a script by path is referring to the latest deployed hash at that path. For each script, a UI is autogenerated from the JSON schema inferred from the script signature, and can be customized further as standalone or embedded into rich UIs using the [App builder](../../7_apps_quickstart/index.mdx). In addition to the UI, sync and async [webhooks](../../../core_concepts/4_webhooks/index.mdx) are generated for each deployment. --- ## R quickstart Source: https://www.windmill.dev/docs/getting_started/scripts_quickstart/r # R quickstart In this quick start guide, we will write our first script in [R](https://www.r-project.org/). {/* Placeholder: Add demo video for R scripts when available */} {/* */} Scripts are the basic building blocks in Windmill. They can be [run and scheduled](../../../triggers/index.mdx) as standalone, chained together to create [Flows](../../../flows/1_flow_editor.mdx) or displayed with a personalized User Interface as [Apps](../../7_apps_quickstart/index.mdx). Scripts consist of 2 parts: - [Code](#code): for R scripts, they must have a `main` function defined as `main <- function(...)`. - [Settings](#settings): settings & metadata about the Script such as its path, summary, description, [jsonschema](../../../core_concepts/13_json_schema_and_parsing/index.mdx) of its inputs (inferred from its signature). When stored in a code repository, these 2 parts are stored separately at `.r` and `.script.yaml` Windmill automatically manages [dependencies](/docs/getting_started/scripts_quickstart/r#dependencies-management) for you. When you use packages in your R script through `library()` or `require()` calls, Windmill parses these dependencies upon saving the script and automatically resolves versions from CRAN, ensuring that the same version of the script is always executed with the same versions of its dependencies. This is a simple example of a script built in R with Windmill: ```r library(httr) library(jsonlite) main <- function(url = "https://httpbin.org/get", message = "Hello from Windmill!") { response <- GET(url, query = list(message = message)) list( status = status_code(response), body = content(response, as = "parsed"), message = "Request completed successfully" ) } ``` In this quick start guide, we'll create a script that greets the operator running it. From the Home page, click **New** and select **Script**. This will take you to the first step of script creation: Metadata. ## Settings ![R Settings](./r-settings.png "R Settings") As part of the [settings](../../../script_editor/settings.mdx) menu, each script has metadata associated with it, enabling it to be defined and configured in depth. - **Path** is the Script's unique identifier that consists of the [script's owner](../../../core_concepts/16_roles_and_permissions/index.mdx), and the script's name. The owner can be either a user, or a group ([folder](../../../core_concepts/8_groups_and_folders/index.mdx#folders)). - **Summary** (optional) is a short, human-readable summary of the Script. It will be displayed as a title across Windmill. If omitted, the UI will use the `path` by default. - **Language** of the script. - **Description** is where you can give instructions through the [auto-generated UI](../../../core_concepts/6_auto_generated_uis/index.mdx) to users on how to run your Script. It supports markdown. - **Script kind**: Action (by default), [Trigger](../../../flows/10_flow_trigger.mdx), [Approval](../../../flows/11_flow_approval.mdx) or [Error handler](../../../flows/7_flow_error_handler.md). This acts as a tag to filter appropriate scripts from the [flow editor](../../6_flows_quickstart/index.mdx). This menu also has additional settings on [Runtime](../../../script_editor/settings.mdx#runtime), [Generated UI](#generated-ui) and [Triggers](../../../script_editor/settings.mdx#triggers). Now click on the code editor on the left side, and let's build our Hello World! ## Code Windmill provides an online editor to work on your Scripts. The left-side is the editor itself. The right-side [previews the UI](../../../core_concepts/6_auto_generated_uis/index.mdx) that Windmill will generate from the Script's signature - this will be visible to the users of the Script. You can preview that UI, provide input values, and [test your script](#instant-preview--testing) there. ![R Editor](./r-editor.png "R Editor") As we picked `R` for this example, Windmill provided some R boilerplate. Let's take a look: ```r library(crayon) library(dplyr) library(zoo) library(jsonlite) main <- function( x, name = "default", age = 25, data = list(1, 2, 3), flag = TRUE ) { # Use Windmill helpers: # var <- get_variable("f/my_var") # res <- get_resource("f/my_resource") df <- tibble(name = name, age = age, x = x) result <- df %>% mutate(greeting = paste("Hello", name)) return(toJSON(result, auto_unbox = TRUE)) } ``` In Windmill, R scripts must have a `main` function defined as `main <- function(...)` that will be the script's entrypoint. There are a few important things to note about the `main` function: - The main arguments are used for generating 1. the [input spec](../../../core_concepts/13_json_schema_and_parsing/index.mdx) of the Script 2. the [frontend](../../../core_concepts/6_auto_generated_uis/index.mdx) that you see when running the Script as a standalone app. - Default values are used to infer argument types and generate the UI form. String defaults create string inputs, numeric defaults create number inputs, list/vector defaults create appropriate JSON inputs, etc. - You can customize the UI in later steps (but not change the input type!). Back to our Hello World. We can clean up the boilerplate, change the main to take in the user's name. Let's also return the `name`, maybe we can use this later if we use this Script within a [flow](../../../flows/1_flow_editor.mdx) or [app](../../../full_code_apps/index.mdx) and need to pass its result on. ```r main <- function(name = "World") { print(paste("Hello", name, "! Greetings from R!")) name } ``` ## Accessing variables and resources R scripts can access Windmill [variables](../../../core_concepts/2_variables_and_secrets/index.mdx) and [resources](../../../core_concepts/3_resources_and_types/index.mdx) using built-in helper functions: ```r library(httr) library(jsonlite) main <- function() { # Get a variable secret <- get_variable("f/examples/secret") # Get a resource (returns a list/object) db_config <- get_resource("f/examples/postgres") # Access context variables from environment user <- Sys.getenv("WM_USERNAME") workspace <- Sys.getenv("WM_WORKSPACE") list( secret = secret, db_host = db_config$host, user = user, workspace = workspace ) } ``` ## Instant preview & testing Look at the UI preview on the right: it was updated to match the input signature. Run a test (`Ctrl` + `Enter`) to verify everything works. You can change how the UI behaves by changing the main signature. For example, if you remove the default for the `name` argument, the UI will consider this field as required. ```r main <- function(name) ``` Now let's go to the last step: the "Generated UI" settings. ## Generated UI From the Settings menu, the "Generated UI" tab lets you customize the script's arguments. The UI is generated from the Script's main function signature, but you can add additional constraints here. For example, we could use the `Customize property`: add a regex by clicking on `Pattern` to make sure users are providing a name with only alphanumeric characters: `^[A-Za-z0-9]+$`. Let's still allow numbers in case you are some tech billionaire's kid. ![Generated UI](../14_ruby_quickstart/customize-ui.png "Generated UI") ## Run! We're done! Now let's look at what users of the script will do. Click on the [Deploy](../../../core_concepts/0_draft_and_deploy/index.mdx) button to load the script. You'll see the user input form we defined earlier. Note that Scripts are [versioned](../../../core_concepts/34_versioning/index.mdx#script-versioning) in Windmill, and each script version is uniquely identified by a hash. Fill in the input field, then hit "Run". You should see a run view, as well as your logs. All script runs are also available in the [Runs](../../../core_concepts/5_monitor_past_and_future_runs/index.mdx) menu on the left. You can also choose to [run the script from the CLI](../../../advanced/3_cli/index.mdx) with the pre-made Command-line interface call. ## Dependencies management R dependencies are automatically detected from `library()` and `require()` calls in your script. Windmill resolves package versions from CRAN: ```r library(httr) library(jsonlite) library(dplyr) library(ggplot2) main <- function(data_url = "https://example.com/data.csv") { # Use httr for HTTP requests response <- GET(data_url) # Parse JSON responses data <- fromJSON(content(response, as = "text")) # Use dplyr for data manipulation result <- data %>% filter(!is.na(value)) %>% summarise(mean_value = mean(value)) result } ``` Windmill will automatically: - Parse your `library()` and `require()` calls when you save the script - Resolve versions from CRAN (with 3-day TTL caching for lockfiles) - Install packages to a shared cache directory - Cache dependencies for faster execution ### Verbose mode By default, renv output is suppressed during package installation. To enable verbose output for debugging, add a `#verbose` annotation at the top of your script: ```r #verbose library(httr) library(jsonlite) main <- function() { # Your code here } ``` ## Caching Every R package dependency is cached on disk by default. Furthermore if you use the [Distributed cache storage](../../../core_concepts/38_object_storage_in_windmill/index.mdx#instance-object-storage-distributed-cache-for-python-rust-go), it will be available to every other worker, allowing fast startup for every worker. ## What's next? This script is a minimal working example, but there's a few more steps that can be useful in a real-world use case: - Pass [variables and secrets](../../../core_concepts/2_variables_and_secrets/index.mdx) to a script. - Connect to [resources](../../../core_concepts/3_resources_and_types/index.mdx). - [Trigger that script](../../../triggers/index.mdx) in many ways. - Compose scripts in [Flows](../../../flows/1_flow_editor.mdx), [low-code apps](../../../apps/0_app_editor/index.mdx) or [full-code apps](../../../full_code_apps/index.mdx). - You can [share your scripts](../../../misc/1_share_on_hub/index.md) with the community on [Windmill Hub](https://hub.windmill.dev). Once submitted, they will be verified by moderators before becoming available to everyone right within Windmill. Scripts are immutable and there is a hash for each deployment of a given script. Scripts are never overwritten and referring to a script by path is referring to the latest deployed hash at that path. For each script, a UI is autogenerated from the jsonschema inferred from the script signature, and can be customized further as standalone or embedded into rich UIs using the [App builder](../../7_apps_quickstart/index.mdx). In addition to the UI, sync and async [webhooks](../../../core_concepts/4_webhooks/index.mdx) are generated for each deployment. --- ## Rest / GraphQL quickstart Source: https://www.windmill.dev/docs/getting_started/scripts_quickstart/rest_graphql # Rest / GraphQL quickstart In this quick start guide, we will write our first script in [Rest](https://restfulapi.net/) & [GraphQL](https://graphql.org/). This tutorial covers how to create a simple script through Windmill web IDE. See the dedicated page to [develop scripts locally](../../../advanced/4_local_development/index.mdx). Scripts are the basic building blocks in Windmill. They can be [run and scheduled](../../../triggers/index.mdx) as standalone, chained together to create [Flows](../../../flows/1_flow_editor.mdx) or displayed with a personalized User Interface as [Apps](../../7_apps_quickstart/index.mdx). Scripts consist of 2 parts: - [Code](#code). - [Settings](#settings): settings & metadata about the Script such as its path, summary, description, [JSON Schema](../../../core_concepts/13_json_schema_and_parsing/index.mdx) of its inputs (inferred from its signature). When stored in a code repository, those 2 parts are stored separately at `.rest` and `.script.yaml`. Below is a simple example of a script built in Rest with Windmill: ```ts export async function main() { const res = await fetch('https://api.supabase.com/v1/organizations', { headers: { Authorization: `Bearer `, 'Content-Type': 'application/json' } }); return res.json(); } ``` ## Settings ![New script](../../../../static/images/script_languages.png "New script") As part of the [settings](../../../script_editor/settings.mdx) menu, each script has metadata associated with it, enabling it to be defined and configured in depth. - **Summary** (optional) is a short, human-readable summary of the Script. It will be displayed as a title across Windmill. If omitted, the UI will use the `path` by default. - **Path** is the Script's unique identifier that consists of the [script's owner](../../../core_concepts/16_roles_and_permissions/index.mdx), and the script's name. The owner can be either a user, or a group ([folder](../../../core_concepts/8_groups_and_folders/index.mdx#folders)). - **Description** is where you can give instructions through the [auto-generated UI](../../../core_concepts/6_auto_generated_uis/index.mdx) to users on how to run your Script. It supports markdown. - **Language** of the script. - **Script kind**: Action (by default), [Trigger](../../../flows/10_flow_trigger.mdx), [Approval](../../../flows/11_flow_approval.mdx), [Error handler](../../../flows/7_flow_error_handler.md) or [Preprocessor](../../../core_concepts/43_preprocessors/index.mdx). This acts as a tag to filter appropriate scripts from the [flow editor](../../6_flows_quickstart/index.mdx). This menu also has additional settings on [Runtime](../../../script_editor/settings.mdx#runtime), [Generated UI](#generated-ui) and [Triggers](../../../script_editor/settings.mdx#triggers). Now click on the code editor on the left side. ## Code Windmill provides an online editor to work on your Scripts. The left-side is the editor itself. The right-side [previews the UI](../../../core_concepts/6_auto_generated_uis/index.mdx) that Windmill will generate from the Script's signature - this will be visible to the users of the Script. You can preview that UI, provide input values. ![Editor for GraphQL](./editor_graphql.png 'Code editor GraphQL') ### Rest As we picked `Rest` for this example, Windmill provided some boilerplate. Let's take a look: ```ts //native //you can add proxy support using //proxy http(s)://host:port // native scripts are bun scripts that are executed on native workers and can be parallelized // only fetch is allowed, but imports will work as long as they also use only fetch and the standard lib //import * as wmill from "windmill-client" export async function main(example_input: number = 3) { // "3" is the default value of example_input, it can be overridden with code or using the UI const res = await fetch(`https://jsonplaceholder.typicode.com/todos/${example_input}`, { headers: { "Content-Type": "application/json" }, }); return res.json(); } ``` Rest scripts are in fact [Bun TypeScript](../1_typescript_quickstart/index.mdx) fetches. They support all the normal signatures of normal TypeScript but only [stdlib](https://en.wikibooks.org/wiki/C_Programming/stdlib.h) JavaScript, and the fetch operations (including fetch operations from npm packages and relative imports). For example, the full [wmill API](../../../advanced/2_clients/ts_client.mdx) is supported, just use: ```ts import * as wmill from './windmill.ts' ``` The `// native` header line will help Windmill automatically convert between 'nativets' and 'bun' scripts based on the presence of this header so you can always just pick TypeScript (Bun) and decide at the end if you want to accelerate it with 'native' if possible. The REST button simply prefills a Bun script with a `//native` header. Fetches can also be done through a regular TypeScript in Windmill (without the `//native` header), but opting for dedicated Rest scripts benefits from a highly efficient runtime. Replace the `` URL with the API endpoint of your choice and customize the headers object according to your fetch requirements. REST scripts benefit from the [auto-generated UI](../../../core_concepts/6_auto_generated_uis/index.mdx) capabilities of any Windmill script. The arguments of the `main` function are used for generating 1. the input spec of the Script, and 2. the frontend that you see when running the Script as a standalone app. Type annotations are used to generate the UI form, and help pre-validate inputs. While not mandatory, they are highly recommended. You can customize the UI in later steps (but not change the input type!). This is also a way to have users fill [resources](../../../core_concepts/3_resources_and_types/index.mdx) or [variables](../../../core_concepts/2_variables_and_secrets/index.mdx) through the auto-generated UI. The UI is available in the script editor to [test your code](../../../core_concepts/23_instant_preview/index.mdx): > In the example above, a custom resource is declared as a parameter of the main function. It will be asked by the user through the auto-generated UI and used directly in the script (here for the bearer token). If the endpoint is and HTTPS endpoint exposing custom certificate, the fetch will fail. Custom certificates can be trusted using the `DENO_CERT` env variable (see [Deno official documentation](https://docs.deno.com/runtime/manual/getting_started/setup_your_environment#environment-variables)) ### GraphQL As we picked `GraphQL` for this example, Windmill provided some boilerplate. Let's take a look: ```ts query($name1: String, $name2: Int, $name3: [String]) { demo(example_name_1: $name1, example_name_2: $name2, example_name_3: $name3) { example_name_1, example_name_2, example_name_3 } } ``` The query itself is similar to any GraphQL query. To trigger each GraphQL script, an input named 'api' will be required. This is a [GraphQL resource](https://hub.windmill.dev/resource_types/112/graphql) defined by the JSON schema: ```js { "type": "object", "$schema": "https://json-schema.org/draft/2020-12/schema", "required": [ "base_url" ], "properties": { "base_url": { "type": "string", "format": "uri", "default": "", "description": "" }, "bearer_token": { "type": "string", "default": "", "description": "" }, "custom_headers": { "type": "object", "description": "", "properties": {}, "required": [] } } } ``` [Resources](../../../core_concepts/3_resources_and_types/index.mdx) are rich objects in JSON that allow to store configuration and credentials. They can be saved, named, and shared within Windmill to control and streamline the execution of GraphQL scripts. The arguments of the query will be used for generating 1. the input spec of the Script, and 2. the frontend that you see when running the Script with an [auto-generated UI](../../../core_concepts/6_auto_generated_uis/index.mdx). Type annotations are used to generate the UI form, and help pre-validate inputs. While not mandatory, they are highly recommended. You can customize the UI in later steps (but not change the input type). The UI is available in the script editor to [test your code](../../../core_concepts/23_instant_preview/index.mdx): > In the example above, a [GraphQL resource](https://hub.windmill.dev/resource_types/112/graphql) is used with details on base URL and bearer token. Also was used the [auto-generated UI](../../../core_concepts/6_auto_generated_uis/index.mdx) to fill the argument `login`. ## Generated UI From the Settings menu, the "Generated UI" tab lets you customize the script's arguments. The UI is generated from the Script's main function signature, but you can add additional constraints here. For example, we could use the `Customize property`: add a regex by clicking on `Pattern` to make sure users are providing a name with only alphanumeric characters: `^[A-Za-z0-9]+$`. Let's still allow numbers in case you are some tech billionaire's kid. ![Advanced settings for TypeScript](./customize_graphql.png.webp) ## Run! We're done! Now let's look at what users of the script will do. Click on the [Deploy](../../../core_concepts/0_draft_and_deploy/index.mdx) button to load the script. You'll see the user input form we defined earlier. Note that Scripts are [versioned](../../../core_concepts/34_versioning/index.mdx#script-versioning) in Windmill, and each script version is uniquely identified by a hash. Fill in the input field, then hit "Run". You should see a run view, as well as your logs. All script runs are also available in the [Runs](../../../core_concepts/5_monitor_past_and_future_runs/index.mdx) menu on the left. ![Run hello world in GraphQL](./run_graphql.png.webp) You can also choose to [run the script from the CLI](../../../advanced/3_cli/index.mdx) with the pre-made Command-line interface call. ## What's next? This script is a minimal working example, but there's a few more steps that can be useful in a real-world use case: - Pass [variables and secrets](../../../core_concepts/2_variables_and_secrets/index.mdx) to a script. - Connect to [resources](../../../core_concepts/3_resources_and_types/index.mdx). - [Trigger that script](../../../triggers/index.mdx) in many ways. - Compose scripts in [Flows](../../../flows/1_flow_editor.mdx), [low-code apps](../../../apps/0_app_editor/index.mdx) or [full-code apps](../../../full_code_apps/index.mdx). - You can [share your scripts](../../../misc/1_share_on_hub/index.md) with the community on [Windmill Hub](https://hub.windmill.dev). Once submitted, they will be verified by moderators before becoming available to everyone right within Windmill. Scripts are immutable and there is an hash for each deployment of a given script. Scripts are never overwritten and referring to a script by path is referring to the latest deployed hash at that path. For each script, a UI is autogenerated from the JSON schema inferred from the script signature, and can be customized further as standalone or embedded into rich UIs using the [App builder](../../7_apps_quickstart/index.mdx). In addition to the UI, sync and async [webhooks](../../../core_concepts/4_webhooks/index.mdx) are generated for each deployment. --- ## Ruby quickstart Source: https://www.windmill.dev/docs/getting_started/scripts_quickstart/ruby # Ruby quickstart In this quick start guide, we will write our first script in [Ruby](https://www.ruby-lang.org/). Scripts are the basic building blocks in Windmill. They can be [run and scheduled](../../../triggers/index.mdx) as standalone, chained together to create [Flows](../../../flows/1_flow_editor.mdx) or displayed with a personalized User Interface as [Apps](../../7_apps_quickstart/index.mdx). Scripts consist of 2 parts: - [Code](#code): for Ruby scripts, they can optionally have a main function. Scripts without a main function will execute the entire file. - [Settings](#settings): settings & metadata about the Script such as its path, summary, description, [jsonschema](../../../core_concepts/13_json_schema_and_parsing/index.mdx) of its inputs (inferred from its signature). When stored in a code repository, these 2 parts are stored separately at `.rb` and `.script.yaml` Windmill automatically manages [dependencies](/docs/getting_started/scripts_quickstart/ruby#dependencies-management) for you. When you use gems in your Ruby script through `gemfile` blocks (compatible with bundler/inline syntax), Windmill parses these dependencies upon saving the script and automatically generates a Gemfile.lock, ensuring that the same version of the script is always executed with the same versions of its dependencies. More to it, to remove vendor lock-in barrier you have ability to extract the lockfile and use it outside Windmill if you want. This is a simple example of a script built in Ruby with Windmill: ```ruby require 'windmill/inline' gemfile do source 'https://rubygems.org' gem 'httparty' gem 'json' end def main(url: "https://httpbin.org/get", message: "Hello from Windmill!") response = HTTParty.get(url, query: { message: message }) return { status: response.code, body: JSON.parse(response.body), message: "Request completed successfully" } end ``` In this quick start guide, we'll create a script that greets the operator running it. From the Home page, click **New** and select **Script**. This will take you to the first step of script creation: Metadata. ## Settings ![Ruby Settings](./ruby-settings.png "Ruby Settings") As part of the [settings](../../../script_editor/settings.mdx) menu, each script has metadata associated with it, enabling it to be defined and configured in depth. - **Path** is the Script's unique identifier that consists of the [script's owner](../../../core_concepts/16_roles_and_permissions/index.mdx), and the script's name. The owner can be either a user, or a group ([folder](../../../core_concepts/8_groups_and_folders/index.mdx#folders)). - **Summary** (optional) is a short, human-readable summary of the Script. It will be displayed as a title across Windmill. If omitted, the UI will use the `path` by default. - **Language** of the script. - **Description** is where you can give instructions through the [auto-generated UI](../../../core_concepts/6_auto_generated_uis/index.mdx) to users on how to run your Script. It supports markdown. - **Script kind**: Action (by default), [Trigger](../../../flows/10_flow_trigger.mdx), [Approval](../../../flows/11_flow_approval.mdx) or [Error handler](../../../flows/7_flow_error_handler.md). This acts as a tag to filter appropriate scripts from the [flow editor](../../6_flows_quickstart/index.mdx). This menu also has additional settings on [Runtime](../../../script_editor/settings.mdx#runtime), [Generated UI](#generated-ui) and [Triggers](../../../script_editor/settings.mdx#triggers). Now click on the code editor on the left side, and let's build our Hello World! ## Code Windmill provides an online editor to work on your Scripts. The left-side is the editor itself. The right-side [previews the UI](../../../core_concepts/6_auto_generated_uis/index.mdx) that Windmill will generate from the Script's signature - this will be visible to the users of the Script. You can preview that UI, provide input values, and [test your script](#instant-preview--testing) there. ![Ruby Editor](./ruby-startpage.png "Ruby Editor") As we picked `ruby` for this example, Windmill provided some Ruby boilerplate. Let's take a look: ```ruby # Builtin mini windmill client require 'windmill/mini' require 'windmill/inline' # Add your gem dependencies here using gemfile syntax gemfile do source 'https://rubygems.org' gem 'httparty', '~> 0.21' gem 'json', '~> 2.6' end # You can import any gem from RubyGems. # See here for more info: https://www.windmill.dev/docs/getting_started/scripts_quickstart/ruby#dependencies-management def main( name = "Nicolas Bourbaki", age = 42, obj = { "even" => "hashes" }, l = ["or", "arrays!"] ) puts "Hello World and a warm welcome especially to #{name}" puts "and its acolytes.. #{age} #{obj} #{l}" # retrieve variables, resources using built-in methods begin # Imported from windmill mini client secret = get_variable("f/examples/secret") rescue => e secret = "No secret yet at f/examples/secret!" end puts "The variable at `f/examples/secret`: #{secret}" # fetch context variables user = ENV['WM_USERNAME'] # return value is converted to JSON return { "split" => name.split, "user" => user, "message" => "Hello from Ruby!" } end ``` In Windmill, scripts can optionally have a `main` function that will be the script's entrypoint. If no main function is defined, the entire script will be executed. There are a few important things to note about the `main` function: - The main arguments are used for generating 1. the [input spec](../../../core_concepts/13_json_schema_and_parsing/index.mdx) of the Script 2. the [frontend](../../../core_concepts/6_auto_generated_uis/index.mdx) that you see when running the Script as a standalone app. - Default values are used to infer argument types and generate the UI form. String defaults create string inputs, numeric defaults create number inputs, hash/array defaults create appropriate JSON inputs, etc. - You can customize the UI in later steps (but not change the input type!). The first import line imports the Windmill Ruby client, which provides access to built-in methods for accessing [variables](../../../core_concepts/2_variables_and_secrets/index.mdx) and [resources](../../../core_concepts/3_resources_and_types/index.mdx). Back to our Hello World. We can clean up the boilerplate, change the main to take in the user's name. Let's also return the `name`, maybe we can use this later if we use this Script within a [flow](../../../flows/1_flow_editor.mdx) or [app](../../../full_code_apps/index.mdx) and need to pass its result on. ```ruby def main(name = "World") puts "Hello #{name}! Greetings from Ruby!" return name end ``` ## Instant preview & testing Look at the UI preview on the right: it was updated to match the input signature. Run a test (`Ctrl` + `Enter`) to verify everything works. You can change how the UI behaves by changing the main signature. For example, if you remove the default for the `name` argument, the UI will consider this field as required. ```ruby def main(name) ``` Now let's go to the last step: the "Generated UI" settings. ## Generated UI From the Settings menu, the "Generated UI" tab lets you customize the script's arguments. The UI is generated from the Script's main function signature, but you can add additional constraints here. For example, we could use the `Customize property`: add a regex by clicking on `Pattern` to make sure users are providing a name with only alphanumeric characters: `^[A-Za-z0-9]+$`. Let's still allow numbers in case you are some tech billionaire's kid. ![Generated UI](./customize-ui.png "Generated UI") ## Run! We're done! Now let's look at what users of the script will do. Click on the [Deploy](../../../core_concepts/0_draft_and_deploy/index.mdx) button to load the script. You'll see the user input form we defined earlier. Note that Scripts are [versioned](../../../core_concepts/34_versioning/index.mdx#script-versioning) in Windmill, and each script version is uniquely identified by a hash. Fill in the input field, then hit "Run". You should see a run view, as well as your logs. All script runs are also available in the [Runs](../../../core_concepts/5_monitor_past_and_future_runs/index.mdx) menu on the left. You can also choose to [run the script from the CLI](../../../advanced/3_cli/index.mdx) with the pre-made Command-line interface call. ## Dependencies management Ruby dependencies are managed using a `gemfile` block that is fully compatible with bundler/inline syntax. The gemfile block must include a single global source: ```ruby require 'windmill/inline' gemfile do source 'https://rubygems.org' gem 'httparty', '~> 0.21' gem 'redis', '>= 4.0' gem 'activerecord', '7.0.0' gem 'pg', require: 'pg' gem 'dotenv', require: false end ``` ### Private gem sources You can use private gem repositories using different syntax options: **Option 1: Per-gem source specification** ```ruby require 'windmill/inline' gemfile do source 'https://rubygems.org' gem 'httparty' gem 'private-gem', source: 'https://gems.example.com' end ``` **Option 2: Source block syntax** ```ruby require 'windmill/inline' gemfile do source 'https://rubygems.org' source 'https://gems.example.com' do gem 'private-gem-1' gem 'private-gem-2' end end ``` For authentication with private sources, specify the source URL without credentials in your script. For [Enterprise Edition](/pricing) users, add the authenticated URL to Ruby repositories in instance settings. Navigate to **Instance Settings > Registries > Ruby Repos** and add: ``` https://admin:123@gems.example.com/ ``` ![Ruby Private repos Instance Settings](./ruby-gems-instance-settings.png "Ruby Private repos Instance Settings") Windmill will automatically match the source URL from your script with the authenticated URL from settings and handle authentication seamlessly. ### Network configuration - **TLS/SSL**: Automatically handled as long as the remote certificate is trusted by the system - **Proxy**: Proxy environment variables are automatically handled during lockfile generation, gem installation, and runtime stages Windmill will automatically: - Parse your gemfile block when you save the script - Generate a Gemfile and Gemfile.lock - Install dependencies in an isolated environment - Cache dependencies for faster execution ## Caching Every gem dependency in Ruby is cached on disk by default. Furthermore if you use the [Distributed cache storage](../../../core_concepts/38_object_storage_in_windmill/index.mdx#instance-object-storage-distributed-cache-for-python-rust-go), it will be available to every other worker, allowing fast startup for every worker. ## What's next? This script is a minimal working example, but there's a few more steps that can be useful in a real-world use case: - Pass [variables and secrets](../../../core_concepts/2_variables_and_secrets/index.mdx) to a script. - Connect to [resources](../../../core_concepts/3_resources_and_types/index.mdx). - [Trigger that script](../../../triggers/index.mdx) in many ways. - Compose scripts in [Flows](../../../flows/1_flow_editor.mdx), [low-code apps](../../../apps/0_app_editor/index.mdx) or [full-code apps](../../../full_code_apps/index.mdx). - You can [share your scripts](../../../misc/1_share_on_hub/index.md) with the community on [Windmill Hub](https://hub.windmill.dev). Once submitted, they will be verified by moderators before becoming available to everyone right within Windmill. Scripts are immutable and there is a hash for each deployment of a given script. Scripts are never overwritten and referring to a script by path is referring to the latest deployed hash at that path. For each script, a UI is autogenerated from the jsonschema inferred from the script signature, and can be customized further as standalone or embedded into rich UIs using the [App builder](../../7_apps_quickstart/index.mdx). In addition to the UI, sync and async [webhooks](../../../core_concepts/4_webhooks/index.mdx) are generated for each deployment. --- ## Rust quickstart Source: https://www.windmill.dev/docs/getting_started/scripts_quickstart/rust # Rust quickstart In this quick start guide, we will write our first script in [Rust](https://www.rust-lang.org/). ![Editor for Rust](./editor_rust.png "Script in Rust") This tutorial covers how to create a simple script through Windmill web IDE. See the dedicated page to [develop scripts locally](../../../advanced/4_local_development/index.mdx). Scripts are the basic building blocks in Windmill. They can be [run and scheduled](../../../triggers/index.mdx) as standalone, chained together to create [Flows](../../../flows/1_flow_editor.mdx) or displayed with a personalized User Interface as [Apps](../../7_apps_quickstart/index.mdx). Scripts consist of 2 parts: - [Code](#code): for Rust scripts, it must have at least a main function. - [Settings](#settings): settings & metadata about the Script such as its path, summary, description, [JSON Schema](../../../core_concepts/13_json_schema_and_parsing/index.mdx) of its inputs (inferred from its signature). When stored in a code repository, these 2 parts are stored separately at `.rs` and `.script.yaml` Windmill automatically manages [dependencies](../../../advanced/6_imports/index.mdx) for you. When you import libraries in your Rust script, Windmill parses these imports upon saving the script and automatically generates a list of dependencies. It then spawns a dependency job to associate these crates with a lockfile, ensuring that the same version of the script is always executed with the same versions of its dependencies. From the Home page, click **New** and select **Script**. This will take you to the first step of script creation: [Metadata](../../../script_editor/settings.mdx#metadata). ## Settings ![New script](../../../../static/images/script_languages.png "New script") As part of the [settings](../../../script_editor/settings.mdx) menu, each script has metadata associated with it, enabling it to be defined and configured in depth. - **Summary** (optional) is a short, human-readable summary of the Script. It will be displayed as a title across Windmill. If omitted, the UI will use the `path` by default. - **Path** is the Script's unique identifier that consists of the [script's owner](../../../core_concepts/16_roles_and_permissions/index.mdx), and the script's name. The owner can be either a user, or a group ([folder](../../../core_concepts/8_groups_and_folders/index.mdx#folders)). - **Description** is where you can give instructions through the [auto-generated UI](../../../core_concepts/6_auto_generated_uis/index.mdx) to users on how to run your Script. It supports markdown. - **Language** of the script. - **Script kind**: Action (by default), [Trigger](../../../flows/10_flow_trigger.mdx), [Approval](../../../flows/11_flow_approval.mdx), [Error handler](../../../flows/7_flow_error_handler.md) or [Preprocessor](../../../core_concepts/43_preprocessors/index.mdx). This acts as a tag to filter appropriate scripts from the [flow editor](../../6_flows_quickstart/index.mdx). This menu also has additional settings on [Runtime](../../../script_editor/settings.mdx#runtime), [Generated UI](#generated-ui) and [Triggers](../../../script_editor/settings.mdx#triggers). Now click on the code editor on the left side. ## Code Windmill provides an online editor to work on your Scripts. The left-side is the editor itself. The right-side [previews the UI](../../../core_concepts/6_auto_generated_uis/index.mdx) that Windmill will generate from the Script's signature - this will be visible to the users of the Script. You can preview that UI, provide input values, and [test your script](#instant-preview--testing) there. ![Editor for Rust](./editor_rust.png "Editor for Rust") As we picked `rust` for this example, Windmill provided some rust boilerplate. Let's take a look: ```rust //! Add dependencies in the following partial Cargo.toml manifest //! //! ```cargo //! [dependencies] //! anyhow = "1.0.86" //! rand = "0.7.2" //! # wmill = "^1.0" # Windmill client SDK for API interactions //! ``` //! //! Note that serde is used by default with the `derive` feature. //! You can still reimport it if you need additional features. use anyhow::anyhow; use rand::seq::SliceRandom; use serde::Serialize; #[derive(Serialize, Debug)] struct Ret { msg: String, number: i8, } fn main(who_to_greet: String, numbers: Vec) -> anyhow::Result { println!( "Person to greet: {} - numbers to choose: {:?}", who_to_greet, numbers ); Ok(Ret { msg: format!("Greetings {}!", who_to_greet), number: *numbers .choose(&mut rand::thread_rng()) .ok_or(anyhow!("There should be some numbers to choose from"))?, }) } ``` In Windmill, scripts need to have a `main` function that will be the script's entrypoint. There are a few important things to note about the `main`. - The main arguments are used for generating 1. the [input spec](../../../core_concepts/13_json_schema_and_parsing/index.mdx) of the Script 2. the [frontend](../../../core_concepts/6_auto_generated_uis/index.mdx) that you see when running the Script as a standalone app. - Type annotations are used to generate the UI form, and help pre-validate inputs. While not mandatory, they are highly recommended. You can customize the UI in later steps (but not change the input type!). Packages can be installed using cargo. Just add the dependencies you need in the partial Cargo.toml manifest and Windmill will install them for you: ```rust //! ```cargo //! [dependencies] //! anyhow = "1.0.86" //! rand = "0.7.2" //! ``` ``` ### Private Cargo registries On [Enterprise Edition](/pricing), you can configure private Cargo registries from [Instance settings](../../../advanced/18_instance_settings/index.mdx#registries) -> Registries -> Cargo registries. Provide the content of a Cargo `config.toml` file — Windmill writes it to `.cargo/config.toml` in the job directory during execution. ## Instant preview & testing Look at the UI preview on the right: it was updated to match the input signature. Run a test (`Ctrl` + `Enter`) to verify everything works. ## Generated UI From the Settings menu, the "Generated UI" tab lets you customize the script's arguments. The UI is generated from the Script's main function signature, but you can add additional constraints here. For example, we could use the `Customize property`: add a regex by clicking on `Pattern` to make sure users are providing a name with only alphanumeric characters: `^[A-Za-z0-9]+$`. Let's still allow numbers in case you are some tech billionaire's kid. ![Advanced settings for rust](./customize_rust.png "Advanced settings for rust") ## Run! We're done! Now let's look at what users of the script will do. Click on the [Deploy](../../../core_concepts/0_draft_and_deploy/index.mdx) button to load the script. You'll see the user input form we defined earlier. Note that Scripts are [versioned](../../../core_concepts/34_versioning/index.mdx#script-versioning) in Windmill, and each script version is uniquely identified by a hash. Fill in the input field, then hit "Run". You should see a run view, as well as your logs. All script runs are also available in the [Runs](../../../core_concepts/5_monitor_past_and_future_runs/index.mdx) menu on the left. ![Run in rust](./run_rust.png "Run in rust") You can also choose to [run the script from the CLI](../../../advanced/3_cli/index.mdx) with the pre-made Command-line interface call. ## Caching Every bundle on Rust is cached on disk by default. Furthermore if you use the [Distributed cache storage](../../../core_concepts/38_object_storage_in_windmill/index.mdx#instance-object-storage-distributed-cache-for-python-rust-go), it will be available to every other worker, allowing fast startup for every worker. ## Fast development iteration Windmill optimizes Rust development for faster iteration cycles. When developing and testing scripts: - **Debug mode builds**: Preview and test runs use debug mode compilation, which is significantly faster to build but produces unoptimized binaries - **Shared build directory**: All Rust scripts share a common build directory, reducing redundant compilation and improving cache efficiency - **Release mode deployment**: When scripts are deployed to production, they are compiled in release mode for optimal performance This approach provides the best of both worlds - fast development cycles during script creation and testing, with optimized performance in production deployments. ## What's next? This script is a minimal working example, but there's a few more steps that can be useful in a real-world use case: - Pass [variables and secrets](../../../core_concepts/2_variables_and_secrets/index.mdx) to a script. - Connect to [resources](../../../core_concepts/3_resources_and_types/index.mdx). - Use the [Rust client SDK](../../../advanced/2_clients/rust_client.mdx) to interact with Windmill's API from your applications. - [Trigger that script](../../../triggers/index.mdx) in many ways. - Compose scripts in [Flows](../../../flows/1_flow_editor.mdx), [low-code apps](../../../apps/0_app_editor/index.mdx) or [full-code apps](../../../full_code_apps/index.mdx). - You can [share your scripts](../../../misc/1_share_on_hub/index.md) with the community on [Windmill Hub](https://hub.windmill.dev). Once submitted, they will be verified by moderators before becoming available to everyone right within Windmill. Scripts are immutable and there is an hash for each deployment of a given script. Scripts are never overwritten and referring to a script by path is referring to the latest deployed hash at that path. For each script, a UI is autogenerated from the JSON schema inferred from the script signature, and can be customized further as standalone or embedded into rich UIs using the [App builder](../../7_apps_quickstart/index.mdx). In addition to the UI, sync and async [webhooks](../../../core_concepts/4_webhooks/index.mdx) are generated for each deployment. --- ## Quickstart PostgreSQL, MySQL, MS SQL, BigQuery, Snowflake Source: https://www.windmill.dev/docs/getting_started/scripts_quickstart/sql # PostgreSQL, MySQL, MS SQL, BigQuery, Snowflake, Redshift, Oracle, DuckDB In this quick start guide, we will write our first script in SQL. We will see how to connect a Windmill instance to an external SQL service and then send queries to the database using Windmill Scripts. ![Windmill & PostgreSQL, MySQL, BigQuery and Snowflake](./sqls.png) This tutorial covers how to create a simple script through Windmill web IDE. See the dedicated page to [develop scripts locally](../../../advanced/4_local_development/index.mdx). Windmill supports [PostgreSQL](https://www.postgresql.org/), [MySQL](https://www.mysql.com/), [Microsoft SQL Server](https://www.microsoft.com/sql-server), [BigQuery](https://cloud.google.com/bigquery) and [Snowflake](https://www.snowflake.com/). In any case, it requires creating a dedicated resource. Although all users can use BigQuery, Snowflake and MS SQL through resources and [community-available languages](../index.mdx) (TypeScript, Python, Go, Bash etc.), only instances under [Enterprise edition](/pricing) and cloud workspaces can use BigQuery, Snowflake, Oracle DB and MS SQL runtimes as a dedicated language. ## Create resource Windmill provides integrations with many different apps and services with the use of [Resources](../../../core_concepts/3_resources_and_types/index.mdx). Resources are rich objects in JSON that allow to store configuration and credentials. Each Resource has a _Resource Type_ ([PostgreSQL](https://hub.windmill.dev/resource_types/114/postgresql), [MySQL](https://hub.windmill.dev/resource_types/111/mysql), [MS SQL](https://hub.windmill.dev/resource_types/132/ms_sql_server), [BigQuery](https://hub.windmill.dev/resource_types/108/bigquery), [Snowflake](https://hub.windmill.dev/resource_types/107/snowflake)) that defines the schema that the resource of this type needs to implement. Schemas implement the [JSON Schema specification](https://json-schema.org/). :::tip You can find a list of all the officially supported Resource types on [Windmill Hub](https://hub.windmill.dev/resource_types). ::: :::tip You can pin a resource to an SQL query by adding a `-- database resource_path` line to your script. The query will automatically use the resource without having to specify it in the autogenerated user interface. ::: ### PostgreSQL To be able to connect to a [PostgreSQL](https://www.postgresql.org/) instance ([Supabase](../../../integrations/supabase.md), [Neon.tech](../../../integrations/neon.md), etc.), we'll need to define a Resource with the `PostgreSQL` Resource Type first. Head to the [Resources](../../../core_concepts/3_resources_and_types/index.mdx) page, click on "Add resource" in the top right corner and select the `PostgreSQL` type. ![Select PostgreSQL Resource Type](../../../assets/integrations/psql-1-resources.png.webp) Fill out the form with the information of your PostgreSQL instance and "Test connection" if needed. ![Paste in Resource Values](../../../assets/integrations/psql-2-postgres-rt.png.webp) :::tip For testing purposes, you can use the sample PostgreSQL Resource provided to every user. It is available under the path `f/examples/demo_windmillshowcases`. ::: #### PostgreSQL: Add a Supabase database Windmill provides a wizard to easily add a [Supabase](../../../integrations/supabase.md) database through PostgreSQL. When creating a new PostgreSQL resource, just "Add a Supabase DB". This will lead you to a Supabase page where you need to pick your organization. Then on Windmill pick a database, fill with database password and that's it. #### Use SQL to build on external APIs using Sequin With [Sequin](https://sequin.io), developers can build on top of third-party services like Salesforce or HubSpot using SQL. More details at: ### MySQL To be able to connect to a [MySQL](https://www.mysql.com/) instance, we'll need to define a Resource with the `MySQL` Resource Type first. Head to the [Resources](../../../core_concepts/3_resources_and_types/index.mdx) page, click on "Add resource" in the top right corner and select the `MySQL` type. ![Select MySQL Resource Type](./select_mysql.png.webp) Fill out the form with the information of your MySQL instance and "Test connection" if needed. ![Paste in Resource Values](./fill_mysql.png.webp) | Property | Type | Description | Default | Required | Where to Find | | -------- | ------ | --------------- | ------- | -------- | -------------------------------------------------------------------------------------- | | host | string | Instance host | | false | Your hosting provider's control panel or in your server's MySQL configuration file | | port | number | Instance port | 3306 | false | Your hosting provider's control panel or in your server's MySQL configuration file | | user | string | Username | | true | Created in MySQL (e.g., via phpMyAdmin or MySQL Workbench) or provided by your hosting | | database | string | Database name | | true | Created in MySQL (e.g., via phpMyAdmin or MySQL Workbench) or provided by your hosting | | password | string | User's password | | true | Created in MySQL (e.g., via phpMyAdmin or MySQL Workbench) or provided by your hosting | ### MS SQL To be able to connect to a [Microsoft SQL Server](https://www.microsoft.com/sql-server) instance, we'll need to define a Resource with the `ms_sql_server` Resource Type first. Head to the [Resources](../../../core_concepts/3_resources_and_types/index.mdx) page, click on "Add resource" in the top right corner and select the `ms_sql_server` type. ![Select MySQL Resource Type](./select_mssql.png.webp) Fill out the form with the information of your MS SQL instance and "Test connection" if needed. ![Paste in Resource Values](./fill_mssql.png.webp) | Property | Type | Description | Default | Required | Where to Find | | --------- | ------ | ------------------------ | ------- | -------- | ----------------------------------------------------------------------------------------------- | | host | string | Instance host | | true | Your hosting provider's control panel or in your server's MS SQL configuration file | | port | number | Instance port | | false | Your hosting provider's control panel or in your server's MS SQL configuration file | | user | string | Username | | false | Created in MS SQL (e.g., via SQL Server Management Studio) or provided by your hosting | | dbname | string | Database name | | true | Created in MS SQL (e.g., via SQL Server Management Studio) or provided by your hosting | | password | string | User's password | | false | Created in MS SQL (e.g., via SQL Server Management Studio) or provided by your hosting | | integrated_auth | bool | Use Windows Integrated Authentication | false | false | Enable to use the worker's Kerberos credentials instead of username/password | | aad_token | object | OAuth token AD | | false | Requires OAuth setup in Windmill | | instance_name | string | Named instance | | false | For named SQL Server instances (e.g., `MSSQLSERVER`) | | encrypt | bool | Enable TLS encryption | true | false | Set to false only for local development | | trust_cert| bool | Trust server certificate | true | false | If true, the server certificate will be trusted even if it is not signed by a trusted authority | | ca_cert | string | CA certificate | | false | CA certificate to verify the server certificate. [More information on MS SQL certificates](https://learn.microsoft.com/en-us/sql/linux/sql-server-linux-docker-container-security?view=sql-server-ver16#encrypt-connections-to-sql-server-linux-containers) | #### Authentication methods MS SQL Server supports three authentication methods: 1. **Username/Password**: Provide `user` and `password` fields. 2. **Azure AD (Entra)**: Use the `aad_token` field with OAuth. 3. **Windows Integrated Authentication (Kerberos)**: Enable `integrated_auth`. #### Windows Integrated Authentication For enterprise environments using Active Directory, enable `integrated_auth` to use Kerberos authentication. When enabled, the worker's service account credentials are used instead of username/password. **Requirements:** - The worker must have a valid Kerberos ticket (e.g., via `kinit` or a keytab) - The worker must have access to a valid `/etc/krb5.conf` with the correct realm configuration - The service account must have permissions on the target database **Docker/Kubernetes setup:** 1. Mount a keytab file to the worker container 2. Configure `/etc/krb5.conf` with your realm settings 3. Optionally run `kinit` at container startup or use `KRB5_KTNAME` environment variable To specify the application intent for read-only requests, add `-- ApplicationIntent=ReadOnly` to the script. :::info Azure AD (Entra) When using domain credentials via Entra (Azure Active Directory) you need to add the scope `https://database.windows.net//.default` to the [Windmill OAuth instance setting](../../../advanced/27_setup_oauth/index.mdx#azure-oauth). ::: ### BigQuery To be able to connect to a [BigQuery](https://cloud.google.com/bigquery) instance, we'll need to define a Resource with the `BigQuery` Resource Type first. Head to the [Resources](../../../core_concepts/3_resources_and_types/index.mdx) page, click on "Add resource" in the top right corner and select the `BigQuery` type. ![Select BigQuery Resource Type](./select_bigquery.png.webp) | Property | Type | Description | Required | | --------------------------- | ------ | ---------------------------------------------- | -------- | | auth_provider_x509_cert_url | string | Auth provider X.509 certificate URL. | false | | client_x509_cert_url | string | Client X.509 certificate URL. | false | | private_key_id | string | ID of the private key used for authentication. | false | | client_email | string | Email associated with the service account. | false | | private_key | string | Private key used for authentication. | false | | project_id | string | Google Cloud project ID. | true | | token_uri | string | OAuth 2.0 token URI. | false | | client_id | string | Client ID used for OAuth 2.0 authentication. | false | | auth_uri | string | OAuth 2.0 authorization URI. | false | | type | string | Type of the authentication method. | false | Here's a step-by-step guide on where to find each detail. 1. **Service account creation**: - Go to the [Google Cloud Console](https://console.cloud.google.com/). - Select the appropriate project from the top menu. - In the left navigation pane, go to "IAM & Admin" > "Service accounts". - Click on the "+ CREATE SERVICE ACCOUNT" button. - Provide a name and optional description for the service account. - Click "Create". 2. **Assign roles**: - After creating the service account, you'll be prompted to grant roles to it. Select "BigQuery" roles such as "BigQuery Admin" or "BigQuery Data Editor" based on your needs. - Click "Continue" and "Done" to create the service account. 3. **Generate key**: - In the "Service accounts" section, find the newly created service account in the list. - Click on the three dots on the right and select "Manage keys", then "Add Key". - Choose the key type as "JSON" and click "Create". 4. **Properties Details**: Once you've generated the key, the downloaded JSON file will contain all the required properties. You can directly "Test connection" if needed. ### Snowflake To be able to connect to [Snowflake](https://www.snowflake.com/), you can choose to either setup [OAuth for Snowflake](../../../advanced/27_setup_oauth/index.mdx#oauth) or by defining a Snowflake Resource. If a Snowflake OAuth connection is present, you can create a new Resource by heading to [Resources](../../../core_concepts/3_resources_and_types/index.mdx), clicking on "Add Resource" in the top right corner and selecting `snowflake_oauth`. Take a look at [this guide](../../../misc/9_guides/snowflake_app_with_user_roles/index.mdx#sample-app-setup) to learn more about how to build an App with Snowflake OAuth integration. If you do not wish to use OAuth, click on "Add Resource" in the top right corner and select the `Snowflake` type instead. ![Select Snowflake Resource Type](./select_snowflake.png.webp) | Property | Type | Description | Required | | ------------------ | ------ | ---------------------------------------------------------------------- | -------- | | account_identifier | string | Snowflake account identifier in the format `-`. | true | | private_key | string | Private key used for authentication. | true | | public_key | string | Public key used for authentication. | true | | warehouse | string | Snowflake warehouse to be used for queries. | false | | username | string | Username for Snowflake login. | true | | database | string | Name of the Snowflake database to connect to. | true | | schema | string | Schema within the Snowflake database. | false | | role | string | Role to be assumed upon connection. | false | Here's a step-by-step guide on where to find each detail. 1. **Account identifier**: The account identifier typically follows the format: `-`. You can find it in the Snowflake web interface: - Log in to your Snowflake account. - The account identifier can often be found in the URL or at the top of the Snowflake interface after you log in (in the format `https://app.snowflake.com/orgname/account_name/`). [Snowflake Documentation on Account Identifier](https://docs.snowflake.com/en/user-guide/admin-account-identifier) 2. **Username**: The username is the Snowflake user you will use to connect to the database. You will need to create a user if you don't have one: - In the Snowflake web interface, go to the "ACCOUNT" tab. - Select "Users" from the left navigation pane. - Click the "+ CREATE USER" button to create a new user with a username and password. 3. **Public key and private key**: To create the public and private keys, you will need to generate them using a tool like OpenSSL: - Open a terminal window. - Use OpenSSL to generate a public and private key pair. The exact commands may vary based on your operating system. - For example, to generate a public key: `openssl rsa -pubout -in private_key.pem -out public_key.pem` Once you have the keys, you can copy the content and paste them into the respective fields in your configuration. [Snowflake Documentation on Key Pair Authentication & Key Pair Rotation](https://docs.snowflake.com/en/user-guide/key-pair-auth) 4. **Warehouse, schema, database, and role**: These parameters are specific to your Snowflake environment and will depend on how your Snowflake instance is configured: - `warehouse`: The name of the Snowflake warehouse you want to connect to. - `schema`: The name of the Snowflake schema you want to use. - `database`: The name of the Snowflake database you want to connect to. - `role`: The role you want to use for authentication. You can find these details in the Snowflake web interface: - Log in to your Snowflake account. - You can find the names of warehouses, schemas, databases, and roles in the interface or by running SQL queries. You can directly "Test connection" if needed. ### Amazon Redshift To connect to an Amazon Redshift instance, we need to add the corresponding resource. Redshift is compatible with Windmill's PostgreSQL resources and scripts, so we'll start by adding a new PostgreSQL resource type. ![Select PostgreSQL Resource Type](../../../assets/integrations/psql-1-resources.png.webp) Get the required values from the AWS console under CLUSTERS > your Redshift cluster. ![AWS Redshift Cluster Screen](../../../assets/integrations/redshift-aws-console.png) Find the value named 'endpoint' it should look like this: ``` default-workgroup.475893240789.us-east-1.redshift-serverless.amazonaws.com:5439/dev ``` From there you can deduce your host, port and database name: - host: default-workgroup.475893240789.us-east-1.redshift-serverless.amazonaws.com - port: 5439 - dbname: dev Now you can fill those values in Windmill, fill also the user and password for the db and press "Test connection" to check that it's working. ![Fill in the required values](../../../assets/integrations/redshift-resource-filled.png) Once it's working press save and you have successfully added your Redshift instance as a PostgreSQL resource! ### Oracle To be able to connect to an [Oracle database](https://www.oracle.com/database/), you need to define an Oracle resource. Head to the [Resources](../../../core_concepts/3_resources_and_types/index.mdx) page, click on "Add resource" in the top right corner and select the `Oracle` type. ![Select Oracle Resource Type](./select_oracle.png) | Property | Type | Description | Required | | --------- | ------ | --------------- | -------- | | database | string | Database name | true | | user | string | Username | true | | password | string | User's password | true | Here's a step-by-step guide on where to find each detail. 1. **Database**: The name of the Oracle database you want to connect to. This can be found in your Oracle database configuration or by consulting your database administrator. 2. **Username**: The username for Oracle login. You will need to create a user if you don't have one: - In the Oracle database interface, go to the "Users" section. - Create a new user with a username and password. 3. **Password**: The password associated with the Oracle username. You can directly "Test connection" if needed. ### DuckDB DuckDB scripts run in-memory out-of-the-box. ## Create script Next, let's create a script that will use the newly created Resource. From the Home page, click **New** and select **Script**. Name the Script, give it a summary, and select your preferred language, [PostgreSQL](#postgresql-1), [MySQL](#mysql-1), [MS SQL](#ms-sql-1), [BigQuery](#bigquery-1), [Snowflake](#snowflake-1). ![Script creation first step](../../../assets/integrations/sql_new_script.png.webp) You can also give more details to your script, in the [settings section](../../../script_editor/settings.mdx), you can also get back to that later at any point. ### PostgreSQL Arguments need to be passed in the given format: ```sql -- $1 name1 = default arg -- $2 name2 INSERT INTO demo VALUES ($1::TEXT, $2::INT) RETURNING * ``` "name1", "name2" being the names of the arguments, and "default arg" the optional default value. Database resource can be specified from the UI or directly within script with a line `-- database resource_path`. You can then write your prepared statement. ### MySQL Arguments need to be passed in the given format: ```sql -- :name1 (text) = default arg -- :name2 (int) INSERT INTO demo VALUES (:name1, :name2) ``` "name1", "name2" being the names of the arguments, and "default arg" the optional default value. Database resource can be specified from the UI or directly within script with a line `-- database resource_path`. You can then write your prepared statement. ![Mysql statement](./mysql_statement.png.webp) ### MS SQL Arguments need to be passed in the given format: ```sql -- @P1 name1 (varchar) = default arg -- @P2 name2 (int) INSERT INTO demo VALUES (@P1, @P2) ``` "name1", "name2" being the names of the arguments, and "default arg" the optional default value. Database resource can be specified from the UI or directly within script with a line `-- database resource_path`. You can then write your prepared statement. ![Mysql statement](./mssql_statement.png.webp) ### BigQuery Arguments need to be passed in the given format: ```sql -- @name1 (string) = default arg -- @name2 (integer) -- @name3 (string[]) INSERT INTO `demodb.demo` VALUES (@name1, @name2, @name3) ``` "name1", "name2", "name3" being the names of the arguments, "default arg" the optional default value and `string`, `integer` and `string[]` the types. Database resource can be specified from the UI or directly within script with a line `-- database resource_path`. You can then write your prepared statement. ### Snowflake Arguments need to be passed in the given format: ```sql -- ? name1 (varchar) = default arg -- ? name2 (int) INSERT INTO demo VALUES (?, ?) ``` "name1", "name2" being the names of the arguments, "default arg" the optional default value and `varchar` & `int` the types. Database resource can be specified from the UI or directly within script with a line `-- database resource_path`. You can then write your prepared statement. ![Snowflake statement](./snowflake_statement.png.webp) ### Amazon Redshift Since Redshift is compatible with Windmill's PostgreSQL, you can follow the same instructions as for [PostgreSQL scripts](#postgresql-1). Make sure to select your Redshift instance as a resource. Remember when using a a Redshift resource, you should write valid Redshift, and not PostgreSQL. For example the `RETURNING *` syntax is not supported, so you may want to change the default script to something like: ```sql -- $1 name1 = default arg -- $2 name2 INSERT INTO demo VALUES ($1::TEXT, $2::INT) ``` Learn more about [the differences here](https://docs.aws.amazon.com/redshift/latest/dg/c_redshift-and-postgres-sql.html). ### Oracle Arguments need to be passed in the given format: ```sql -- database f/your/path -- :name1 (text) = default arg -- :name2 (int) -- :name3 (int) INSERT INTO demo VALUES (:name1, :name2); UPDATE demo SET col2 = :name3 WHERE col2 = :name2; ``` "name1", "name2", "name3" being the names of the arguments, and "default arg" the optional default value. ### DuckDB DuckDB arguments need to be passed in the given format: ```sql -- $name1 (text) = default arg -- $name2 (int) INSERT INTO demo VALUES ($name1, $name2) ``` "name1", "name2" being the names of the arguments, and "default arg" the optional default value. You can pass a file on S3 as an argument of type s3object. This will substitute it with the correct 's3:///...' path at runtime. You can then query this file using the standard read_csv/read_parquet/read_json functions : ```sql -- $file (s3object) SELECT * FROM read_parquet($file) ``` The other native SQL dialects (PostgreSQL, MSSQL, MySQL, BigQuery, Snowflake) also accept `(s3object)` arguments, but bind the file's contents as a JSON parameter that the user SQL reads with the dialect's JSON-table function (`OPENJSON`, `jsonb_to_recordset`, `JSON_TABLE`, ...). See [Native SQL ↔ S3](../../../core_concepts/65_sql_to_s3_streaming/index.mdx#reading-s3-files-as-parameters). Alternatively, you can reference files on the workspace directly using s3:// notation. For primary workspace storage: ```sql SELECT * FROM read_parquet('s3:///path/to/file.parquet') ``` For secondary storage: ```sql SELECT * FROM read_parquet('s3:///path/to/file.parquet') ``` This notation also works with glob patterns: ```sql SELECT * FROM read_parquet('s3:///myfiles/*.parquet') ``` The s3:// notation now uses the Windmill [S3 Proxy](../../../core_concepts/38_object_storage_in_windmill/index.mdx#s3-proxy) by default. You can also attach to other database resources (BigQuery, PostgreSQL and MySQL). We use the official and community DuckDB extensions under the hood : ```sql ATTACH '$res:u/demo/amazed_postgresql' AS db (TYPE postgres); SELECT * FROM db.public.friends; ``` Database resource can be specified from the UI or directly within the script with a line `-- database resource_path`. You can then write your prepared statement. ## Result collection You can choose what the script will return with the `result_collection` directive : | Collection strategies | Output | | --------------------------------- | ---------------------------------------- | | last_statement_all_rows (default) | Array of records | | last_statement_first_row | Record | | last_statement_all_rows_scalar | Array of scalars | | last_statement_first_row_scalar | Scalar | | all_statements_all_rows | Array of array of records | | all_statements_first_row | Array of records | | all_statements_all_rows_scalar | Array of array of scalars | | all_statements_first_row_scalar | Array of scalars | | legacy (deprecated) | Behavior before introduction of the flag | Examples: ```sql -- result_collection=all_statements_first_row_scalar SELECT 1; SELECT 2; SELECT 3; -- Result: [1, 2, 3] ``` ```sql -- result_collection=last_statement_all_rows INSERT INTO my_table VALUES ('a', 'b', 'c'); INSERT INTO my_table VALUES ('1', '2', '3'); SELECT * FROM my_table; -- Result: [ -- { "col1": "a", "col2": "b", "col3": "c" }, -- { "col1": "1", "col2": "2", "col3": "3" } -- ] -- ``` ## Contextual variables You can use [contextual variables](../../../core_concepts/47_environment_variables/index.mdx#contextual-variables) in your queries. They need to be wrapped in `%%` like this: ```sql SELECT '%%WM_WORKSPACE%%' ``` ## Raw queries ### Safe interpolated arguments To allow more flexibility than with prepared statements, Windmill offers the possibility to do safe string interpolation in your queries thanks to [backend schema validation](../../../core_concepts/13_json_schema_and_parsing/index.mdx#backend-schema-validation). This allows you to use script parameters for elements you would usually not be able to, such as table or column names. In order to avoid SQL injections however, these parameters are checked at runtime and the job will fail if any of these rules is not followed: - The parameter is a non-empty string. - The characters are all either alphabetical (ASCII only), numeric, or an underscore (`_`). Meaning no whitespace or symbol is allowed. - The string does not start with a number. - If the parameter is an enum, it must be one of the defined variants. These rules are strict enough to protect from any kind of unexpected injection, but lenient enough to have some powerful use cases. Let's look at an example: ```sql -- :daily_minimum_calories (int) -- %%table_name%% fruits/vegetables/cereals SELECT name, calories FROM %%table_name%% WHERE calories > daily_minimum_calories ``` In this example the argument `table_name` is defined as a string that can be either `"fruits"`, `"vegetables"` or `"cereals"`, and the user of the script can then choose which table to query by setting this argument. If the user of the script tries to query a different table, the job will fail before making a connection to the DB, and thus protecting potentially sensitive data. It the enum variants are omitted, the field is considered to be a regular string and only the other rules apply: ```sql -- :daily_minimum_calories (int) -- %%table_name%% SELECT name, calories FROM %%table_name%% WHERE calories > daily_minimum_calories ``` Keep in mind that this means users of this script can try this query against all existent and non-existent tables of the database. ### Unsafe interpolation on a REST script A more convenient but less secure option is to execute raw queries with a TypeScript, Deno or Python client. You can for instance do string interpolation to make the name of the table a parameter of your script: `SELECT * FROM ${table}`. However this is dangerous since the string is directly interpolated and this open the door for [SQL injections](https://en.wikipedia.org/wiki/SQL_injection). Use with care and only in trusted environment. #### PostgreSQL ```ts // Define the resource type as specified type Postgresql = { host: string, port: number, user: string, dbname: string, sslmode: string, password: string, root_certificate_pem: string } // The main function that will execute a query on a Postgresql database try { // Connect to the database await client.connect(); // Execute the query const res = await client.query(query); // Close the connection await client.end(); // Return the query result return res.rows; } catch (error) { console.error('Database query failed:', error); // Rethrow the error to handle it outside or log it appropriately throw error; } } ``` View script on [Windmill Hub](https://hub.windmill.dev/scripts/postgresql/7105/execute-arbitrary-query-and-return-results-postgresql). ```ts type Postgresql = { host: string; port: number; user: string; dbname: string; sslmode: string; password: string; }; export async function main(db: Postgresql, query: Sql = "SELECT * FROM demo;") { if (!query) { throw Error("Query must not be empty."); } const { rows } = await pgClient(db).queryObject(query); return rows; } export function pgClient(db: any) { let db2 = { ...db, hostname: db.host, database: db.dbname, tls: { enabled: false, }, } return new Client(db2) } ``` View script on [Windmill Hub](https://hub.windmill.dev/scripts/postgresql/1294/execute-query-and-return-results-postgresql). ```python from typing import TypedDict, Dict, Any import psycopg2 # Define the PostgreSQL resource type as specified class postgresql(TypedDict): host: str port: int user: str dbname: str sslmode: str password: str root_certificate_pem: str def main(query: str, db_config: postgresql) -> Dict[str, Any]: # Connect to the PostgreSQL database conn = psycopg2.connect( host=db_config["host"], port=db_config["port"], user=db_config["user"], password=db_config["password"], dbname=db_config["dbname"], sslmode=db_config["sslmode"], sslrootcert=db_config["root_certificate_pem"], ) # Create a cursor object cur = conn.cursor() # Execute the query cur.execute(query) # Fetch all rows from the last executed statement rows = cur.fetchall() # Close the cursor and connection cur.close() conn.close() # Convert the rows to a list of dictionaries to make it more readable columns = [desc[0] for desc in cur.description] result = [dict(zip(columns, row)) for row in rows] return result ``` View script on [Windmill Hub](https://hub.windmill.dev/scripts/postgresql/7106/execute-arbitrary-query-postgresql). :::tip You can find more Script examples related to PostgreSQL on [Windmill Hub](https://hub.windmill.dev/?app=postgresql). ::: #### MySQL The same logic goes for MySQL. ```ts // Define the Mysql resource type as specified type Mysql = { ssl: boolean, host: string, port: number, user: string, database: string, password: string } // The main function that will execute a query on a Mysql resource // Connect to the MySQL database connection.connect(err => { if (err) { reject(err); return; } // Execute the query provided as a parameter connection.query(query, (error, results) => { // Close the connection after the query execution connection.end(); if (error) { reject(error); } else { resolve(results); } }); }); }); } ``` View script on [Windmill Hub](https://hub.windmill.dev/scripts/mysql/7108/execute-arbitrary-query-mysql). ```ts // Define the MySQL resource type as specified type Mysql = { ssl: boolean, host: string, port: number, user: string, database: string, password: string } // The main function that executes a query on a MySQL database // Create a new connection pool using the provided MySQL resource const pool = createMysqlPool({ host: mysqlResource.host, user: mysqlResource.user, database: mysqlResource.database, password: mysqlResource.password, port: mysqlResource.port, // Use the adjusted SSL configuration ssl: sslConfig, waitForConnections: true, connectionLimit: 10, queueLimit: 0, }); try { // Get a connection from the pool and execute the query const [rows] = await pool.query(query); return rows; } catch (error) { // If an error occurs, throw it to be handled by the caller throw new Error(`Failed to execute query: ${error}`); } finally { // Always close the pool after the operation is complete await pool.end(); } } ``` View script on [Windmill Hub](https://hub.windmill.dev/scripts/mysql/7107/execute-arbitrary-query-mysql). ```python from typing import TypedDict import mysql.connector as mysql_connector # Define the MySQL resource type class mysql(TypedDict): ssl: bool host: str port: float user: str database: str password: str def main(mysql_credentials: mysql, query: str) -> str: # Connect to the MySQL database using the provided credentials connection = mysql_connector.connect( host=mysql_credentials["host"], user=mysql_credentials["user"], password=mysql_credentials["password"], database=mysql_credentials["database"], port=int(mysql_credentials["port"]), ssl_disabled=not mysql_credentials["ssl"], ) # Create a cursor object cursor = connection.cursor() # Execute the query cursor.execute(query) # Fetch one result result = cursor.fetchone() # Close the cursor and connection cursor.close() connection.close() # Return the result return str(result[0]) ``` View script on [Windmill Hub](https://hub.windmill.dev/scripts/mysql/7109/execute-arbitrary-query-mysql). And so on for [MS SQL](#ms-sql), [BigQuery](#bigquery) and [Snowflake](#snowflake). ## Customize your script After you're done, click on "[Deploy](../../../core_concepts/0_draft_and_deploy/index.mdx)", which will save it to your workspace. You can now use this Script in your [Flows](../../../flows/1_flow_editor.mdx), [app](../../../full_code_apps/index.mdx) or as standalone. Feel free to customize your script's metadata ([path](../../../core_concepts/16_roles_and_permissions/index.mdx#path), name, description), runtime ([concurrency limits](../../../script_editor/concurrency_limit.mdx), [worker group](../../../script_editor/settings.mdx#worker-group-tag), [cache](../../../core_concepts/24_caching/index.md), [dedicated workers](../../../core_concepts/25_dedicated_workers/index.mdx)) and [generated UI](../../../script_editor/customize_ui.mdx). ![Customize SQL](./customize_sql.png 'Customize SQL') ## What's next? Those scripts are minimal working examples, but there's a few more steps that can be useful in a real-world use case: - Pass [variables and secrets](../../../core_concepts/2_variables_and_secrets/index.mdx) to a script. - Connect to [resources](../../../core_concepts/3_resources_and_types/index.mdx). - [Trigger that script](../../../triggers/index.mdx) in many ways. - Compose scripts in [Flows](../../../flows/1_flow_editor.mdx), [low-code apps](../../../apps/0_app_editor/index.mdx) or [full-code apps](../../../full_code_apps/index.mdx) (in particular, [Database studio](#database-studio) to visualize and manage your databases in apps). - You can [share your scripts](../../../misc/1_share_on_hub/index.md) with the community on [Windmill Hub](https://hub.windmill.dev). Once submitted, they will be verified by moderators before becoming available to everyone right within Windmill. Scripts are immutable and there is a hash for each deployment of a given script. Scripts are never overwritten and referring to a script by path is referring to the latest deployed hash at that path. For each script, a UI is autogenerated from the JSON schema inferred from the script signature, and can be customized further as standalone or embedded into rich UIs using the [App builder](../../7_apps_quickstart/index.mdx). In addition to the UI, sync and async [webhooks](../../../core_concepts/4_webhooks/index.mdx) are generated for each deployment. ## Database studio From Windmill's [low-code app editor](../../../apps/0_app_editor/index.mdx) (legacy), you can also use the [Database studio](../../../apps/4_app_configuration_settings/database_studio.mdx) component to visualize and manage your databases (PostgreSQL / MySQL / MS SQL / Snowflake / BigQuery are all supported). ![Database studio](../../../assets/apps/4_app_component_library/db_studio.png "Database studio") The Database studio component allows you to: - Display the content of a table. - Edit the content of a table by directly editing the cells (only when the cell is editable). - Add a new row. - Delete a row. All details at: ## Streaming large query results to S3 (Enterprise feature) Sometimes, your SQL script will return too much data which exceeds the 10 000 rows query limit within Windmill. In this case, you will want to use the s3 flag to stream your query result to a file. --- ## TypeScript quickstart Source: https://www.windmill.dev/docs/getting_started/scripts_quickstart/typescript # TypeScript quickstart In this quick start guide, we will write our first script in TypeScript. Windmill uses [Bun](https://bun.sh/), [Nodejs](#nodejs) and [Deno](https://deno.land/) as the available TypeScript runtimes. This tutorial covers how to create a simple "Hello World" script in TypeScript through Windmill web IDE, with the standard mode of handling dependencies in TypeScript (Lockfile per script inferred from imports). See the dedicated pages to [develop scripts locally](../../../advanced/4_local_development/index.mdx) and other methods of [handling dependencies in TypeScript](../../../advanced/14_dependencies_in_typescript/index.mdx). Scripts are the basic building blocks in Windmill. They can be [run and scheduled](../../../triggers/index.mdx) as standalone, chained together to create [Flows](../../../flows/1_flow_editor.mdx) or displayed with a personalized User Interface as [Apps](../../7_apps_quickstart/index.mdx). Scripts consist of 2 parts: - [Code](#code): for TypeScript scripts, it must have at least a main function. - [Settings](#settings): settings & metadata about the Script such as its path, summary, description, [JSON Schema](../../../core_concepts/13_json_schema_and_parsing/index.mdx) of its inputs (inferred from its signature). When stored in a code repository, those 2 parts are stored separately at `.ts` and `.script.yaml`. [This](https://hub.windmill.dev/scripts/slack/1284/send-message-to-channel-slack) is a simple example of a script built in TypeScript with Windmill: ```ts type Slack = { token: string; }; // Use the chat.postMessage method from the Slack WebClient to send a message await web.chat.postMessage({ channel: channel, text: message }); } ``` In this quick start guide, we'll create a script that greets the operator running it. From the Home page, click **New** and select **Script**. This will take you to the first step of script creation: [Metadata](../../../script_editor/settings.mdx#metadata). ## Settings ![New script](../../../../static/images/script_languages.png "New script") As part of the [settings](../../../script_editor/settings.mdx) menu, each script has metadata associated with it, enabling it to be defined and configured in depth. - **Summary** (optional) is a short, human-readable summary of the Script. It will be displayed as a title across Windmill. If omitted, the UI will use the `path` by default. - **Path** is the Script's unique identifier that consists of the [script's owner](../../../core_concepts/16_roles_and_permissions/index.mdx), and the script's name. The owner can be either a user, or a group ([folder](../../../core_concepts/8_groups_and_folders/index.mdx#folders)). - **Description** is where you can give instructions through the [auto-generated UI](../../../core_concepts/6_auto_generated_uis/index.mdx) to users on how to run your Script. It supports markdown. - **Language** of the script. - **Script kind**: Action (by default), [Trigger](../../../flows/10_flow_trigger.mdx), [Approval](../../../flows/11_flow_approval.mdx), [Error handler](../../../flows/7_flow_error_handler.md) or [Preprocessor](../../../core_concepts/43_preprocessors/index.mdx). This acts as a tag to filter appropriate scripts from the [flow editor](../../6_flows_quickstart/index.mdx). This menu also has additional settings on [Runtime](../../../script_editor/settings.mdx#runtime), [Generated UI](#generated-ui) and [Triggers](../../../script_editor/settings.mdx#triggers). Now click on the code editor on the left side. ## Code Windmill provides an online editor to work on your Scripts. The left-side is the editor itself. The right-side [previews the UI](../../../core_concepts/6_auto_generated_uis/index.mdx) that Windmill will generate from the Script's signature - this will be visible to the users of the Script. You can preview that UI, provide input values, and [test your script](#instant-preview--testing) there. ![Demo TS](./demo_ts.png 'Demo TS') There are two options for runtimes in TypeScript: - Bun (with a [Nodejs](#nodejs) mode if needed) - [Deno](#deno) As we picked `TypeScript (Bun)` for this example, Windmill provided some TypeScript boilerplate. Let's take a look: ```typescript // there are multiple modes to add as header: //nobundling //native //npm //nodejs // https://www.windmill.dev/docs/getting_started/scripts_quickstart/typescript#modes // import { toWords } from "number-to-words@1" } ``` In Windmill, scripts need to have a `main` function that will be the script's entrypoint. There are a few important things to note about the `main`. - The main arguments are used for generating 1. the [input spec](../../../core_concepts/13_json_schema_and_parsing/index.mdx) of the Script 2. the [frontend](../../../core_concepts/6_auto_generated_uis/index.mdx) that you see when running the Script as a standalone app. - Type annotations are used to generate the UI form, and help pre-validate inputs. While not mandatory, they are highly recommended. You can customize the UI in later steps (but not change the input type!). Also take a look at the import statement lines that are commented out. In TypeScript, [dependencies](../../../advanced/14_dependencies_in_typescript/index.mdx) and their versions are contained in the script and hence there is no need for any additional steps. The TypeScript runtime is Bun, which is 100% compatible with Node.js without any code modifications. You can use npm imports directly in Windmill. The last import line imports the Windmill client, that is needed for example, to access [variables](../../../core_concepts/2_variables_and_secrets/index.mdx) or [resources](../../../core_concepts/3_resources_and_types/index.mdx). We won't go into that here. Back to our "Hello World". We can clear up unused import statements, change the main to take in the user's name. Let's also return the `name`, maybe we can use this later if we use this Script within a [flow](../../../flows/1_flow_editor.mdx) or [app](../../../full_code_apps/index.mdx) and need to pass its result on. ```typescript return { name }; } ``` ## Modes ### Pre-bundling and nobundling Windmill [pre-bundles](/changelog/pre-bundle-bun-scripts) your script at deployment time using [Bun bundler](https://bun.sh/docs/bundler). This improve memory usage and speed up the execution time of your script. If you would like to disable this feature, you can add the following comment at the top of your script: ```ts //nobundling ``` ### Native Windmill provides a runtime that is more lightweight but supports less features and allow to run scripts in a more lightweight manner with direct bindings to v8. To enable, you can add the following comment at the top of your script: ```ts //native ``` To learn more, see [Rest](../6_rest_grapqhql_quickstart/index.mdx) scripts, that are under the hood Bun TypeScripts with a `//native` header. ### NodeJS Windmill provides a true NodeJS compatibility mode. This means that you can run your existing NodeJS code without any modifications. The only thing you need to do is to select `TypeScript (Bun)` as the runtime and as the first line, use: ```ts //nodejs ``` ![Nodejs Compatibility](./nodejs_compatibility.png 'Nodejs Compatibility') This method is an escape hatch for using another runtime (NodeJS), but it is slower than Deno and Bun since it resorts to Bun under the hood. This feature is exclusive to [Cloud plans and Self-Hosted Enterprise edition](/pricing). ### Npm Similarly, you can use `npm install` instead of `bun install` to install dependencies. This is useful as an escape hatch for cases not supported by bun: ```ts //npm ``` This is also exclusive to [Cloud plans and Self-Hosted Enterprise edition](/pricing). ### Deno You can also pick `TypeScript (Deno)` as the language instead of `TypeScript (Bun)`. The walkthrough is the same as with Bun ([above](#code)): a `main` function as entrypoint, an [auto-generated UI](../../../core_concepts/6_auto_generated_uis/index.mdx) from its signature, and [dependencies](../../../advanced/14_dependencies_in_typescript/index.mdx) resolved directly from imports. The differences with Bun are: - The resolution of imports is done by [Deno](https://deno.com/runtime), so npm imports use the `npm:` prefix (e.g. `import * as wmill from "npm:windmill-client@1.525.0"`), and `https://` and `jsr:` imports are also supported. - The Bun-specific [modes](#modes) (`//nobundling`, `//native`, `//nodejs`, `//npm`) do not apply. ```typescript // Deno uses "npm:" prefix to import from npm (https://deno.land/manual@v1.36.3/node/npm_specifiers) // import * as wmill from "npm:windmill-client@1.525.0" } ``` ## Instant preview & testing Look at the UI preview on the right: it was updated to match the input signature. Run a test (`Ctrl` + `Enter`) to verify everything works. You can change how the UI behaves by changing the main signature. For example, if you add a default for the `name` argument, the UI won't consider this field as required anymore. ```typescript main(name: string = "you") ``` Now let's go to the last step: the "Generated UI" settings. ## Generated UI From the Settings menu, the "Generated UI" tab lets you customize the script's arguments. The UI is generated from the Script's main function signature, but you can add additional constraints here. For example, we could use the `Customize property`: add a regex by clicking on `Pattern` to make sure users are providing a name with only alphanumeric characters: `^[A-Za-z0-9]+$`. Let's still allow numbers in case you are some tech billionaire's kid. ![Advanced settings for TypeScript](./customize_ts.png.webp) ## Workflows as code One way to write distributed programs that execute distinct jobs is to use [flows](../../../flows/1_flow_editor.mdx) that chain scripts together. Another approach is to write a program that defines the jobs and their dependencies, and then execute that program directly in your script. This is known as [workflows as code](../../../core_concepts/31_workflows_as_code/index.mdx). Wrap your orchestration function with `workflow()` and annotate task functions with `task()`. Each task runs as a separate job with its own logs and timeline entry, while the workflow suspends between tasks (releasing its worker slot). ![Flow as code in TypeScript](./flow_as_code_ts.png 'Flow as code in TypeScript') All details at: ## Run! We're done! Now let's look at what users of the script will do. Click on the [Deploy](../../../core_concepts/0_draft_and_deploy/index.mdx) button to load the script. You'll see the user input form we defined earlier. Note that Scripts are [versioned](../../../core_concepts/34_versioning/index.mdx#script-versioning) in Windmill, and each script version is uniquely identified by a hash. Fill in the input field, then hit "Run". You should see a run view, as well as your logs. All script runs are also available in the [Runs](../../../core_concepts/5_monitor_past_and_future_runs/index.mdx) menu on the left. ![Run hello world in TypeScript](./run_ts.png.webp) You can also choose to [run the script from the CLI](../../../advanced/3_cli/index.mdx) with the pre-made Command-line interface call. ## Caching Every bundle on Bun is cached on disk by default. Furthermore if you use the [Distributed cache storage](../../../core_concepts/38_object_storage_in_windmill/index.mdx#instance-object-storage-distributed-cache-for-python-rust-go), it will be available to every other worker, allowing fast startup for every worker. ## What's next? This script is a minimal working example, but there's a few more steps that can be useful in a real-world use case: - Pass [variables and secrets](../../../core_concepts/2_variables_and_secrets/index.mdx) to a script. - Connect to [resources](../../../core_concepts/3_resources_and_types/index.mdx). - [Trigger that script](../../../triggers/index.mdx) in many ways. - Compose scripts in [Flows](../../../flows/1_flow_editor.mdx) or [Apps](../../7_apps_quickstart/index.mdx). - You can [share your scripts](../../../misc/1_share_on_hub/index.md) with the community on [Windmill Hub](https://hub.windmill.dev). Once submitted, they will be verified by moderators before becoming available to everyone right within Windmill. Scripts are immutable and there is an hash for each deployment of a given script. Scripts are never overwritten and referring to a script by path is referring to the latest deployed hash at that path. For each script, a UI is autogenerated from the JSON schema inferred from the script signature, and can be customized further as standalone or embedded into rich UIs using the [App builder](../../7_apps_quickstart/index.mdx). In addition to the UI, sync and async [webhooks](../../../core_concepts/4_webhooks/index.mdx) are generated for each deployment. --- ## resource usage Source: https://www.windmill.dev/docs/integrations/_resource_usage {/* Shared boilerplate for integration pages: how to use the resource once created. Usage: import ResourceUsage from './_resource_usage.mdx'; `hub` (optional) is the app slug on https://hub.windmill.dev. */} Your resource can be [passed as a parameter](../core_concepts/3_resources_and_types/index.mdx#passing-resources-as-parameters-to-scripts-preferred) or [fetched directly](../core_concepts/3_resources_and_types/index.mdx#fetching-them-from-within-a-script-by-using-the-wmill-client-in-the-respective-language) within [scripts](../script_editor/index.mdx), [flows](../flows/1_flow_editor.mdx), [low-code apps](../apps/0_app_editor/index.mdx) and [full-code apps](../full_code_apps/index.mdx). :::tip {props.hub && ( Find some pre-set interactions with {props.name} on the{' '} Hub. )} Feel free to create your own {props.name} scripts on [Windmill](../getting_started/00_how_to_use_windmill/index.mdx). ::: --- ## Airtable Source: https://www.windmill.dev/docs/integrations/airtable # Airtable integration [Airtable](https://www.airtable.com/) is a cloud collaboration platform for organizing and managing data. There are two resources associated with Airtable. Both are required to use Airtable's API from Windmill. ## Airtable account Airtable authenticates with personal access tokens (legacy API keys were removed in February 2024). Create a token on Airtable's Builder hub, grant it the scopes you need (e.g. `data.records:read`, `data.records:write`) and access to the bases you want to use, then paste it on Windmill as the `apiKey` field of the resource. | Property | Type | Description | Default | Required | Where to find | | -------- | ------ | ------------------------------ | ----------------- | -------- | ---------------------------------------------------- | | apiKey | string | Airtable personal access token | patXXXXXXXXXXXXXX | true | airtable.com/create/tokens > Create token | ## Airtable table Now specify Airtable which database and table you want to interact with: - **Database ID** can be found on the URL of the page. It starts with "app" and ends before the next "/". e.g. appcy7pfdzgJIhto. - **Table name** is the name of the tab. By default it is called "Table 1". | Property | Type | Description | Required | Where to find | | --------- | ------ | ---------------------------------------------- | -------- | ------------------------------------------ | | baseId | string | Unique identifier for a specific Airtable base | True | Page URL | | tableName | string | Name of an individual table within that base | True | In Airtable. Name of the tab of a database | --- ## Appwrite Source: https://www.windmill.dev/docs/integrations/appwrite # Appwrite integration [Appwrite](https://appwrite.io/) is an end-to-end backend server for web and mobile apps. To integrate Appwrite to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). ![Add Appwrite Resource](../assets/integrations/add-apprite.png.webp) | Property | Type | Description | Default | Required | Where to find | | ----------- | ------- | ------------------------------------------------------------- | -------- | -------- | ------------------------------------------------------- | | endpoint | string | url of your appwrite server | https:// | true | Your Appwrite server's URL | | project | string | ID of your appwrite project | | true | Appwrite Dashboard > Your Project > Settings > ID | | key | string | API key of your appwrite project | | true | Appwrite Dashboard > Your Project > API Keys | | self_signed | boolean | use self signed certificates on server (only for development) | | false | (This is a configuration option, not found on Appwrite) | --- ## Aws Source: https://www.windmill.dev/docs/integrations/aws # AWS integration [AWS](https://aws.amazon.com/) is a cloud computing platform offering various services like computing, storage and databases. To integrate AWS with Windmill, you can configure either a classic **AWS resource** using access keys, or a more secure **AWS OIDC resource**, which assumes IAM roles via OpenID Connect. These should be saved as a [resource](../core_concepts/3_resources_and_types/index.mdx). :::info Self-host If you're looking for a way to self-host Windmill using AWS, see [Self-Host Windmill](../advanced/1_self_host/index.mdx). ::: --- ## AWS Resource | Property | Type | Description | Default | Required | Where to Find | | ------------------ | ------ | ---------------------------------- | ------- | -------- | ------------------------------------------------------------------------- | | awsAccessKeyId | string | AWS Access Key ID for your account | | true | AWS Management Console > IAM > Users > [Your User] > Security Credentials | | awsSecretAccessKey | string | AWS Secret Access Key for account | | true | AWS Management Console > IAM > Users > [Your User] > Security Credentials | | region | string | AWS Region for your resources | | false | AWS Management Console > Top Right Corner (e.g., "N. Virginia") | --- ## AWS OIDC Resource | Property | Type | Description | Default | Required | Where to Find / Define | |----------|--------|-----------------------------------------------------|---------|----------|-----------------------------------------------------------------------------------------| | roleArn | string | ARN of the IAM role to assume using OIDC | | true | AWS Management Console > IAM > Roles > [Your Role] > ARN | | region | string | AWS Region for your resources | | false | AWS Management Console > Top Right Corner (e.g., "us-west-2") | > ℹ️ Ensure the IAM role trusts Windmill's OIDC provider and has sufficient permissions for the services you intend to use. --- ## Usage --- ## Aws s3 Source: https://www.windmill.dev/docs/integrations/aws-s3 # Amazon S3 integration [Amazon S3](https://aws.amazon.com/s3/) is a cloud storage service. Amazon S3 is used in Windmill through the generic `s3` [resource type](https://hub.windmill.dev/resource_types/42/). The [S3 APIs integrations](./s3.mdx) page is the canonical reference: it covers the resource fields, how to use the resource in scripts, flows and apps, and how to plug an S3 bucket as [workspace or instance object storage](../core_concepts/38_object_storage_in_windmill/index.mdx). :::info Self-host If you're looking for a way to self-host Windmill using AWS, see [Self-Host Windmill](../advanced/1_self_host/index.mdx). ::: ## Where to find the resource details on AWS | Property | Value for Amazon S3 | | --------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | bucket | Name of the S3 bucket, from the AWS Management Console | | region | Region where the bucket is located, in the form `eu-west-3`. Also visible in the bucket's endpoint URL | | endPoint | Varies by region, in the form `s3.eu-west-3.amazonaws.com`. See the [AWS documentation](https://docs.aws.amazon.com/general/latest/gr/s3.html) | | useSSL | `true` - SSL/TLS is required for Amazon S3 | | pathStyle | `false` - Amazon S3 uses virtual-hosted-style URLs | | accessKey | Required. Access key ID from the IAM section of the AWS Management Console under "My Security Credentials". Make sure the user has the right policies allocated | | secretKey | Required. Secret access key from the IAM section of the AWS Management Console under "My Security Credentials". Make sure the user has the right policies allocated | --- ## Bigquery Source: https://www.windmill.dev/docs/integrations/bigquery # BigQuery integration [BigQuery](https://cloud.google.com/bigquery) is a cloud-based data warehousing platform. Windmill provides a framework to support BigQuery databases. ![Integration between BigQuery and Windmill](../assets/integrations/bigquery_header.png 'Connect a BigQuery instance with Windmill') Please refer to the [SQL Getting started section](../getting_started/0_scripts_quickstart/5_sql_quickstart/index.mdx). --- ## Clickhouse Source: https://www.windmill.dev/docs/integrations/clickhouse # ClickHouse integration [ClickHouse](https://clickhouse.com/) is an open-source column-oriented database management system. To integrate ClickHouse to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). ![Add ClickHouse Resource](../assets/integrations/add-clickhouse.png.webp) | Property | Type | Description | Default | Required | Where to Find | | -------- | ------ | ----------------------------------- | ------- | -------- | -------------------------------------------------------------------------------------------- | | host | string | Hostname or IP of ClickHouse server | | true | Provided by your hosting provider or found in the ClickHouse config file (`config.xml`) | | username | string | Username for ClickHouse connection | | false | Found in the ClickHouse users config file (`users.xml`) or provided by your hosting provider | | password | string | Password for ClickHouse connection | | false | Found in the ClickHouse users config file (`users.xml`) or provided by your hosting provider | --- ## Cloudflare r2 Source: https://www.windmill.dev/docs/integrations/cloudflare-r2 # Cloudflare R2 integration [Cloudflare R2](https://www.cloudflare.com/products/r2/) is a cloud object storage service for data-intensive applications. Its API follows the same schema as any S3-compatible API. Cloudflare R2 is used in Windmill through the generic `s3` [resource type](https://hub.windmill.dev/resource_types/42/). The [S3 APIs integrations](./s3.mdx) page is the canonical reference: it covers the resource fields, how to use the resource in scripts, flows and apps, and how to plug a bucket as [workspace or instance object storage](../core_concepts/38_object_storage_in_windmill/index.mdx). ## Where to find the resource details on Cloudflare See the [R2 S3 API documentation](https://developers.cloudflare.com/r2/api/s3/api/) for details. | Property | Value for Cloudflare R2 | | --------- | ----------------------------------------------------------------------------------------------------- | | bucket | Name of the bucket, from the R2 dashboard | | region | Set when creating the bucket; `auto` routes to the closest available region | | endPoint | In the form `.r2.cloudflarestorage.com`, from the bucket settings in the R2 dashboard | | useSSL | `true` - SSL/TLS is required for Cloudflare R2 | | pathStyle | `false` - virtual-hosted-style URLs are used by default in R2 | | accessKey | Access key ID of an R2 API token, created from the R2 dashboard under "Manage R2 API Tokens" | | secretKey | Secret access key of the same R2 API token | --- ## Datadog Source: https://www.windmill.dev/docs/integrations/datadog # Datadog integration [Datadog](https://www.datadoghq.com/) is a monitoring and analytics platform for cloud-scale infrastructure and applications. To integrate Datadog to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). ![Add Datadog Resource](../assets/integrations/add-datadog.png.webp) | Property | Type | Description | Default | Required | Where to Find | | -------- | ------ | ----------------------------------- | ------- | -------- | ------------------------------------------------------------------------------------------------ | | apiKey | string | Datadog API key for authentication | | true | Datadog Dashboard > Integrations > APIs > API Keys | | appKey | string | Datadog APP key for specific access | | false | Datadog Dashboard > Integrations > APIs > Application Keys | | apiBase | string | Base URL for the Datadog API | | false | Datadog API documentation (default: `https://api.datadoghq.com` or region-specific API base URL) | --- ## Discord Source: https://www.windmill.dev/docs/integrations/discord # Discord integration [Discord](https://discord.com/) is a voice, video, and text communication platform. To integrate Discord to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). | Property | Type | Description | Default | Required | Where to Find | | ----------- | ------ | ----------------------- | ------- | -------- | ------------------------------------------------------------------------------------------- | | webhook_url | string | The Discord webhook URL | | true | Discord Server > Server Settings > Integrations > Webhooks > Create Webhook or Edit Webhook | Windmill also defined a [resource type](https://hub.windmill.dev/resource_types/104/discord_bot_configuration) for Discord bots. An example is given by our [Documentation Discord bot using Supabase and OpenAI's GPT to help support teams](/blog/knowledge-base-discord-bot) tutorial. :::tip Windmill Discord Windmill has its own Discord server for its community, questions and collaborations. Join following [this link](https://discord.com/invite/V7PM2YHsPB). ::: --- ## Duckdb Source: https://www.windmill.dev/docs/integrations/duckdb # DuckDB integration [DuckDB](https://duckdb.org/) is an open-source, in-process SQL OLAP database management system designed for fast analytical query workloads. Windmill supports seamless integration with DuckDB, allowing you to manipulate data from S3 (csv, parquet, json), [Azure Blob Storage](./microsoft-azure-blob.md), BigQuery, PostgreSQL, and MySQL. DuckDB in Windmill supports automatic column detection on S3 objects. You can query S3 paths directly without wrapping them in `read_parquet()` — for example `SELECT col1, col2 FROM 's3:///file.parquet'` — and the SQL parser will infer the referenced columns. The standard `read_parquet()`, `read_csv()`, and `read_json()` table functions also support column detection when used with S3 paths. ![Integration between DuckDB and Windmill](../assets/integrations/duckdb.png 'Run a DuckDB script with Windmill') ## Azure Blob Storage support DuckDB scripts in Windmill can read from and write to [Azure Blob Storage](./microsoft-azure-blob.md). When Azure Blob is configured as a [workspace storage](../core_concepts/38_object_storage_in_windmill/index.mdx), DuckDB can use the same storage paths to query and write data in Parquet, CSV, or JSON format. This works with the same S3-compatible path syntax, and requires an Azure Blob storage resource to be configured in the workspace. ## Pipelines and macro libraries DuckDB is also the engine of Windmill [pipelines](../core_concepts/63_pipelines/index.mdx): DuckDB steps can [materialize](../core_concepts/63_pipelines/materialization.mdx) managed [DuckLake](../core_concepts/11_persistent_storage/ducklake.mdx) tables, and shared SQL logic can be published as workspace [macro libraries](../core_concepts/63_pipelines/macros.mdx) callable from any DuckDB script. To get started, check out the [SQL Getting Started section](/docs/getting_started/scripts_quickstart/sql#duckdb-1). --- ## Excel Source: https://www.windmill.dev/docs/integrations/excel # Microsoft Excel integration Windmill doesn't have a direct API integration with Microsoft Excel, but it provides powerful ways to work with Excel files through file uploads and processing. You can handle Excel files (.xlsx, .xls) using two main approaches: [base64 encoded strings](#base64-encoded-strings) for smaller files or [S3 object storage](#s3-object-storage-recommended-for-larger-files) for larger files and better performance. Excel files in Windmill can be: - **Uploaded directly** in [low-code apps](../apps/0_app_editor/index.mdx) (legacy) using [file input components](../apps/4_app_configuration_settings/file_input.mdx) or within scripts and flows' [auto-generated UIs](../core_concepts/6_auto_generated_uis/index.mdx). - **Stored and processed** using [workspace object storage (S3)](../core_concepts/38_object_storage_in_windmill/index.mdx). - **Parsed and manipulated** using popular libraries like [pandas](https://pandas.pydata.org/) ([Python](../getting_started/0_scripts_quickstart/2_python_quickstart/index.mdx)) or [xlsx](https://www.npmjs.com/package/xlsx) ([TypeScript](../getting_started/0_scripts_quickstart/1_typescript_quickstart/index.mdx)). - **Processed in batch** through [flows](../flows/1_flow_editor.mdx) for multiple files. - **Visualized** in scripts and flows using [rich-rendering](../core_concepts/19_rich_display_rendering/index.mdx) or in [low-code apps](../apps/0_app_editor/index.mdx) (legacy) using tables and charts using file input and display components (AgGrid Table, Text, etc.) ## Base64 encoded strings For smaller Excel files (< 10MB), you can use [base64 encoded strings](../core_concepts/18_files_binary_data/index.mdx). This is the simplest approach for basic file processing. ```python import pandas as pd import io def main(excel: bytes, sheet_name: str = None): """ Read specific sheet from Excel file with options. Args: excel: The Excel file as bytes sheet_name: Optional sheet name (defaults to first sheet) Returns: Processed Excel data """ excel_buffer = io.BytesIO(excel) # Read specific sheet or first sheet df = pd.read_excel(excel_buffer, sheet_name=sheet_name or 0) # Example processing: get summary statistics summary = { "rows": len(df), "columns": len(df.columns), "column_names": df.columns.tolist(), "data_preview": df.head(10).to_dict(orient='records'), "numeric_summary": df.describe().to_dict() if df.select_dtypes(include='number').shape[1] > 0 else None } return summary ``` ```typescript // Parse Excel file const workbook = XLSX.read(data, { type: 'array' }); // Get sheet name (first sheet if not specified) const targetSheet = sheetName || workbook.SheetNames[0]; const worksheet = workbook.Sheets[targetSheet]; // Convert to JSON const jsonData = XLSX.utils.sheet_to_json(worksheet); return { sheets: workbook.SheetNames, selectedSheet: targetSheet, rows: jsonData.length, data: jsonData.slice(0, 10), // Preview first 10 rows summary: { totalSheets: workbook.SheetNames.length, availableSheets: workbook.SheetNames } }; } ``` More details on how to use base64 encoded strings in scripts and flows can be found in the [Handling files and binary data](../core_concepts/18_files_binary_data/index.mdx) section: ## S3 object storage (recommended for larger files) For better performance and larger files, use Windmill's [workspace object storage integration](../core_concepts/38_object_storage_in_windmill/index.mdx). ```python import pandas as pd import wmill from wmill import S3Object def main(excel_file: S3Object, sheet_name: str = None): """ Process Excel file from S3 storage. Args: excel_file: S3Object pointing to the Excel file sheet_name: Optional sheet name Returns: Processed data and summary """ # Load file from S3 file_content = wmill.load_s3_file(excel_file) # Read Excel from bytes df = pd.read_excel(file_content, sheet_name=sheet_name or 0) # Process data processed_data = { "shape": df.shape, "columns": df.columns.tolist(), "dtypes": df.dtypes.to_dict(), "sample_data": df.head(5).to_dict(orient='records'), "missing_values": df.isnull().sum().to_dict(), "numeric_stats": df.describe().to_dict() if len(df.select_dtypes(include='number').columns) > 0 else None } return processed_data ``` ```typescript // Parse Excel file const workbook = XLSX.read(fileContent, { type: 'array' }); // Get target sheet const targetSheet = sheetName || workbook.SheetNames[0]; const worksheet = workbook.Sheets[targetSheet]; // Convert to JSON const jsonData = XLSX.utils.sheet_to_json(worksheet); // Process and return results return { file_info: { sheets: workbook.SheetNames, processed_sheet: targetSheet, total_rows: jsonData.length }, data_preview: jsonData.slice(0, 10), column_info: jsonData.length > 0 ? Object.keys(jsonData[0]) : [], summary: { empty_rows: jsonData.filter(row => Object.values(row).every(val => val === '' || val == null)).length, total_columns: jsonData.length > 0 ? Object.keys(jsonData[0]).length : 0 } }; } ``` More details on how to use S3 object storage in scripts and flows can be found in the [Object storage in Windmill (S3)](../core_concepts/38_object_storage_in_windmill/index.mdx) section: --- ## Faunadb Source: https://www.windmill.dev/docs/integrations/faunadb # FaunaDB integration (deprecated) :::caution Fauna service shutdown The Fauna cloud service was [shut down at the end of May 2025](https://www.infoq.com/news/2025/03/fauna-shuts-down/). The `faunadb` resource type only worked with the hosted service and can no longer be used. The core database was [released as open source](https://github.com/fauna/faunadb), but it does not expose the same hosted API this integration relied on. If you are migrating off Fauna, you can connect Windmill to alternatives such as [PostgreSQL](../getting_started/0_scripts_quickstart/5_sql_quickstart/index.mdx), [MongoDB](./mongodb.md) or [Supabase](./supabase.md). ::: FaunaDB was a serverless, document-oriented database. The integration relied on a [resource](../core_concepts/3_resources_and_types/index.mdx) with a `region` and a `secret` API key, both obtained from the Fauna dashboard, which no longer exists. --- ## Funkwhale Source: https://www.windmill.dev/docs/integrations/funkwhale # Funkwhale integration [Funkwhale](https://funkwhale.audio/) is an open-source music streaming and sharing platform. To integrate Funkwhale to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). ![Add Funkwhale Resource](../assets/integrations/add-funkwhale.png.webp) | Property | Type | Description | Default | Required | Where to Find | | -------- | ------ | ------------------------------------------------------------------------------- | ------- | -------- | ----------------------------------------------------- | | baseUrl | string | Base URL of your Funkwhale instance | | true | Authorize URL is at /authorize | | token | string | Access token to act as a logged-in user (optional for unauthenticated requests) | | false | Funkwhale > Settings > Applications > New Application | --- ## Gcal Source: https://www.windmill.dev/docs/integrations/gcal # Google Calendar integration [Google Calendar](https://calendar.google.com/) is a time-management and scheduling web application. The Google Calendar integration is done through OAuth. You just need to sign in from your Google account on your browser. The access will be automatically saved to the workspace as a [resource](../core_concepts/3_resources_and_types/index.mdx). On [self-hosted instances](../advanced/1_self_host/index.mdx), integrating an OAuth API will require [Setup OAuth and SSO](../advanced/27_setup_oauth/index.mdx). Your resource can be used [passed as parameters](../core_concepts/3_resources_and_types/index.mdx#passing-resources-as-parameters-to-scripts-preferred) or [directly fetched](../core_concepts/3_resources_and_types/index.mdx#fetching-them-from-within-a-script-by-using-the-wmill-client-in-the-respective-language) within [scripts](../script_editor/index.mdx), [flows](../flows/1_flow_editor.mdx), [low-code apps](../apps/0_app_editor/index.mdx) and [full-code apps](../full_code_apps/index.mdx). > Example of a Supabase resource being used in two different manners from a script in Windmill. ## Native triggers You can use [native triggers](../triggers/11_native_triggers/index.mdx) to automatically run scripts or flows when calendar events are created, updated, or deleted. Native triggers receive real-time push notifications so your runnables execute as soon as events occur. :::tip Find some pre-set interactions with Google Calendar on the [Hub](https://hub.windmill.dev/?app=gcal). Feel free to create your own Google Calendar scripts on [Windmill](../getting_started/00_how_to_use_windmill/index.mdx). ::: --- ## Gcp Source: https://www.windmill.dev/docs/integrations/gcp # Google Cloud Platform integration [GCP](https://cloud.google.com/gcp) is a suite of cloud computing services for building and deploying applications. To integrate GCP to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). ![Add Google Cloud Platform Resource](../assets/integrations/add-gcp.png.webp) :::info Self-host If you're looking for a way to self-host Windmill using GCP, see [Self-Host Windmill](../advanced/1_self_host/index.mdx). ::: | Property | Type | Description | Default | Required | Where to Find | | --------------------------- | ------ | ---------------------------------------------------- | ------- | -------- | ------------------------------------------------------------------------------------------------- | | type | string | Type of credentials object | | false | Google Cloud Console > APIs & Services > Credentials > Create service account key > JSON key file | | project_id | string | Google Cloud Platform project ID | | false | Google Cloud Console > Home > Project ID | | private_key_id | string | Private key ID for the service account | | false | Google Cloud Console > APIs & Services > Credentials > Create service account key > JSON key file | | private_key | string | Private key for the service account | | false | Google Cloud Console > APIs & Services > Credentials > Create service account key > JSON key file | | client_email | string | Email address associated with the service account | | false | Google Cloud Console > APIs & Services > Credentials > Create service account key > JSON key file | | client_id | string | Client ID for the service account | | false | Google Cloud Console > APIs & Services > Credentials > Create service account key > JSON key file | | auth_uri | string | Authentication URI for the service account | | false | Google Cloud Console > APIs & Services > Credentials > Create service account key > JSON key file | | token_uri | string | Token URI for the service account | | false | Google Cloud Console > APIs & Services > Credentials > Create service account key > JSON key file | | auth_provider_x509_cert_url | string | Auth provider X.509 cert URL for the service account | | false | Google Cloud Console > APIs & Services > Credentials > Create service account key > JSON key file | | client_x509_cert_url | string | Client X.509 cert URL for the service account | | false | Google Cloud Console > APIs & Services > Credentials > Create service account key > JSON key file | --- ## Gdrive Source: https://www.windmill.dev/docs/integrations/gdrive # Google Drive integration [Google Drive](https://drive.google.com/drive/my-drive) is cloud-based storage platform. The Google Drive integration is done through OAuth. You just need to sign in from your Google account on your browser. The access will be automatically saved to the workspace as a [resource](../core_concepts/3_resources_and_types/index.mdx). On [self-hosted instances](../advanced/1_self_host/index.mdx), integrating an OAuth API will require [Setup OAuth and SSO](../advanced/27_setup_oauth/index.mdx). Your resource can be used [passed as parameters](../core_concepts/3_resources_and_types/index.mdx#passing-resources-as-parameters-to-scripts-preferred) or [directly fetched](../core_concepts/3_resources_and_types/index.mdx#fetching-them-from-within-a-script-by-using-the-wmill-client-in-the-respective-language) within [scripts](../script_editor/index.mdx), [flows](../flows/1_flow_editor.mdx), [low-code apps](../apps/0_app_editor/index.mdx) and [full-code apps](../full_code_apps/index.mdx). > Example of a Supabase resource being used in two different manners from a script in Windmill. ## Native triggers You can use [native triggers](../triggers/11_native_triggers/index.mdx) to automatically run scripts or flows when files or folders change in Google Drive. Native triggers receive real-time push notifications so your runnables execute as soon as events occur. :::tip Find some pre-set interactions with Google Drive on the [Hub](https://hub.windmill.dev/?app=gdrive). Feel free to create your own Google Drive scripts on [Windmill](../getting_started/00_how_to_use_windmill/index.mdx). ::: --- ## Git repository Source: https://www.windmill.dev/docs/integrations/git_repository # Git integration [Git](https://git-scm.com/) is a distributed version control system designed to handle everything from small to very large projects with speed and efficiency. Windmill has a dedicated [resource Type](https://hub.windmill.dev/resource_types/135/git_repository) used for [Git sync](../advanced/11_git_sync/index.mdx), to sync Windmill workspace to a remote repository that will automatically be committed and pushed scripts, flows and apps on each [deploy](../core_concepts/0_draft_and_deploy/index.mdx). More: This video shows how to set up a Git repository for a workspace. ## GitHub App Instead of using a long lived personal access token to authenticate with GitHub for [Git sync](../advanced/11_git_sync/index.mdx), you can use the GitHub App to authenticate with GitHub. This allows you to control which repositories can be accessed by your Windmill deployment using a short-live [GitHub app installation token](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/authenticating-as-a-github-app-installation). GitHub App is available under [Windmill Enterprise](/pricing). ![GitHub App](/integrations/visual_elements/github_app_installation.png.webp) ### Network requirements The GitHub App feature requires your Windmill instance to communicate with `https://stats.windmill.dev` to obtain GitHub installation tokens. This is the same endpoint used for [telemetry](../advanced/18_instance_settings/index.mdx#telemetry). If your GitHub organization uses [IP allow lists](https://docs.github.com/en/organizations/keeping-your-organization-secure/managing-security-settings-for-your-organization/managing-allowed-ip-addresses-for-your-organization), you will need to whitelist the IP address of `stats.windmill.dev` to allow it to request installation tokens from GitHub on behalf of your Windmill instance. Contact support@windmill.dev to get the current IP address. :::info This network requirement only applies to the Windmill-managed GitHub App. If you use a [self-managed GitHub App](#self-managed-github-app), your Windmill instance communicates directly with your GitHub instance. In that case, if your GitHub organization uses IP allow lists, whitelist your Windmill instance's IP address instead. ::: As a [Windmill workspace admin](../core_concepts/16_roles_and_permissions/index.mdx#admin), you can install the GitHub app to multiple organizations and link them to your Windmill workspaces. Once an app has been installed to a workspace, you can install it to other workspace where you have the admin role. :::warning You will only be able to use the installation token for [Git sync](../advanced/11_git_sync/index.mdx). ::: ### Importing / Exporting to/from other windmill instance A GitHub app can only be installed to a GitHub organization once. Hence to associate an installation to multiple windmill instances you need to export the associated JWT token on the source instance using the "Export" button and paste the JWT in the destination instance to import the installation. :::warning The JWT token associated to your GitHub app installation is sensitive and has the rights to request a short lived installation token. To revoke the JWT, you need to uninstall the GitHub app from your organization and re-install it to re-associate it with a windmill instance. ::: ### Self-managed GitHub App Instead of using the Windmill-managed GitHub App, you can register your own GitHub App on any GitHub instance — whether GitHub.com or a GitHub Enterprise Server (GHES) instance. This gives you full control over the app configuration and removes the dependency on `stats.windmill.dev`, as tokens are exchanged directly between your Windmill instance and your GitHub instance. This feature is [Enterprise Edition](/pricing) only and is configured at the instance level by a [superadmin](../core_concepts/16_roles_and_permissions/index.mdx#superadmin). To set up a self-managed GitHub App: 1. Register a new GitHub App on your GitHub instance (github.com or your GHES instance) 2. In Windmill [Instance Settings](../advanced/18_instance_settings/index.mdx#github-enterprise-app), go to **Advanced > GitHub Enterprise App** and enable "Self-managed GitHub App (for GHES or custom GitHub App)" 3. Fill in the app details: Base URL (e.g. `https://github.com` or your GHES URL), App ID, App Slug, Client ID, and Private Key (PEM) 4. Install the GitHub App to your organization on your GitHub instance Once configured, the self-managed GitHub App can be used for [Git sync](../advanced/11_git_sync/index.mdx) authentication in the same way as the managed GitHub App. Host-based installation filtering ensures tokens are scoped to the correct GitHub instance, preventing token leakage across instances. ![GitHub Enterprise App settings](./ghes_app_settings.png.webp) --- ## Github Source: https://www.windmill.dev/docs/integrations/github # GitHub integration [GitHub](https://github.com/) is a web-based platform for version control and collaboration. The GitHub integration is done through OAuth. You just need to sign in from your GitHub account on your browser. The access will be automatically saved to the workspace as a [resource](../core_concepts/3_resources_and_types/index.mdx). On [self-hosted instances](../advanced/1_self_host/index.mdx), integrating an OAuth API will require [Setup OAuth and SSO](../advanced/27_setup_oauth/index.mdx). You might also look for how to [version control from GitHub / GitLab](../advanced/9_deploy_gh_gl/index.mdx). --- ## Gitlab Source: https://www.windmill.dev/docs/integrations/gitlab # GitLab integration [GitLab](https://about.gitlab.com/) is a web-based Git-repository manager with CI/CD capabilities. The GitLab integration is done through OAuth. You just need to sign in from your GitLab account on your browser. The access will be automatically saved to the workspace as a [resource](../core_concepts/3_resources_and_types/index.mdx). On [self-hosted instances](../advanced/1_self_host/index.mdx), integrating an OAuth API will require [Setup OAuth and SSO](../advanced/27_setup_oauth/index.mdx). ![Add GitLab Resource](../assets/integrations/add-gitlab.png.webp) --- ## Gmail Source: https://www.windmill.dev/docs/integrations/gmail # Gmail integration [Gmail](https://mail.google.com/mail/) is a free email service provided by Google. The Gmail integration is done through OAuth. You just need to sign in from your Google account on your browser. The access will be automatically saved to the workspace as a [resource](../core_concepts/3_resources_and_types/index.mdx). On [self-hosted instances](../advanced/1_self_host/index.mdx), integrating an OAuth API will require [Setup OAuth and SSO](../advanced/27_setup_oauth/index.mdx). --- ## Google cloud storage Source: https://www.windmill.dev/docs/integrations/google-cloud-storage # Google Cloud Storage integration [Google Cloud Storage](https://cloud.google.com/storage) is Google's cloud storage service, an alternative to S3. :::info Windmill for data pipelines You can link a Windmill workspace to a Google Cloud Storage bucket and use it as source and/or target of your processing steps seamlessly, without any boilerplate. See [Windmill for data pipelines](../core_concepts/27_data_pipelines/index.mdx) for more details. ::: To integrate Google Cloud Storage to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). | Property | Type | Description | Default | Required | Where to Find | Additional Details | | --------- | ------- | ---------------------------- | ------- | --------------- | -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | bucket | string | Google Cloud Storage bucket name | | true | Google Cloud Console > Storage > Buckets | Name of the GCS bucket | | serviceAccountKey | resource | Service Account Key | | true | Google Cloud Console > IAM & Admin > Service Accounts | Reference to a `gcloud` resource | --- ## Gsheets Source: https://www.windmill.dev/docs/integrations/gsheets # Google Sheets integration [Google Sheets](https://www.google.com/sheets/about/) is an online spreadsheet application. The Google Sheets integration is done through OAuth. You just need to sign in from your Google account on your browser. The access will be automatically saved to the workspace as a [resource](../core_concepts/3_resources_and_types/index.mdx). On [self-hosted instances](../advanced/1_self_host/index.mdx), integrating an OAuth API will require [Setup OAuth and SSO](../advanced/27_setup_oauth/index.mdx). --- ## Gworkspace Source: https://www.windmill.dev/docs/integrations/gworkspace # Google Workspace integration [Google Workspace](https://workspace.google.com/) is Google's suite of cloud collaboration tools. The `gworkspace` resource type provides OAuth access to the Google Admin Directory API for managing users, groups, org units, and security settings. The Google Workspace integration is done through OAuth. You just need to sign in from your Google account on your browser. The access will be automatically saved to the workspace as a [resource](../core_concepts/3_resources_and_types/index.mdx). The default OAuth scopes are: - `https://www.googleapis.com/auth/admin.directory.group` - `https://www.googleapis.com/auth/admin.directory.user` - `https://www.googleapis.com/auth/admin.directory.user.security` - `https://www.googleapis.com/auth/admin.directory.orgunit` On [self-hosted instances](../advanced/1_self_host/index.mdx), integrating an OAuth API will require [Setup OAuth and SSO](../advanced/27_setup_oauth/index.mdx). Your resource can be used [passed as parameters](../core_concepts/3_resources_and_types/index.mdx#passing-resources-as-parameters-to-scripts-preferred) or [directly fetched](../core_concepts/3_resources_and_types/index.mdx#fetching-them-from-within-a-script-by-using-the-wmill-client-in-the-respective-language) within [scripts](../script_editor/index.mdx), [flows](../flows/1_flow_editor.mdx), [low-code apps](../apps/0_app_editor/index.mdx) and [full-code apps](../full_code_apps/index.mdx). > Example of a Supabase resource being used in two different manners from a script in Windmill. ## Native triggers The `gworkspace` resource type is also used by Google [native triggers](../triggers/11_native_triggers/index.mdx) to watch for changes in Google Drive and Google Calendar. When configured through the native triggers workspace integration, the resource is created with different scopes (`drive.readonly`, `calendar.readonly`, `calendar.events`) tailored to receiving push notifications. :::tip Find some pre-set interactions with Google Workspace on the [Hub](https://hub.windmill.dev/?app=gworkspace). Feel free to create your own Google Workspace scripts on [Windmill](../getting_started/00_how_to_use_windmill/index.mdx). ::: --- ## Hubspot Source: https://www.windmill.dev/docs/integrations/hubspot # HubSpot integration [HubSpot](https://www.hubspot.com/) is an inbound marketing, sales, and customer service platform. To integrate HubSpot to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). ![Add HubSpot Resource](../assets/integrations/add-hubspot.png.webp) | Property | Type | Description | Default | Required | Where to Find | | -------- | ------ | ----------------------------- | ------- | -------- | ---------------------------------------------------------------------- | | token | string | Access token of a private app | | false | HubSpot > Settings > Integrations > API key > Create private app token | --- ## Integrations on windmill Source: https://www.windmill.dev/docs/integrations/integrations_on_windmill # Integrations on Windmill Integrations are key on Windmill as they allow databases (internal & external) and service providers to interact. Using Windmill, integrations are referred to as [resources and resource types](../core_concepts/3_resources_and_types/index.mdx). Each Resource has a Resource Type (RT for short) - for example MySQL, MongoDB, Slack, etc. - that defines the schema that the resource needs to implement. We already have pre-set integrations (or resource types), the list is available on our [Hub](https://hub.windmill.dev/resource_types) (200+ resource types), [using Windmill](../intro.mdx) (most up-to-date version), and at the end of this article. ## How third-party systems connect Windmill does not ship vendor-branded connectors. Any external system - database, SaaS API, queue, internal service, network device, cloud provider - is reached the same way: - A [resource type](../core_concepts/3_resources_and_types/index.mdx#create-a-resource-type) defines the JSON schema (host, credentials, options) for that system. Pick one of the 200+ resource types on the [Hub](https://hub.windmill.dev/resource_types), or define your own. - A resource holds an instance of that schema, with [variables and secrets](../core_concepts/2_variables_and_secrets/index.mdx) inlined for credentials. - Scripts in any [supported language](../getting_started/0_scripts_quickstart/index.mdx) receive the resource as a typed parameter and call the system using whatever client library or CLI is appropriate. A few cases worth calling out: - **Network devices** (switches, routers, firewalls) are driven from regular scripts using the ecosystem's standard tooling: [Ansible](../getting_started/0_scripts_quickstart/10_ansible_quickstart/index.mdx) (native step in Windmill), or Python libraries such as NAPALM, Netmiko or Scrapli imported in a [Python script](../getting_started/0_scripts_quickstart/2_python_quickstart/index.mdx). Device credentials live in a custom resource type. - **Terraform** has no native step in Windmill. Run it as a CLI from a [Bash script](../getting_started/0_scripts_quickstart/4_bash_quickstart/index.mdx) on a worker where the binary has been provisioned via [preinstall binaries](../advanced/8_preinstall_binaries/index.mdx) or a custom worker image. How resources and resource types work in Windmill: ![Recap Resources and Types](../core_concepts/3_resources_and_types/recap_resources_and_types.png "Recap Resources and Types") ## Using integrations Interacting with an integration means using a resource, see the dedicated doc part: ## You feel one integration is missing? ### Create one You can [create a resource type](../core_concepts/3_resources_and_types/index.mdx#create-a-resource-type). Use the "Add Property" button to add a field to the resource type. You can specify constraints for the field (a type, making it mandatory, specifying a default, etc). You can also view the schema by toggling the "As JSON" option. Once you're comfortable with the new integration, we would be super grateful if you could [share it on Hub](../misc/1_share_on_hub/index.md). You will be asked to fill Name, Integration (the corresponding service it interacts with) and Schema (the JSON Schema of the Resource Type). Verified Resource types on the Hub are directly added to the list of available Resource types on each new Windmill instance synced with the Hub. ![Share resource type](../core_concepts/3_resources_and_types//new_resource_type_hub.png) ### Ask for one Just [reach out to us](../misc/6_getting_help/index.mdx), we'll be happy to listen to your needs and add your request to the list of integrations. ## List of integrations ### OAuth APIs On [self-hosted instances](../advanced/1_self_host/index.mdx), integrating OAuth APIs will require [Setup OAuth and SSO](../advanced/27_setup_oauth/index.mdx). | Service Provider | Description | | --------------------------------------- | -------------------------------------------------------- | | [Google Calendar (gcal)](./gcal.md) | Time-management and scheduling web application | | [Google Drive (gdrive)](./gdrive.md) | Cloud-based storage platform | | [Gmail](./gmail.md) | Free email service provided by Google | | [Google Sheets (gsheets)](./gsheets.md) | Online spreadsheet application | | [Google Workspace (gworkspace)](./gworkspace.md) | Manage Google Workspace users, groups, org units and security via the Admin Directory API | | [GitHub](./github.mdx) | Web-based platform for version control and collaboration | | [GitLab](./gitlab.mdx) | Web-based Git-repository manager with CI/CD capabilities | | [LinkedIn](./linkedin.md) | Professional networking and career development platform | | [Slack](./slack.mdx) | Instant messaging and collaboration platform | | [Microsoft Teams](./teams.mdx) | Team collaboration and communication platform | ### Non OAuth APIs & resources | Service Provider | Description | | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | | [Airtable](./airtable.md) | Cloud collaboration platform for organizing and managing data | | [Appwrite](./appwrite.md) | End-to-end backend server for web and mobile apps | | [AWS](./aws.md) | Cloud computing platform offering various services like computing, storage and databases | | [AWS S3](./aws-s3.mdx) | Cloud storage service | | [BigQuery](./bigquery.mdx) | Cloud-based data warehousing platform | | [ClickHouse](./clickhouse.md) | Open-source column-oriented database management system | | [Cloudflare R2](./cloudflare-r2.mdx) | Cloud object storage service for data-intensive applications | | [Datadog](./datadog.md) | Monitoring and analytics platform for cloud-scale infrastructure and applications | | [Discord](./discord.md) | Voice, video, and text communication platform for gamers | | [DuckDB](./duckdb.md) | Open-source, in-process SQL OLAP database management system | | [FaunaDB](./faunadb.md) | Deprecated - the Fauna cloud service was shut down in May 2025 | | [Funkwhale](./funkwhale.md) | Open-source music streaming and sharing platform | | [Git repository](./git_repository.mdx) | Remote git repository for distributed version control systems | | [Google Cloud Platform (gcp)](./gcp.md) | Suite of cloud computing services for building and deploying applications | | [Google Cloud Storage](./google-cloud-storage.mdx) | Google's cloud storage service, an alternative to S3 | | [HubSpot](./hubspot.md) | Inbound marketing, sales, and customer service platform | | [Linear](./linear.md) | Project management tool for software development teams | | [Linkding](./linkding.md) | Self-hosted bookmark manager | | [Mailchimp](./mailchimp.md) | All-in-one marketing platform for small businesses | | [Mailchimp Mandrill](./mailchimp_mandrill.md) | Delivery service for transactional emails from websites and applications | | [Mastodon](./mastodon.md) | Open-source, decentralized social network | | [Matrix](./matrix.md) | Open standard for decentralized, real-time communication | | [Microsoft Azure Blob](./microsoft-azure-blob.md) | Microsoft's cloud storage service, an alternative to S3 | | [Microsoft Excel](./excel.mdx) | Microsoft's spreadsheet application | | [MongoDB](./mongodb.md) | NoSQL document-oriented database | | [MQTT](./mqtt.md) | Lightweight messaging protocol for small sensors and mobile devices | | [MS SQL](./mssql.md) | Database management system | | [MySQL](./mysql.md) | Open-source relational database management system | | [Neon](./neon.md) | Serverless Postgres database | | [Nextcloud](./nextcloud.md) | Suite of client-server software for creating and using file hosting services | | [Notion](./notion.md) | Productivity and note-taking web application | | [OpenAI](./openai.md) | Artificial Intelligence service provider | | [PostgreSQL](./postgresql.md) | Open-source object-relational database management system | | [Raycast](./raycast.mdx) | Application launcher and productivity software developed for macOS | | [Redis](./redis.md) | In-memory data structure store used as a database, cache, and message broker | | [Redshift](./redshift.mdx) | Fully managed, scalable data warehouse service designed for large-scale data storage and analytical processing. | | [RSS](./rss.md) | Web feed that allows users and applications to access updates to websites | | [S3 compatible APIs](./s3.mdx) | Cloud-based object storage service designed to store and retrieve any amount of data | | [SendGrid](./sendgrid.md) | Email API and delivery service | | [SMTP](./smtp.md) | Internet standard for electronic mail transmission | | [Snowflake](./snowflake.mdx) | Cloud-based data warehousing platform | | [Square](./square.md) | Payment and financial services provider | | [Stripe](./stripe.md) | Payment processing platform | | [Supabase](./supabase.md) | Open-source Firebase alternative | | [SurrealDB](./surrealdb.md) | Cloud-hosted NoSQL database | | [Svix](./svix.mdx) | Webhooks as a service | | [Telegram](./telegram.md) | Cloud-based instant messaging and voice over IP service | | [Tigris](./tigris.mdx) | S3-compatible object storage service with edge caching and zero egress fees | | [Toggl](./toggl.md) | Time tracking software | | [Upstash](./upstash.md) | Serverless and low-latency Redis-compatible data store for modern applications | --- ## Linear Source: https://www.windmill.dev/docs/integrations/linear # Linear integration [Linear](https://linear.app/) is a project management tool for software development teams. To integrate Linear to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). ![Add Linear Resource](../assets/integrations/add-linear.png "Add Linear Resource") | Property | Type | Description | Default | Required | Where to Find | | -------- | ------ | ----------------------------------------------------------------------- | ------- | -------- | -------------------------------------------------------------------------------------------------- | | apiKey | string | The API key for the Linear API. | https://linear.app/settings/api | false | | --- ## Linkding Source: https://www.windmill.dev/docs/integrations/linkding # Linkding integration [Linkding](https://github.com/sissbruecker/linkding) is a self-hosted bookmark manager. To integrate Linkding to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). ![Add Linkding Resource](../assets/integrations/add-linkding.png.webp) | Property | Type | Description | Default | Required | Where to Find | | -------- | ------ | ------------------------------- | ------- | -------- | --------------------------------------------- | | token | string | API token to connect to the API | | true | Linkding > User Settings > Generate API token | | baseUrl | string | The base URL of the instance | | false | Provided by your Linkding hosting provider | --- ## Linkedin Source: https://www.windmill.dev/docs/integrations/linkedin # LinkedIn integration [LinkedIn](https://www.linkedin.com/) is a professional networking and career development platform. The LinkedIn integration is done through OAuth. You just need to sign in from your LinkedIn account on your browser. The access will be automatically saved to the workspace as a [resource](../core_concepts/3_resources_and_types/index.mdx). On [self-hosted instances](../advanced/1_self_host/index.mdx), integrating an OAuth API will require [Setup OAuth and SSO](../advanced/27_setup_oauth/index.mdx). --- ## Mailchimp Source: https://www.windmill.dev/docs/integrations/mailchimp # Mailchimp integration [Mailchimp](https://mailchimp.com/) is an all-in-one marketing platform for small businesses. :::info Using emails to trigger scripts & flows To trigger scripts and flows by emails using Mailchimp, refer to the [Mailchimp Mandrill Integration](./mailchimp_mandrill.md) for seamless integration. ::: To integrate Mailchimp to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). ![Add Mailchimp Resource](../assets/integrations/add-mailchimp.png.webp) | Property | Type | Description | Default | Required | Where to Find | | -------- | ------ | ------------------------------------------------------- | ------- | -------- | ------------------------------------------------------ | | api_key | string | Mailchimp API key | | false | Mailchimp > Account > Extras > API keys > Create A Key | | server | string | The data center for your Mailchimp account (e.g., us12) | | false | Found in your API key (e.g., "us12" in "123abc-us12") | --- ## Mailchimp mandrill Source: https://www.windmill.dev/docs/integrations/mailchimp_mandrill # Mailchimp Mandrill integration [Mailchimp Mandrill](https://mailchimp.com/en/features/transactional-email/) is a delivery service for transactional emails from websites and application. Integrating Mailchimp Mandrill is a powerful way of [triggering scripts and flows](../triggers/index.mdx) by e-mail. :::info TLDR The present tutorial explains how to use Mailchimp to trigger Windmill [scripts](../getting_started/0_scripts_quickstart/1_typescript_quickstart/index.mdx) and [flows](../getting_started/6_flows_quickstart/index.mdx) through sending parsed inbound emails to Windmill via [webhooks](../core_concepts/4_webhooks/index.mdx). Note that Windmill now natively supports [email triggers](../triggers/1_email_triggers/index.mdx) through SMTP. ::: ## Webhooks in Windmill [Webhooks](../core_concepts/4_webhooks/index.mdx) in Windmill are a versatile and efficient method for triggering scripts or flows based on external events. Every script or flow created within the platform is automatically assigned a set of autogenerated webhooks, which can be found on the "Detail" page of the script/flow. Combined with a token created in Windmill, these webhooks can be interacted with using standard web technologies, making them compatible with a broad range of external systems and services, including Mailchimp Mandrill for email-triggered executions. ![Webhook endpoints](../core_concepts/4_webhooks/webhook_endpoints.png) > Each script or flow has webhook endpoints. Bearer token must be passed as either an Authorization: Bearer <TOKEN> header, or as a token query parameter: https://<instance>/<route>?token=<TOKEN> ## Using Mailchimp to trigger Windmill webhooks from emails All the details on the Mailchimp side are explained in [this tutorial](https://mailchimp.com/developer/transactional/guides/set-up-inbound-email-processing/). :::tip The flow we used for the example is available on [Windmill Hub](https://hub.windmill.dev/flows/41/). ::: Here are the steps to follow: 1. Create a [script](../getting_started/0_scripts_quickstart/1_typescript_quickstart/index.mdx) or [flow](../getting_started/6_flows_quickstart/index.mdx) in Windmill. Make sure it has an input designed to receive the parsed results (for example a string called "mandrill_events"). 2. [Sign up to Mailchimp](https://mailchimp.com/signup/) (we used the free trial at first). 3. Go to the [inbound menu of Mailchimp Mandrill](https://mandrillapp.com/inbound). 4. Add a domain or sub-domain. For example `webhooks.domain.com`. 5. Add the MX records to your DNS provider and validate them from Mandrill. 6. From Mandrill, click the dropdown next to the `Test DNS Settings` button, select `Routes`, and then select the `Add New Route button`. Configure a new route. That will be the accepted email(s) to trigger your script or flow. Please note that these e-mail addresses do not have to pre-exist to be treated by Mailchimp. 7. From Windmill, go to the `Details` menu of your script or flow. If not any, create a token. Pick a webhook (`UUID/Async` [is commonly preferred](../core_concepts/4_webhooks/index.mdx#synchronous)). 8. In Mandrill's "Post to URL" box, paste the webhook in the form `https://app.windmill.dev/.../rest_of_the_webhook/?token=TOKEN`. 9. Click the `Test DNS Settings` button to check it's working. Now you're all set! Maybe you want to deal with the specific elements of the email. It is likely the payload has been sent as a string, so add a parsing to json step and use its results as inputs for further steps. Parses payload to Json. Code below: ```js } ``` With "x" = our only input `flow_input.mandrill_events`. ### Extract information from directly routed e-mails If your routing rules are set to capture emails that are directly addressed to certain recipients (for instance, any emails sent to name@company.com), extracting the relevant information from these emails is straightforward. It works as followed: - Mailchimp already parses the e-mail's details. - The parsed results are sent as an array, of which Windmill treats each element as a potential output & input. :::info Mailbox Routing In email systems, routing rules are not strictly tied to a single email address. Instead, they are flexible mechanisms that govern how an email server handles incoming messages based on various criteria. A Mailbox Route allows at the server-level incoming emails to be automatically directed to specific destinations based on predefined rules. These rules match criteria like recipient address or subject line, and the system performs actions such as delivering the message to a mailbox, forwarding it, or applying filters Thus, you can use the Mailchimp address effectively and have most emails be parsed directly without anyone knowing there is a Mailchimp domain. For instance: all emails sent to sales@domain.com and joe@domain.com will also be sent to windmill@windmill.domain.com to be analyzed and parsed by Windmill and if deemed relevant added to the CRM. ::: Then you just have to pick the details you want by "connecting inputs" of the JSON.parse step. ![Get details from email](../assets/integrations/mailchimp_email_inputs.png.webp) > _On the bottom right corner, relevant details from the email to be picked as inputs for further steps_ ### Extract information from forwarded e-mails If your routing rules are designed to process emails that are forwarded from certain recipients (for instance, any emails forwarded by name@company.com), you'll need an additional step to extract the relevant information from the original emails. Indeed, Mailchimp already parses the e-mail's details but does it as a single e-mail. So for example the considered sender will be the one who forwarded the e-mail, not the sender of the original e-mail. So we recommend you to add the following step an connecting "input_email" to `results.c[0].msg.text`, c being the JSON parser step: Example of a simple parser to get info from the forwarded email. Code below: ```python import re def main(input_email): from_pattern = re.compile(r'From: .+ <(.+)>') subject_pattern = re.compile(r'Subject: (.+)') date_pattern = re.compile(r'Date: (.+)') to_pattern = re.compile(r'To: <(.+)>') content_pattern = re.compile(r'\n\n(.*)\n', re.DOTALL) from_field = re.search(from_pattern, input_email) date_field = re.search(date_pattern, input_email) subject_field = re.search(subject_pattern, input_email) to_field = re.search(to_pattern, input_email) content_field = re.search(content_pattern, input_email) return { 'from': from_field.group(1) if from_field else None, 'date': date_field.group(1) if date_field else None, 'subject': subject_field.group(1) if subject_field else None, 'to': to_field.group(1) if to_field else None, 'content': content_field.group(1).strip() if content_field else None } ``` ![Get details from email](../assets/integrations/mailchimp_forward_inputs.png.webp) > _On the bottom right corner, relevant details parsed from the forwarded email to be picked as inputs for further steps_ ### Extract information from both directly routed and forwarded e-mails At last, you may be in a situation where you want to extract the relevant details for both forwarded and directly routed e-mails. Meaning: - if A directly sent e-mail, treat contact details of A - if A forwarded e-mail from B, treat contact details of B Then you have two solutions: 1. From Mailchimp, set-up two routes (and therefore, two workflows and two different emails configurations) to handle each cases. It has the advantage of being clear but it will be poor to handle human mistakes on the right address to send to. 2. From Windmill, use [branches](../flows/13_flow_branches.md) to condition the behaviour of the workflow. For example, have a script that reads the subject of the e-mail and returns a value if it contains "Fwd", value on which a conditional branch will depend. Example of a script that returns true if the mail Topic contains given value. Code below: ```js return { containsSubstring }; } interface Output { containsSubstring: boolean; } ``` Here is what it does when "input" = `results.c[0].msg.headers.Subject` (c being the JSON.parse script) and string = "Fwd": In this flow, when "Fwd" is not found, it considers it was not a forwarded e-mail and the default branch executes. Of course this can be customized to your needs. ## How to go further? The present example is a very simple use case: when an email is transferred to a given address, it triggers a flow that reports the main details of the mail to Slack. However, you could go with much more complex workflows: - Play with branches [branches](../flows/13_flow_branches.md) and have this flow report on different media depending of the content of the e-mail. - The email could be dealt with to automatically update a CRM (e.g. [Hubspot](https://hub.windmill.dev/?app=hubspot), [Salesforce](https://hub.windmill.dev/?app=salesforce), [Airtable](https://hub.windmill.dev/?app=airtable)). - Use the parsed details to [branch the execution of your flow](../flows/13_flow_branches.md) on a condition. - Have [OpenAI](https://hub.windmill.dev/?app=openai) summarize the content of the email. - Invoicing and Accounting: If your business receives invoices or receipts via email, you can parse these emails to extract relevant information and automatically update your accounting software or database. - Monitoring and Alerting: If you're receiving system or application alerts via email, you can parse these emails and trigger specific workflows based on the type of alert. For example, you could automatically create a task in a project management tool, send a message in a Slack channel, or even trigger a script to attempt to resolve the issue automatically. On top of much more custom-made uses only you can think of. --- ## Mastodon Source: https://www.windmill.dev/docs/integrations/mastodon # Mastodon integration [Mastodon](https://mastodon.social/) is an open-source, decentralized social network. To integrate Mastodon to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). ![Add Mastodon Resource](../assets/integrations/add-mastodon.png.webp) | Property | Type | Description | Default | Required | Where to Find | | -------- | ------ | ----------------------------------------------------------------------- | ------- | -------- | -------------------------------------------------------------------------------------------------- | | baseUrl | string | The URL of the Mastodon instance (e.g., "https://mastodon.example.com") | | true | Provided by your Mastodon hosting provider or Mastodon instance URL for self-hosted instances | | token | string | An access token to act as a logged-in user | | false | Mastodon > Preferences > Development > Your Applications > New Application > Generate access token | --- ## Matrix Source: https://www.windmill.dev/docs/integrations/matrix # Matrix integration [Matrix](https://matrix.org/) is an open standard for decentralized, real-time communication. To integrate Matrix to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). ![Add Matrix Resource](../assets/integrations/add-matrix.png.webp) | Property | Type | Description | Default | Required | Where to Find | | -------- | ------ | --------------------------------------------------------------- | ------- | -------- | ----------------------------------------------------------------------------------------- | | baseUrl | string | The URL of a Matrix server (e.g., "https://matrix.example.com") | | true | Provided by your Matrix hosting provider or Matrix instance URL for self-hosted instances | | token | string | An access token to act as a logged-in user | | false | Matrix > Settings > Security & Privacy > Access Token > Reveal Access Token | --- ## Microsoft azure blob Source: https://www.windmill.dev/docs/integrations/microsoft-azure-blob # Microsoft Azure Blob integration [Microsoft Azure Blob](https://azure.microsoft.com/products/storage/blobs) is Microsoft's cloud storage service, an alternative to S3. :::info Windmill for data pipelines You can link a Windmill workspace to an Azure Blob storage account and use it as source and/or target of your processing steps seamlessly, without any boilerplate. See [Windmill for data pipelines](../core_concepts/27_data_pipelines/index.mdx) for more details. ::: To integrate Microsoft Azure Blob Storage to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). | Property | Type | Description | Default | Required | Where to Find | Additional Details | | --------- | ------- | ---------------------------- | ------- | --------------- | -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | accountName | string | Azure Account Name | | true | Azure Portal > Storage account | Name of the Storage Account | | containerName | string | Name of the container holding the data | | true | Azure Portal > Storage account > Containers | Each storage account can have multiple containers. Choose one to plug to use inside Windmill | | useSSL | boolean | Use SSL for connections | true | true | Whether the endpoint is using HTTPS or HTTP | Unless you're hosting you own Blob storage, the ones hosted on Azure all uses HTTPS | | endPoint | string | Azure Blob Storage endpoint | | false | The endpoint of Azure's Blob storage | unless you're hosting your own Blob Storage or using proxies, this can be left empty | | accessKey | string | Azure Blob Access Key | | false | Azure Portal > Storage account > Access Keys | Access Key to use to authenticate API calls | --- ## Mongodb Source: https://www.windmill.dev/docs/integrations/mongodb # MongoDB integration [MongoDB](https://www.mongodb.com/) is a NoSQL document-oriented database. This guide shows how to create a connection from your Windmill instance to a MongoDB database (self-hosted or [MongoDB Atlas][mongodb-atlas]), then use it to make queries with the official MongoDB drivers. :::caution Atlas Data API deprecation Previous versions of this guide relied on the MongoDB Atlas Data API through the `mongodb_rest` resource type. MongoDB [shut down the Atlas Data API][mongo-data-api-eol] on September 30, 2025, so that resource type no longer works. Use the `mongodb` resource type with the official drivers as shown below instead. ::: ![Integration between MongoDB and Windmill](../assets/integrations/0-header.png.webp 'Connect a MongoDB database with Windmill') ## Create resource Windmill provides integration with many different apps and services with the use of [Resources][docs-resources]. Each Resource has a **Resource type**, which controls the shape of it. To be able to connect to a MongoDB instance, we'll need to define a Resource with the [mongodb](https://hub.windmill.dev/resource_types/22/mongodb) Resource Type. :::tip You can find a list of all the officially supported Resource types on [Windmill Hub][hub-resources]. ::: Head to the Resources page in the Windmill app, click on "Add resource" in the top right corner and select the `mongodb` type, then provide the following parameters: | Property | Type | Description | Default | Required | Where to Find | Additional Details | | ------------------ | ------- | -------------------------- | ----------- | -------- | ----------------------- | --------------------------------------------------- | | db | string | Database name | | true | MongoDB Atlas Dashboard | Name of the database you want to connect to | | tls | boolean | Use TLS for connections | true | false | Your own preference | Set to true for secure connections | | servers | array | Array of server objects | | true | MongoDB Atlas Dashboard | Each server object should contain `host` and `port` | | host (nested) | string | Server address | | true | MongoDB Atlas Dashboard | Hostname of the MongoDB instance | | port (nested) | integer | Port number | 27017 | false | MongoDB Atlas Dashboard | Default MongoDB port is `27017` | | credential | object | Authentication information | | true | MongoDB Atlas Dashboard | Contains `username`, `password`, `db`, `mechanism` | | username (nested) | string | Database username | | true | MongoDB Atlas Dashboard | Your database user's username | | password (nested) | string | Database password | | true | MongoDB Atlas Dashboard | Your database user's password | | db (nested) | string | Authentication database | | true | MongoDB Atlas Dashboard | The database used for authentication | | mechanism (nested) | string | Authentication mechanism | SCRAM-SHA-1 | false | Your own preference | Default authentication mechanism is `"SCRAM-SHA-1"` | On MongoDB Atlas, you can find the hostnames of your cluster from the Atlas dashboard under "Connect" > "Drivers": they are the hosts listed in the connection string. ## Create script Next, let's create a Script that uses the newly created Resource. Head on to the [Home][wm-app-home] page, click **New** and select **Script**. The examples below query a collection and return the matching documents, with support for querying by `_id` (which is stored as an ObjectId, a special type in MongoDB that needs an explicit conversion). In TypeScript (Bun), using the official [mongodb npm driver](https://www.npmjs.com/package/mongodb): ```typescript type Mongodb = { db: string; tls: boolean; servers: { host: string; port: number }[]; credential: { username: string; password: string; db: string; mechanism: string }; }; export async function main( auth: Mongodb, collection: string, filter: Record ) { const hosts = auth.servers.map((s) => `${s.host}:${s.port ?? 27017}`).join(','); const client = new MongoClient(`mongodb://${hosts}`, { tls: auth.tls, auth: { username: auth.credential.username, password: auth.credential.password }, authSource: auth.credential.db }); try { if ('_id' in filter) { filter['_id'] = new ObjectId(filter['_id']); } const documents = client.db(auth.db).collection(collection); return await documents.find(filter).toArray(); } finally { await client.close(); } } ``` Or in Python, using [PyMongo](https://pypi.org/project/pymongo/): ```python from pymongo import MongoClient from bson.objectid import ObjectId mongodb = dict def main(auth: mongodb, collection: str, filter: dict): hosts = ",".join( f"{s['host']}:{s.get('port', 27017)}" for s in auth["servers"] ) client = MongoClient( f"mongodb://{hosts}", tls=auth["tls"], username=auth["credential"]["username"], password=auth["credential"]["password"], authSource=auth["credential"]["db"], ) try: if "_id" in filter: filter["_id"] = ObjectId(filter["_id"]) documents = client[auth["db"]][collection] return [{**doc, "_id": str(doc["_id"])} for doc in documents.find(filter)] finally: client.close() ``` In case you are using the [sample dataset][mongo-sample-data] of MongoDB Atlas, you'll have a `sample_restaurants` database filled with restaurants. To make a query for a specific restaurant name, the arguments of the Script would look like the following (**casing matters**): - **auth** - select the Resource we created in the previous step (`my_mongodb`) - **collection** - `restaurants` - **filter** - `{ "name": "Nordic Delicacies" }` (or by ID: `{ "_id": "5eb3d668b31de5d588f4293b" }`) After filling the inputs, try running the Script by clicking "Test" or pressing `Ctrl` + `Enter`. You should see exactly one restaurant returned. :::tip You can find more Script examples related to MongoDB on [Windmill Hub][hub-mongo]. ::: Once you're done, click on "Save", which will save it to your workspace. You can now use this Script in your [Flows][docs-flows], [Apps][docs-apps] or as standalone. [wm-app-resources]: https://app.windmill.dev/resources [wm-app-home]: https://app.windmill.dev [hub-resources]: https://hub.windmill.dev/resource_types [hub-mongo]: https://hub.windmill.dev/?app=mongodb [docs-resources]: /docs/core_concepts/resources_and_types [docs-path]: /docs/core_concepts/roles_and_permissions#path [docs-flows]: /docs/getting_started/flows_quickstart [docs-apps]: /docs/getting_started/apps_quickstart [mongodb-atlas]: https://www.mongodb.com/atlas/database [mongo-data-api-eol]: https://www.mongodb.com/docs/atlas/app-services/data-api/data-api-deprecation/ [mongo-sample-data]: https://www.mongodb.com/docs/atlas/sample-data/ --- ## Mqtt Source: https://www.windmill.dev/docs/integrations/mqtt # MQTT integration [MQTT](https://mqtt.org/) (Message Queuing Telemetry Transport) is a lightweight messaging protocol designed for small sensors and mobile devices optimized for low-bandwidth, high-latency, or unreliable networks. Windmill allows you to create MQTT triggers, enabling subscriptions to a specific MQTT broker. When a message is received on the subscribed topic, the designated script or workflow set at trigger creation will be executed automatically. >This video shows how to set up an MQTT trigger in Windmill. Additionally, the video illustrates the execution of the script linked to the created trigger when a message is received. For more details, please refer to the [MQTT documentation](../triggers/mqtt_triggers). --- ## Mssql Source: https://www.windmill.dev/docs/integrations/mssql # MS SQL integration [MS SQL](https://www.microsoft.com/sql-server/sql-server-downloads) is a database management system. Windmill provides a framework to support MS SQL databases, either with native SQL scripts or through TypeScript for raw queries. ![Integration between MS SQL and Windmill](../assets/integrations/windmill_and_mssql.png 'Connect a MS SQL instance with Windmill') ## Authentication methods Windmill supports multiple authentication methods for MS SQL Server: - **Username/Password**: Standard SQL Server authentication - **Azure AD (Entra)**: OAuth-based authentication for Azure-hosted databases - **Windows Integrated Authentication**: Kerberos-based authentication for Active Directory environments For detailed setup instructions, including Windows Integrated Authentication configuration, refer to the [SQL Getting started section](../getting_started/0_scripts_quickstart/5_sql_quickstart/index.mdx#ms-sql). --- ## Mysql Source: https://www.windmill.dev/docs/integrations/mysql # MySQL integration [MySQL](https://www.mysql.com/) is an open-source relational database management system. Windmill provides a framework to support MySQL databases, either with native SQL scripts or through TypeScript for raw queries. ![Integration between MySQL and Windmill](../assets/integrations/mysql_header.png 'Connect a MySQL instance with Windmill') Please refer to the [SQL Getting started section](../getting_started/0_scripts_quickstart/5_sql_quickstart/index.mdx). --- ## Neon Source: https://www.windmill.dev/docs/integrations/neon # Neon integration [Neon](https://neon.com/) is an open-source cloud database platform that provides fully managed PostgreSQL databases with high availability and scalability. As a Postgres database service provider, Neon follows the regular Postgres protocol and therefore can be integrated as any [PostgreSQL resource](../getting_started/0_scripts_quickstart/5_sql_quickstart/index.mdx). 1. Sign-up to Neon's Cloud App or [Self-Host](https://github.com/neondatabase/neon) it. 2. [Set up a project and add data](https://neon.com/docs/manage/projects). 3. Get a [Connection string](https://neon.com/docs/connect/connect-from-any-app). You can obtain it from the Connect widget on the Neon Dashboard: select a branch, a role, and the database you want to connect to and a connection string will be constructed for you. 4. From Windmill, add your Neon connection string as a [Postgresql resource](https://hub.windmill.dev/resource_types/114/postgresql) and [Execute queries](https://hub.windmill.dev/scripts/postgresql/1294/execute-query-and-return-results-postgresql). :::tip Adding the connection string as a Postgres resource requires to parse it. For example, for `psql postgres://daniel:@ep-restless-rice.us-east-2.aws.neon.tech/neondb`, that would be: ```json { "host": "ep-restless-rice.us-east-2.aws.neon.tech", "port": 5432, "user": "daniel", "dbname": "neondb", "sslmode": "require", "password": "" } ``` Where the sslmode should be "require" and Neon uses the default PostgreSQL port, `5432`. ::: --- ## Nextcloud Source: https://www.windmill.dev/docs/integrations/nextcloud # Nextcloud integration [Nextcloud](https://nextcloud.com/) is a suite of client-server software for creating and using file hosting services. To integrate Nextcloud to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). ![Add Nextcloud Resource](../assets/integrations/add-nextcloud.png.webp) | Property | Type | Description | Default | Required | Where to Find | | -------- | ------ | ----------------------------------------------------------- | ------- | -------- | ------------------------------------------------------- | | username | string | The username for accessing the Nextcloud instance | | true | Your Nextcloud account credentials | | password | string | The password associated with the provided username | | true | Your Nextcloud account credentials | | baseUrl | string | The base URL of the Nextcloud instance (e.g., "https://nextcloud.example.com") | | true | Found in the address bar of your Nextcloud instance | Your resource can be used [passed as parameters](../core_concepts/3_resources_and_types/index.mdx#passing-resources-as-parameters-to-scripts-preferred) or [directly fetched](../core_concepts/3_resources_and_types/index.mdx#fetching-them-from-within-a-script-by-using-the-wmill-client-in-the-respective-language) within [scripts](../script_editor/index.mdx), [flows](../flows/1_flow_editor.mdx), [low-code apps](../apps/0_app_editor/index.mdx) and [full-code apps](../full_code_apps/index.mdx). > Example of a Supabase resource being used in two different manners from a script in Windmill. ## Native triggers You can use [native triggers](../triggers/11_native_triggers/index.mdx) to automatically run scripts or flows when files or folders change on your Nextcloud instance. Native triggers receive real-time push notifications so your runnables execute as soon as events occur. :::tip Find some pre-set interactions with Nextcloud on the [Hub](https://hub.windmill.dev/?app=nextcloud). Feel free to create your own Nextcloud scripts on [Windmill](../getting_started/00_how_to_use_windmill/index.mdx). ::: --- ## Notion Source: https://www.windmill.dev/docs/integrations/notion # Notion integration [Notion](https://www.notion.so/) is a productivity and note-taking web application. To integrate Notion to Windmill, you need to save a [token](https://developers.notion.com/reference/create-a-token) within a Notion [resource](../core_concepts/3_resources_and_types/index.mdx). ![Add Notion Resource](../assets/integrations/add-notion.png "Add Notion Resource") To create a token, on Notion go to the [My integrations](https://www.notion.so/my-integrations) page, create a new integration and associate it with the proper workspace. Do not forget to "Connect to" Windmill the page (and therefore subpages) for which you want Windmill to have access to. Your resource can then be used [passed as parameters](../core_concepts/3_resources_and_types/index.mdx#passing-resources-as-parameters-to-scripts-preferred) or [directly fetched](../core_concepts/3_resources_and_types/index.mdx#fetching-them-from-within-a-script-by-using-the-wmill-client-in-the-respective-language) within [scripts](../script_editor/index.mdx), [flows](../flows/1_flow_editor.mdx), [low-code apps](../apps/0_app_editor/index.mdx) and [full-code apps](../full_code_apps/index.mdx). --- ## Openai Source: https://www.windmill.dev/docs/integrations/openai # OpenAI integration [OpenAI](https://openai.com/) is an artificial intelligence service provider. :::info Windmill AI An OpenAI resource can also power [Windmill AI](../core_concepts/22_ai_generation/index.mdx) (AI chat in the code and flow editors) and [AI agent steps](../core_concepts/22_ai_generation/index.mdx) in flows. OpenAI is one of the supported AI providers, alongside Anthropic, Google AI, Mistral, Groq, OpenRouter and others. ::: To integrate OpenAI to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). | Property | Type | Description | Default | Required | Where to Find | | --------------- | ------ | -------------------------------------------------------------------------------------------------------------- | ------- | -------- | ---------------------------------------------------------------------- | | api_key | string | API key for OpenAI | | true | OpenAI Dashboard > API Keys > Create new key or view existing keys | | organization_id | string | Only needed for users who belong to multiple organizations and want to use an organization other than default | | false | OpenAI Dashboard > Account Settings > Organizations > Organization ID | | base_url | string | Custom API base URL, for OpenAI-compatible endpoints (Azure OpenAI, local models, proxies) | `https://api.openai.com/v1` | false | Your OpenAI-compatible provider's documentation | --- ## Postgresql Source: https://www.windmill.dev/docs/integrations/postgresql # PostgreSQL integration [PostgreSQL](https://www.postgresql.org/) is an open-source object-relational database management system. Windmill provides a framework to support PostgreSQL databases, either with native SQL scripts or through TypeScript for raw queries. ![Integration between PostgreSQL and Windmill](../assets/integrations/psql-0-header.png.webp 'Connect a PostgreSQL instance with Windmill') Please refer to the [SQL Getting started section](../getting_started/0_scripts_quickstart/5_sql_quickstart/index.mdx). --- ## IAM authentication for AWS RDS and Aurora :::info Enterprise This feature is available on [Windmill Enterprise Edition](/pricing) only. ::: Instead of using static passwords, you can authenticate to AWS RDS or Aurora PostgreSQL databases using [IAM database authentication](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.html). Windmill workers generate short-lived authentication tokens automatically, so no database password needs to be stored in the resource. This works with any of the standard AWS credential methods: - [IRSA](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html) (IAM Roles for Service Accounts) - [EKS Pod Identity](https://docs.aws.amazon.com/eks/latest/userguide/pod-identities.html) - [EC2 Instance Profiles](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html) ### Setup 1. **Enable IAM authentication on your RDS instance.** In the AWS console, go to your RDS instance settings and enable IAM database authentication. 2. **Create a database user with the `rds_iam` role:** ```sql CREATE USER myuser WITH LOGIN; GRANT rds_iam TO myuser; ``` 3. **Grant IAM permissions to your worker.** The IAM principal attached to your Windmill worker (via IRSA, Pod Identity, or Instance Profile) needs the `rds-db:connect` action. Example IAM policy: ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "rds-db:connect", "Resource": "arn:aws:rds-db:::dbuser:/" } ] } ``` 4. **Create a PostgreSQL resource with IAM auth enabled.** Set `use_iam_auth` to `true` and fill in `host`, `user`, and `dbname`. The `password` field is ignored when IAM auth is enabled. ```json { "host": "mydb.cluster-abc123.us-east-1.rds.amazonaws.com", "port": 5432, "user": "myuser", "dbname": "mydb", "sslmode": "require", "use_iam_auth": true, "region": "us-east-1" } ``` The `region` field is optional if the `AWS_REGION` environment variable is set on the worker. SSL is enforced automatically for IAM connections. --- ## Raycast Source: https://www.windmill.dev/docs/integrations/raycast # Raycast integration [Raycast](https://www.raycast.com/) is an application launcher and productivity software developed for macOS. [![RayCast integration](../assets/integrations/windmill_and_raycast.png "RayCast integration")](https://www.raycast.com/emiliobool/windmill) Raycast provided a [guide](https://www.raycast.com/emiliobool/windmill) on how to set up a Windmill Extension to run workflows directly from Raycast. --- ## Redis Source: https://www.windmill.dev/docs/integrations/redis # Redis integration [Redis](https://redis.io/) is an in-memory data structure store used as a database, cache, and message broker. Redis follows the same connection method as [MongoDB](./mongodb.md), providing the database name, TLS settings, server information, and credentials for authentication. ![Add Mongodb](../assets/integrations/add_mongodb.png.webp) Here's a table detailing the properties for Redis integration using the [MongoDB resource type](https://hub.windmill.dev/resource_types/22/mongodb): Here's the table filled out for an Upstash connection: | Property | Type | Description | Default | Required | Where to Find | Additional Details | | ----------------- | ------- | -------------------------- | ------- | -------- | ------------------- | ---------------------------------------------------- | | db | integer | Database index | 0 | false | Upstash Dashboard | Index of the Upstash database you want to connect to | | tls | boolean | Use TLS for connections | true | false | Your own preference | Set to true for secure connections | | servers | array | Array of server objects | | true | Upstash Dashboard | Each server object should contain `host` and `port` | | host (nested) | string | Server address | | true | Upstash Dashboard | Hostname of the Upstash instance | | port (nested) | integer | Port number | 6379 | false | Upstash Dashboard | Default Redis port is `6379` | | credential | object | Authentication information | | true | Upstash Dashboard | Contains `password` | | password (nested) | string | Database password | | true | Upstash Dashboard | Your Upstash server's password | --- ## Redshift Source: https://www.windmill.dev/docs/integrations/redshift # Amazon Redshift integration Amazon [Redshift](https://aws.amazon.com/redshift/) is a fully managed, petabyte-scale data warehouse service in the cloud provided by Amazon Web Services (AWS). It is designed to handle large-scale data storage and complex query processing for data analysis. Windmill provides a framework to support [PostgreSQL](./postgresql.md) databases which makes it compatible with Amazon Redshift. To start using your Redshift instance on Windmill just follow the same steps as for a PostgreSQL database. Please refer to the [SQL Getting started section](../getting_started/0_scripts_quickstart/5_sql_quickstart/index.mdx) for more details. --- ## Rss Source: https://www.windmill.dev/docs/integrations/rss # RSS integration [RSS](https://rss.com/) is a web feed that allows users and applications to access updates to websites. To integrate RSS to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). ![Add RSS Resource](../assets/integrations/add-rss.png.webp) | Property | Type | Description | Default | Required | Where to Find | | -------- | ------ | ----------- | ------- | -------- | --------------------------------------- | | url | string | Feed URL | | true | Provided by the RSS feed source website | --- ## S3 Source: https://www.windmill.dev/docs/integrations/s3 # S3 APIs integrations S3 is a cloud-based object storage service designed to store and retrieve any amount of data. Instance and workspace object storage are different from using S3 resources within scripts, flows, and apps, which is free and unlimited. This is what is [described in this page](#add-a-s3-resource). At the [workspace level](../core_concepts/38_object_storage_in_windmill/index.mdx#workspace-object-storage), what is exclusive to the [Enterprise](/pricing) version is using the integration of Windmill with S3 that is a major convenience layer to enable users to read and write from S3 without having to have access to the credentials. Additionally, for [instance integration](../core_concepts/38_object_storage_in_windmill/index.mdx#instance-object-storage), the Enterprise version offers advanced features such as large-scale log management and distributed dependency caching. Windmill provides a unique [resource type](https://hub.windmill.dev/resource_types/42/) for any API following the typical S3 schema. ## Add a S3 resource Here are the required details: ![S3 resource type](../assets/integrations/add-s3.png.webp) | Property | Type | Description | Default | Required | | --------- | ------- | ----------------------------------------------- | ------- | -------- | | bucket | string | S3 bucket name | | true | | region | string | S3 region for the bucket (e.g. `eu-west-3`) | | true | | endPoint | string | S3 endpoint (e.g. `s3.eu-west-3.amazonaws.com`) | | true | | useSSL | boolean | Use SSL for connections | true | false | | pathStyle | boolean | Use path-style addressing | false | false | | accessKey | string | Access key ID | | false | | secretKey | string | Secret access key | | false | `accessKey` and `secretKey` are optional in the resource type but required by most providers (Amazon S3, Cloudflare R2, Tigris); leave them empty only for setups relying on public buckets or ambient credentials. For guidelines on where to find these details on a given platform, see the provider pages: ## Workspace object storage Once you've created an S3, Azure Blob, or Google Cloud Storage resource in Windmill, you can use Windmill's native integration with S3, Azure Blob, or GCS, making it the recommended storage for large objects like files and binary data. ![Workspace object storage Infographic](../core_concepts/11_persistent_storage/s3_infographics.png "Workspace object storage Infographic") The workspace object storage is exclusive to the [Enterprise](/pricing) edition. It is meant to be a major convenience layer to enable users to read and write from S3 without having to have access to the credentials. ## Instance object storage Under [Enterprise Edition](/pricing), instance object storage offers advanced features to enhance performance and scalability at the [instance](../advanced/18_instance_settings/index.mdx) level. This integration is separate from the [Workspace object storage](#workspace-object-storage) and provides solutions for large-scale log management and distributed dependency caching. ![S3/Azure for Python/Go cache & large logs](../core_concepts/20_jobs/s3_azure_cache.png "S3/Azure for Python/Go cache & large logs") --- ## Sendgrid Source: https://www.windmill.dev/docs/integrations/sendgrid # SendGrid integration [SendGrid](https://sendgrid.com/) is an email API and delivery service. To integrate SendGrid to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). ![Add SendGrid Resource](../assets/integrations/add-sendgrid.png.webp) | Property | Type | Description | Required | Where to find | | -------- | ------ | ----------- | -------- | ---------------------------------------------------------------------------------- | | token | string | API token | true | 1. https://app.sendgrid.com/settings/api_keys 2.Create an API key 3. Copy your key | --- ## Slack Source: https://www.windmill.dev/docs/integrations/slack # Slack integration [Slack](https://slack.com/) is an instant messaging and collaboration platform. There are two ways to build interactions between Slack and Windmill: run code on Windmill by a Slack command or use the Slack API directly from Windmill. In this guide, we'll cover both approaches. ## Action on Windmill from Slack The goal here is to be able to invoke a Windmill Script from Slack, by using `/windmill` command. First, you need to be a **workspace admin**. Then connect either through the UI or the [CLI](../advanced/3_cli/index.mdx): The CLI lets you connect Slack without going through the browser OAuth flow — useful for configure-as-code setups, CI provisioning, and scripted onboarding. It produces the same DB state as the UI flow (workspace_settings fields, `g/slack` group, `slack_bot` folder, encrypted `f/slack_bot/bot_token` variable and resource). First, create a Slack app at [api.slack.com/apps](https://api.slack.com/apps) (or use your existing one). In **OAuth & Permissions**, add the bot scopes Windmill expects (`commands`, `chat:write`, `chat:write.public`, `channels:join`, `files:write`, `app_mentions:read`, `im:history`, `im:read`), then click **Install to Workspace** to get the **Bot User OAuth Token** (`xoxb-…`). Fetch the `team_id` and `team_name` by calling `auth.test`: ```bash curl -s -X POST https://slack.com/api/auth.test \ -H "Authorization: Bearer xoxb-YOUR-TOKEN" ``` Then connect the workspace: ```bash wmill workspace connect-slack \ --bot-token xoxb-... \ --team-id T01234ABC \ --team-name "Your Team Name" ``` The command is admin-only and idempotent — re-running with the same arguments is a no-op, and running with a different team rewrites the binding in place. To disconnect later: ```bash wmill workspace disconnect-slack ``` This clears `slack_team_id` and `slack_name`. To also remove the bot token variable/resource/folder/group, delete the corresponding files from your local sync folder and run `wmill sync push`. At the **instance level**, a super-admin can mint the global bot token (used for critical alerts) similarly: ```bash wmill instance connect-slack \ --bot-token xoxb-... \ --team-id T01234ABC \ --team-name "Your Team Name" ``` :::info Before you can use `wmill {workspace,instance} connect-slack` The OAuth client (`client_id` / `client_secret`) still needs to exist in instance settings so fallback paths and the UI "Connect" button keep working. Configure it via the [instance settings UI](../advanced/18_instance_settings/index.mdx) or by editing `instance_settings.yaml`'s `oauths` entry and running `wmill instance push`. ::: ### Using commands on Slack Once you allow access, you will be redirected to the Slack settings in Windmill. We'll create a command handler Script first, so let's click "Create a script to handle Slack command". You will be navigated to the Script editor. Give your script a name (e.g. `slack_command_handler`), a short summary (e.g. "Slack command handler"). You'll get to [this](https://hub.windmill.dev/scripts/slack/1405/example-of-responding-to-a-slack-command-slack) template: ```typescript export async function main(response_url: string, text: string) { const x = await fetch(response_url, { method: 'POST', body: JSON.stringify({ text: `ROGER ${text}` }) }); const username = process.env['WM_USERNAME']; console.log(`user = ${username}`); } ``` After the Script is deployed, navigate back to the Slack settings Choose the "Script" option for adding a command handler and select your newly created Script. ![Connected settings](../assets/integrations/slack-3-connected.png.webp) Congratulations! You've just created a Slack command handler. Now you can use the `/windmill` command in your Slack workspace to trigger the Script. Try it out with `/windmill foo` and you should get back `ROGER foo`. Go ahead and customize the Script to your needs. In addition to `response_url`, the script/flow can use the following parameters, simply by having them as inputs with the proper name: ``` channel_id user_name user_id command trigger_id api_app_id ``` ![Use the Windmill command](../assets/integrations/slack-5-slack-command.png.webp) ### Using @mentions In addition to slash commands, you can trigger your Windmill scripts by @mentioning the Windmill bot in any channel, thread, or direct message. @mentions work identically to `/windmill` commands - they use the same handler script configured in your workspace settings and pass the same parameters. When you @mention the bot (e.g., `@Windmill hello world`), the bot mention is automatically stripped from the text before being passed to your handler script, so it receives just `hello world` - identical to how `/windmill hello world` would work. @mentions work in: - Public channels (when the bot is invited to the channel) - Private channels (when the bot is invited) - Direct messages with the bot - Message threads (reply to any message and @mention the bot) The `command` parameter will be set to `@mention` for @mention triggers (vs `/windmill` for slash commands), allowing your handler to distinguish between the two if needed. Available parameters for @mention triggers: ``` text // Message text with bot mention stripped channel_id // Channel where message was sent user_id // Slack user ID who sent the message command // Set to "@mention" for @mentions event_id // Unique event identifier ts // Message timestamp thread_ts // Thread timestamp (if in a thread) ``` :::note @mentions are delivered via Slack's Events API, which means your handler must respond within 3 seconds. For longer-running operations, launch a job and send updates to Slack asynchronously using the Slack API. ::: #### Responding to `/` commands vs `@` mentions When handling Slack triggers, the response method differs depending on the trigger type: **For `/windmill` slash commands**: Use the `response_url` parameter (a webhook URL) to send your response back to Slack. **For `@mention` triggers**: Use the Slack Web API with the bot token to post messages to the channel. ```typescript type Slack = { token: string; }; export async function main( text: string, response_url: string, channel_id: string, command: string ) { const responseText = `You said: ${text}`; if (command === '@mention') { // For @mentions: use Slack Web API const slack = await wmill.getResource('f/slack_bot/bot_token'); const web = new WebClient(slack.token); await web.chat.postMessage({ channel: channel_id, text: responseText }); } else { // For /windmill commands: use response_url await fetch(response_url, { method: 'POST', body: JSON.stringify({ text: responseText }) }); } } ``` ```typescript type Slack = { token: string; }; export async function main( text: string, response_url: string, channel_id: string, command: string ) { const responseText = `You said: ${text}`; if (command === '@mention') { // For @mentions: use Slack Web API const slack = await wmill.getResource('f/slack_bot/bot_token'); const web = new WebClient(slack.token); await web.chat.postMessage({ channel: channel_id, text: responseText }); } else { // For /windmill commands: use response_url await fetch(response_url, { method: 'POST', body: JSON.stringify({ text: responseText }) }); } } ``` ```python from slack_sdk import WebClient import wmill import requests import json def main(text: str, response_url: str, channel_id: str, command: str): response_text = f"You said: {text}" if command == "@mention": # For @mentions: use Slack Web API slack = wmill.get_resource("f/slack_bot/bot_token") client = WebClient(token=slack["token"]) client.chat_postMessage( channel=channel_id, text=response_text ) else: # For /windmill commands: use response_url requests.post( response_url, data=json.dumps({"text": response_text}), headers={"Content-Type": "application/json"} ) ``` ### Workspace-level Slack app configuration By default, workspaces use the instance-level Slack app configured by your Windmill administrator. However, workspace admins can optionally configure their own Slack app for their workspace. This provides: - **Workspace isolation**: Each workspace uses its own Slack app and credentials - **Separate rate limits**: Avoid sharing rate limits across workspaces - **Independent management**: Workspace admins can manage their own Slack integration To configure a workspace-specific Slack app: The workspace-level OAuth override lives in `settings.yaml` as two fields that round-trip via `wmill sync pull` / `wmill sync push`: ```yaml slack_oauth_client_id: "1234567890.1234567890" slack_oauth_client_secret: "abcdef0123456789" ``` Set both to values and push to upsert; clear them (delete the lines or set both to `""`) and push to remove the workspace-level override and fall back to the instance-level Slack app. `wmill sync pull` always emits the two fields (`null` when unset), so round-trip is bijective — the committed YAML is a complete snapshot. After the override is in place, use the CLI's non-interactive connect command described above, or the UI's workspace-specific "Save and Connect" button (which authorizes using the workspace-level client). :::note The workspace-level Slack app will only apply to the auomatically managed `f/slack_bot/bot_token` resource used for workspace handlers and slack approvals. For general user specific slack oauth tokens, the instance level OAuth connection will be used. ::: You won't be able to have Slack interact with your [resources](../core_concepts/3_resources_and_types/index.mdx) and [variables](../core_concepts/2_variables_and_secrets/index.mdx) before adding them to the `slack` [group](../core_concepts/8_groups_and_folders/index.mdx#groups) that was automatically created by Windmill after you set up your Slack workspace on Windmill. Tutorial below. How to let Slack use your resources and variables: To give the permission, go to "Resources" (and "Variables") menu, click on `Share`, `Group` and pick `slack`. ![Share to slack group](../assets/integrations/slack-10-slack_group.png.webp) One simpler way to handle permissions is to host resources and variables on a [folder](../core_concepts/8_groups_and_folders/index.mdx#folders) that is part of the [group](../core_concepts/8_groups_and_folders/index.mdx#groups) `slack`. ![Share variable to folder](../assets/integrations/slack-11-variable_to_folder.png.webp) ![Share folder to group](../assets/integrations/slack-12-folder_to_group.png.webp) ### Handle multiple commands We cover in a [subsequent article](/blog/handler-slack-commands) how to manage multiple commands & human-in-the-loop steps from your slackbot using [branches](../flows/13_flow_branches.md), a text parser and [approval steps](../flows/11_flow_approval.mdx). ### Monitor who ran the command You can see who ran the `/windmill` command by going to the Runs page on Windmill. The runs will be permissioned through the `g/slack` global group. ![Run info](../assets/integrations/slack-6-run-info.png.webp) You can also monitor and permission it from within the script leveraging the [contextual variable](../core_concepts/47_environment_variables/index.mdx#contextual-variables) `WM_USERNAME` that will get the value of the Slack user. For example our script: ```ts export async function main(response_url: string, text: string) { const x = await fetch(response_url, { method: 'POST', body: JSON.stringify({ text: `ROGER ${text}` }) }); // This part: const username = process.env['WM_USERNAME']; console.log(`user = ${username}`); } ``` will console.log `user = username`, username being the Slack username. ### Slack approval steps [Approval steps](../flows/11_flow_approval.mdx) are a way to suspend a [flow](../flows/1_flow_editor.mdx) until specific event(s) are received, such as approvals or cancellations. You can use them to handle approvals from Slack. The Windmill [Python](../advanced/2_clients/python_client.md) and [TypeScript](../advanced/2_clients/ts_client.mdx) clients both have a helper function to request an interactive approval on Slack. An interactive approval is a Slack message that can be approved or rejected directly from Slack without having to go back to the Windmill UI. The following hub scripts can be used: - Python: [Request Interactive Slack Approval](https://hub.windmill.dev/scripts/slack/11403/request-interactive-slack-approval-(python)-slack) - TypeScript: [Request Interactive Slack Approval](https://hub.windmill.dev/scripts/slack/11402/request-interactive-slack-approval-slack) If you define a [form](../flows/11_flow_approval.mdx#form) on the approval step, the form will be displayed in the Slack message as a modal. ![Approval form slack](../assets/flows/tuto_approval_slack_form.png.webp) Both of these scripts are using the Windmill client helper function: ```python wmill.request_interactive_slack_approval( slack_resource_path="/u/username/my_slack_resource", channel_id="admins-slack-channel", message="Please approve this request", approver="approver123", default_args_json={"key1": "value1", "key2": 42}, dynamic_enums_json={"foo": ["choice1", "choice2"], "bar": ["optionA", "optionB"]}, ) ``` ```ts await wmill.requestInteractiveSlackApproval({ slackResourcePath: "/u/username/my_slack_resource", channelId: "admins-slack-channel", message: "Please approve this request", approver: "approver123", defaultArgsJson: { key1: "value1", key2: 42 }, dynamicEnumsJson: { foo: ["choice1", "choice2"], bar: ["optionA", "optionB"] }, }); ``` Where [dynamic_enums](../flows/11_flow_approval.mdx#dynamics-enums) can be used to dynamically set the options of enum form arguments and [default_args](../flows/11_flow_approval.mdx#default-args) can be used to dynamically set the default values of form arguments. If multiple approvals are required, you can use the client helper directly and send approval requests to different channels: ```python import wmill def main(): # Send approval request to customers wmill.request_interactive_slack_approval( 'u/username/slack_resource', 'customers', ) # Send approval request to admins wmill.request_interactive_slack_approval( 'u/username/slack_resource', 'admins', ) ``` ```ts import * as wmill from "windmill-client" export async function main() { await wmill.requestInteractiveSlackApproval({ slackResourcePath: "/u/username/slack_resource", channelId: "customers" }) await wmill.requestInteractiveSlackApproval({ slackResourcePath: "/u/username/slack_resource", channelId: "admins" }) } ``` For more details on approval steps and their features, check out the [Approval steps documentation](../flows/11_flow_approval.mdx). ## Action on Slack from Windmill The second way to make Slack and Windmill interact is through scripts triggered from Windmill to the Slack API. In other words, our goal here is to allow Windmill Scripts acting on Slack on your behalf. Lets navigate to the Resources page page and click "Add a resource/API". :::info You can read more about Resources in the documentation [here][docs-resource]. ::: ![Create Slack resource](../assets/integrations/slack-7-resources.png.webp) Select the `slack` Resource Type from the "OAuth APIs" list and by clicking "Connect", you will be redirected to the Slack account associated with your browser. Click "Allow" to let Windmill access your Slack workspace. Once that's done, you will be redirected back to Windmill, where you can name your Slack Resource and Save it. :::note On [self-hosted instances](../advanced/1_self_host/index.mdx), integrating an OAuth API will require [Setup OAuth and SSO](../advanced/27_setup_oauth/index.mdx). ::: Your connection is made! Now you can trigger the Slack API from Windmill. For example, try [this script](https://hub.windmill.dev/scripts/slack/1284/send-message-to-channel-slack) to send a message to a channel. ```typescript type Slack = { token: string; }; await web.chat.postMessage({ channel, text }); } ``` ![Test script](../assets/integrations/slack-9-script-result.png.webp) ### Have messages published on Windmill's behalf Using the resource created above, Slack will behave on your behalf (under your name). To have messages published on Windmill's behalf, use the Slack resource created in the [Action on Windmill from Slack](#action-on-windmill-from-slack) section. :::info What's next? Also, explore more Slack scripts, flows and apps on [Windmill Hub](https://hub.windmill.dev/?app=slack). ::: ### Error handlers Slack is an efficient way to be notified of errors on a Windmill run, whole workspace or instance. Windmill provides an integration for error handling on Slack. More details on [Error handling](../core_concepts/10_error_handling/index.mdx) page. ### Critical alerts Slack can be used to receive [critical alerts](../core_concepts/37_critical_alerts/index.mdx) from Windmill. This feature is available in the [Enterprise Edition](/pricing). Critical alerts are generated under the following conditions: - [Job](../core_concepts/20_jobs/index.mdx) is re-run after a crash - [License key](../enterprise/1_plans_details/index.mdx#using-the-license-key-self-host) does not renew - [Workspace error handler](../core_concepts/10_error_handling/index.mdx#workspace-error-handler) fails - Number of running workers in a group falls below a specified threshold - Number of [jobs waiting in queue](../core_concepts/9_worker_groups/index.mdx#queue-metric-alerts) is above a threshold for more than a specified amount of time To set up critical alerts to Slack: 1. Configure [SMTP](../advanced/18_instance_settings/index.mdx#smtp) in the instance settings 2. Connect your instance to Slack in the [instance settings](../advanced/18_instance_settings/index.mdx#critical-alert-channels) 3. Specify the Slack channel where alerts should be sent You can also set up worker group-specific alerts to receive notifications when the number of running workers in a group falls below a specified threshold. This can be configured in the [worker group config](../core_concepts/9_worker_groups/index.mdx#alerts). :::note Instance-wide critical alerts are only visible to users with the [superadmin](../core_concepts/16_roles_and_permissions/index.mdx#superadmin) or [devops](../core_concepts/16_roles_and_permissions/index.mdx#devops) roles. For workspace-specific alerts, users need to have admin privilege over that workspace. ::: [hub-slack]: https://hub.windmill.dev/?app=slack [hub-script]: https://hub.windmill.dev/scripts/slack/649/list-users-slack [docs-resource]: /docs/core_concepts/resources_and_types --- ## Smtp Source: https://www.windmill.dev/docs/integrations/smtp # SMTP integration [SMTP](https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol) (Simple Mail Transfer Protocol) is an internet standard for electronic mail transmission. Note that SMTP can be configured at the instance level to auto-invite users or send [critical alerts](../core_concepts/37_critical_alerts/index.mdx). See [Set up SMTP](../advanced/18_instance_settings/index.mdx#smtp). To add a SMTP [resource](../core_concepts/3_resources_and_types/index.mdx) to Windmill, you need to save the following elements: ![Add SMTP Resource](../assets/integrations/add-smtp.png.webp) | Property | Type | Description | Required | Where to find | | -------- | ------ | ---------------------- | -------- | ------------------------------------------------------- | | host | string | SMTP host address | true | Provided by your SMTP service or email hosting provider | | port | number | Port number to connect | false | Provided by your SMTP service or email hosting provider | | user | string | SMTP username | false | Provided by your SMTP service or email hosting provider | | password | string | SMTP password | false | Provided by your SMTP service or email hosting provider | --- ## Snowflake Source: https://www.windmill.dev/docs/integrations/snowflake # Snowflake integration [Snowflake](https://www.snowflake.com/en/) is a cloud-based data warehousing platform. Windmill natively supports Snowflake scripts: To integrate [Snowflake](https://www.snowflake.com/en/) with Windmill, you can either [create and OAuth connection](../advanced/27_setup_oauth/index.mdx#snowflake) or you need to gather the following information and save it as a [resource](../core_concepts/3_resources_and_types/index.mdx). ![Add Snowflake Resource](../assets/integrations/add-snowflake.png.webp) | Property | Type | Description | Default | Required | Where to Find | | ------------------ | ------ | ------------------------------------- | ------- | -------- | --------------------------------------------------------------- | | account_identifier | string | Snowflake account identifier | | true | Snowflake Console > Account > Account Info > Account Identifier | | private_key | string | Snowflake private key | | true | Snowflake Console > Account > Security > RSA Key Pair | | public_key | string | Snowflake public key | | true | Snowflake Console > Account > Security > RSA Key Pair | | warehouse | string | Snowflake warehouse name | | true | Snowflake Console > Warehouses | | username | string | Snowflake username for authentication | | true | Snowflake Console > Users and Security > Users | | database | string | Snowflake database name | | true | Snowflake Console > Databases | | schema | string | Snowflake schema name | | true | Snowflake Console > Databases > [Your Database] > Schemas | | role | string | Snowflake role for access control | | true | Snowflake Console > Users and Security > Roles | --- ## Square Source: https://www.windmill.dev/docs/integrations/square # Square integration [Square](https://www.squarespace.com/) is a payment and financial services provider. To integrate Square to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). ![Add Sendgrid Resource](../assets/integrations/add-square.png.webp) | Property | Type | Description | Required | Where to find | | -------- | ------ | ----------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | token | string | API token | true | 1. Go to https://developer.squareup.com/apps 2. In the left pane, choose Credentials 3. At the top of the page, choose Production mode for a production access token or Sandbox mode for a Sandbox access token. | --- ## Stripe Source: https://www.windmill.dev/docs/integrations/stripe # Stripe integration [Stripe](https://stripe.com/) is a payment processing platform. To integrate Stripe to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). ![Add Stripe Resource](../assets/integrations/add-stripe.png.webp "Add Stripe Resource") | Property | Type | Description | Required | Where to find | | -------- | ------ | ----------- | -------- | -------------------------------------------------------- | | token | string | API token | true | Stripe Dashboard -> Developers -> API keys -> Secret key | --- ## Supabase Source: https://www.windmill.dev/docs/integrations/supabase # Supabase integration [Supabase](https://supabase.com/) is an open-source Firebase alternative. Learn how to connect to your Supabase project from Windmill Scripts, Flows and Apps. ![Integration between Supabase and Windmill](../assets/integrations/sb-0-header.png.webp 'Connect a Supabase project with Windmill') :::info This tutorial assumes that you already have a Windmill account and a [Supabase](https://supabase.com) project. If you don't, visit the [Windmill documentation](../intro.mdx) or the [Supabase documentation](https://supabase.com/docs) to find out more. ::: ## Through Postgres protocol You can execute queries on Supabase through the regular Postgres protocol. 1. Sign-up to Supabase's Cloud App or [Self-Host](https://supabase.com/docs/guides/self-hosting) it. 2. [Create a new Supabase project](https://supabase.com/docs/guides/getting-started). 3. From Windmill, on the [Resources](../core_concepts/3_resources_and_types/index.mdx) menu click on "Add a resource". Pick "postgresql" and "Add a Supabase DB". This will lead you to a Supabase page where you need to pick your organization. Then on Windmill pick a database, fill with database password and that's it. ## Through Supabase API ### Get the API keys In order to make authenticated requests to the database, you'll need your API key and the URL of your endpoint from Supabase. To get these, select your project, navigate to the `Project Settings` page and select `API` from the menu. You'll find the URL and 2 keys here. ![API settings](../assets/integrations/sb-1-1-settings.png.webp) As the description says, the access level of the `public` key will be controlled by the policies you add and the `secret` key will bypass all of them. You can safely use the `service_rolesecret` `secret` key in Windmill because it'll never be sent to users directly. ### Create a resource To safely use secret values throughout Windmill, you can save them in `reources`. We are regularly updating the list of approved resources but if you want an integration to be supported by Windmill directly, please submit a new `resource type` on [Windmill Hub](https://hub.windmill.dev/resource_types). **Navigate to the Resources page page** ![Resources page](../assets/integrations/sb-2-1-resources.png.webp) **Click "Add resource"** ![Resource selector](../assets/integrations/sb-2-2-drawer.png.webp) **Search for `Supabase` and select the resource type** ![Resource selector](../assets/integrations/sb-2-3-search.png.webp) **Enter the API key and the URL from Supabase and click "Save"** ![Resource selector](../assets/integrations/sb-2-4-resource.png.webp) ### Use the resource You can reference the type of a Supabase resource in a script the following way: ```ts // The type matches the fields of the supabase resource type, // see https://hub.windmill.dev/resource_types type Supabase = { url: string; key: string; }; export async function main(auth: Supabase) { // Function contents } ``` :::tip You can find more examples and premade Supabase scripts on [Windmill Hub](https://hub.windmill.dev/?app=supabase). More tutorials on Supabase: - [How to Send Database Events From Supabase to Windmill](/blog/database-events-from-supabase-to-windmill) - [Create an E-commerce backoffice](../apps/7_app_e-commerce.md) - [Create an Issue Tracker App with Supabase in 15 Minutes](/blog/create-issue-tracker-in-15-minutes) - [Create an Issue Tracker App with Supabase - Part 2 Customize Your App](/blog/create-issue-tracker-part-2) - [Use Supabase Authentication on Windmill to query RLS protected tables for external apps](/blog/supabase-authentication-and-rls-protected-tables-on-windmill) ::: --- ## Surrealdb Source: https://www.windmill.dev/docs/integrations/surrealdb # SurrealDB integration [SurrealDB](https://surrealdb.com/) is a cloud-hosted NoSQL database. To integrate SurrealDB to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). ![Add SurrealDB Resource](../assets/integrations/add-surrealdb.png.webp) | Property | Type | Description | Required | Where to find | | --------- | ------ | ---------------------------------------- | -------- | -------------------------------------------------- | | namespace | string | Namespace for the SurrealDB instance | true | SurrealDB Dashboard -> Namespaces | | database | string | Database name for the SurrealDB instance | true | SurrealDB Dashboard -> Databases | | scope | string | Scope of the SurrealDB instance | true | SurrealDB Dashboard -> Scopes | | user | string | Username for the SurrealDB instance | false | SurrealDB Dashboard -> Users | | pass | string | Password for the SurrealDB instance | false | SurrealDB Dashboard -> Users | | url | string | URL of the SurrealDB instance | false | Provided by SurrealDB when you create the instance | --- ## Svix Source: https://www.windmill.dev/docs/integrations/svix # Svix integration [Svix](https://www.svix.com/) offers the possibility to easily add and manage Windmill [webhooks](../core_concepts/4_webhooks/index.mdx). ## Setting up the Windmill connector Windmill can be added as a Svix [Connector](https://docs.svix.com/connectors). ![Windmill Connector](../assets/integrations/svix-windmill-connector.png) ## Adding Windmill webhooks from the App Portal When a connector is set, webhooks can be easily created by adding a new Endpoint and selecting the Windmill option. First set the URL of the Windmill Instance, then click the `Connect to Windmill` button to let the wizard pop out. ![Connect To Windmill button on Svix App portal](../assets/integrations/svix-connect-to-windmill.png) ![Empty wizard](../assets/integrations/svix-empty-wizard.png) On the wizard, choose the script or flow to be triggered, and set a webhook token. This is generally done by pressing `Generate webhook specific token` > `New Token` and closing the token drawer with the `X` on the top left. ![Token Drawer](../assets/integrations/svix-adding-token.png) ![Token added](../assets/integrations/svix-token-added.png) After confirming that everything is correct, Windmill will redirect to Svix to finish creating the webhook endpoint. --- ## Teams Source: https://www.windmill.dev/docs/integrations/teams # Microsoft Teams integration [Microsoft Teams](https://teams.microsoft.com/) is a collaboration platform that integrates with Office 365. There are two ways to build interactions between Microsoft Teams and Windmill: run code on Windmill by a Microsoft Teams command or use the Microsoft Teams API directly from Windmill. In this guide, we'll cover both approaches. ## Action on Windmill from MicrosoftTeams The goal here is to be able to invoke a Windmill Script from Microsoft Teams, by using `/windmill` command. First, you need to be a **workspace admin**. Then you should go to Workspace Settings Page and select the "Slack / Teams" tab. On there, click "Connect to Microsoft Teams". :::info Self-hosted The Microsoft Teams integration is done through OAuth. On [self-hosted instances](../advanced/1_self_host/index.mdx), integrating an OAuth API will require [Setup OAuth and SSO](../advanced/27_setup_oauth/index.mdx). ::: Note that you can connect multiple Windmill workspaces to the same Teams bot. Only one Windmill workspace can accept the `/windmill` [commands](#using-commands-on-teams) from a given Teams "Team". ### Using commands on Teams Once you connected your workspace to a Teams Team, you can either select an existing script or flow to handle the `/windmill` command, or create a new one by clicking the "Create a script to handle teams command" button. You will be navigated to the Script editor. Give your script a name (e.g. `teams_command_handler`), a short summary (e.g. "Teams command handler"). You'll get to [this](https://hub.windmill.dev/scripts/teams/11409/example-of-responding-to-a-microsoft-teams-command-teams) template: ```typescript import * as wmill from "windmill-client" export async function main( activity_id: string, command: string, from_name: string, team_id: string, teams_message: any ) { // Your business logic const res = "task completed successfully!" // (optional) Send update to Teams channel about completion of job await wmill.TeamsService.sendMessageToConversation( { requestBody: { conversation_id: activity_id, success: true, text: `Hi, ${from_name}, command: ${command} ran successfully with the following result: ${res}` } } ) } ``` After the Script is deployed, navigate back to the Teams settings Choose the "Script" option for adding a command handler and select your newly created Script. ![Connected settings](../assets/integrations/teams-connected.png.webp) Congratulations! You've just created a Teams command handler. Now you can use the `/windmill` command in your Teams workspace to trigger the Script. In addition to activity_id, the script/flow can use the following parameters, simply by having them as inputs with the proper name: ``` # the ID of the activity in the Teams conversation activity_id # the command that was triggered command # the name of the user who triggered the command from_name # Microsoft Teams Team ID team_id # the original payload from the Teams backend teams_message ``` ![Use the Windmill command](../assets/integrations/teams-command.png.webp) You won't be able to have Teams interact with your [resources](../core_concepts/3_resources_and_types/index.mdx) and [variables](../core_concepts/2_variables_and_secrets/index.mdx) before adding them to the `teams` [group](../core_concepts/8_groups_and_folders/index.mdx#groups) that was automatically created by Windmill after you set up your Teams workspace on Windmill. How to let Teams use your resources and variables: To give the permission, go to "Resources" (and "Variables") menu, click on `Share`, `Group` and pick `teams`. ![Share to teams group](../assets/integrations/teams-teams_group.png.webp) One simpler way to handle permissions is to host resources and variables on a [folder](../core_concepts/8_groups_and_folders/index.mdx#folders) that is part of the [group](../core_concepts/8_groups_and_folders/index.mdx#groups) `teams`. ![Share variable to folder](../assets/integrations/teams-variable_to_folder.png.webp) ![Share folder to group](../assets/integrations/teams-folder_to_group.png.webp) ### Handle multiple commands You can extend your workspace script to handle complex commands coming from Teams messages. [This article](/blog/handler-slack-commands) shows how to manage multiple commands & human-in-the-loop steps for a slackbot using [branches](../flows/13_flow_branches.md), a text parser and [approval steps](../flows/11_flow_approval.mdx) and can be easily adapted to Teams. ### Monitor who ran the command You can see who ran the `/windmill` command by going to the Runs page on Windmill. The runs will be permissioned through the `g/teams` global group. ![Run info](../assets/integrations/teams-permissions.png.webp) One of the parameters passed to the script is the "from_name" parameter, which is the name of the user who triggered the command. To process further teams specific details from the command, you can use the "teams_message" parameter. ```javascript ## Example of teams_message payload { "id": "1739549535827", "from": { "id": "29:152-adq1512TRGdQmxTqdgnfTA", "name": "Alexander Petric" }, "text": "WindmillHelperBot. /windmill echo Hello World!", "type": "message", "locale": "en-US", "entities": [ { "text": "WindmillHelperBot", "type": "mention", "mentioned": { "id": "28:0eba9472-83d1-429e-ba2c-1c2993dda84d", "name": "WindmillHelperBot" } }, { "type": "clientInfo", "locale": "en-US", "country": "US", "platform": "Mac", "timezone": "America/New_York" } ], "recipient": { "id": "28:0eba9472-83d1-429e-ba2c-1c2993dda84d", "name": "WindmillHelperBot" }, "timestamp": "2025-02-14T16:12:15.857639Z", "serviceUrl": "https://smba.trafficmanager.net/amer/508f04d5-b0de-4661-b035-b90bb6911ce7/", "attachments": [ { "content": "

WindmillHelperBot. /windmill echo Hello World!

" } ], "channelData": { "team": { "id": "19:4SkiS4RJt1gOqqwRUeYdKJvB0XoklVFX4bL9-mhPLbs1@thread.tacv2" }, "tenant": { "id": "508f04d5-b0de-4661-b035-b90bb6911ce7" }, "channel": { "id": "19:4SkiS4RJt1gOqqwRUeXdKJvB0XoklVFX4bL9-mhPLbs1@thread.tacv2" }, "teamsTeamId": "19:4SkiS4RJt1gOqqwRUeXdKJvB0XoklVFX4bL9-mhPLbs1@thread.tacv2" }, "conversation": { "id": "19:4SkiS4RJt1gOqqwRUeXdKJvB0XoklVFX4bL9-mhPLbs1@thread.tacv2;messageid=1737565847488" } } ``` ### Teams approval steps [Approval steps](../flows/11_flow_approval.mdx) are a way to suspend a [flow](../flows/1_flow_editor.mdx) until specific event(s) are received, such as approvals or cancellations. You can use them to handle approvals from Microsoft Teams. The Windmill [TypeScript](../advanced/2_clients/ts_client.mdx) client exposes helper functions to request approvals on Microsoft Teams. There are two types of approvals: 1. **Interactive approval**: A Teams message that can be approved or rejected directly from Teams without having to go back to the Windmill UI 2. **Basic approval**: A simple link that will open the approval page in the Windmill UI in your browser The following hub scripts can be used: - [Request Interactive Teams Approval](https://hub.windmill.dev/scripts/teams/13935/interactive-microsoft-teams-approval-teams) - [Request Basic Teams Approval](https://hub.windmill.dev/scripts/teams/13936/microsoft-teams-approval-teams) If you define a [form](../flows/11_flow_approval.mdx#form) on the approval step, the form will be displayed in the Teams message as a modal. ![Approval form teams](../assets/flows/tuto_approval_teams.png.webp) Both of these scripts are using the Windmill client helper function: ```typescript await wmill.requestInteractiveTeamsApproval({ teamName: "Windmill", channelName: "General", message: "Please approve this request", approver: "approver123", defaultArgsJson: { key1: "value1", key2: 42 }, dynamicEnumsJson: { foo: ["choice1", "choice2"], bar: ["optionA", "optionB"] }, }); ``` ```typescript const card_block = { "type": "message", "attachments": [ { "contentType": "application/vnd.microsoft.card.adaptive", "content": { "type": "AdaptiveCard", "$schema": "https://adaptivecards.io/schemas/adaptive-card.json", "version": "1.6", "body": [ ... // card body ], }, } ], "conversation": {"id": `${conversation_id}`}, } await wmill.TeamsService.sendMessageToConversation({ requestBody: { conversation_id, text: "A workflow has been suspended and is waiting for approval!", card_block } }) ``` Where [dynamic_enums](../flows/11_flow_approval.mdx#dynamics-enums) can be used to dynamically set the options of enum form arguments and [default_args](../flows/11_flow_approval.mdx#default-args) can be used to dynamically set the default values of form arguments. If multiple approvals are required, you can use the client helper directly and send approval requests to different channels: ```typescript import * as wmill from "windmill-client" export async function main() { // Send approval request to team A await wmill.requestInteractiveTeamsApproval({ teamName: "Team A", channelName: "General", message: "Please approve this request" }) // Send approval request to team B await wmill.requestInteractiveTeamsApproval({ teamName: "Team B", channelName: "General", message: "Please approve this request" }) } ``` For more details on approval steps and their features, check out the [Approval steps documentation](../flows/11_flow_approval.mdx). ## Action on Teams from Windmill ### Write to Teams from Windmill The second way to make Teams and Windmill interact is through scripts triggered from Windmill to the Teams API. In other words, our goal here is to allow Windmill Scripts acting on Teams on your behalf. Let's navigate to the Resources page and click "Add a resource/API". :::info You can read more about Resources in the documentation [here][docs-resource]. ::: ![Create Teams resource](../assets/integrations/teams-resources.png) Select the `ms_teams_webhook` Resource Type from the resources list and add a webhook_url (see Teams [docs](https://learn.microsoft.com/en-us/microsoftteams/platform/webhooks-and-connectors/how-to/add-incoming-webhook?tabs=dotnet) for details on how to create a webhook). You can then use the resource in a script, like this script found on [WindmillHub](https://hub.windmill.dev/scripts/msteams/1582/send-a-message-to-ms-teams-channel-via-webhook-msteams) that "Sends a message to MS Teams channel via webhook": ```typescript // See below for an example card you can pass in type MsTeamsWebhook = { webhook_url: string; }; const ret = await webhook.send(messageCard); return ret; } const example_card = { "@type": "MessageCard", "@context": "https://schema.org/extensions", summary: "This is a test summary", themeColor: "0078D7", title: "This is a test title", sections: [ { activityTitle: "Windmill Webhook", activitySubtitle: "2023-05-25 17:57:55", activityImage: "https://connectorsdemo.azurewebsites.net/images/MSC12_Oscar_002.jpg", text: "This is a test text", }, ], }; ``` ### Error handlers Microsoft Teams is an efficient way to be notified of errors on a Windmill run, whole workspace or instance. Windmill provides an integration for error / success / recovery handling on Microsoft Teams. More details on [Error handling](../core_concepts/10_error_handling/index.mdx) page. ### Critical alerts Microsoft Teams can be used to receive [critical alerts](../core_concepts/37_critical_alerts/index.mdx) from Windmill. This feature is available in the [Enterprise Edition](/pricing). Critical alerts are generated under the following conditions: - [Job](../core_concepts/20_jobs/index.mdx) is re-run after a crash - [License key](../enterprise/1_plans_details/index.mdx#using-the-license-key-self-host) does not renew - [Workspace error handler](../core_concepts/10_error_handling/index.mdx#workspace-error-handler) fails - Number of running workers in a group falls below a specified threshold - Number of [jobs waiting in queue](../core_concepts/9_worker_groups/index.mdx#queue-metric-alerts) is above a threshold for more than a specified amount of time To set up critical alerts to Microsoft Teams: 1. Configure [SMTP](../advanced/18_instance_settings/index.mdx#smtp) in the instance settings 2. Connect your instance to Microsoft Teams in the [instance settings](../advanced/18_instance_settings/index.mdx#critical-alert-channels) 3. Specify the Teams channel where alerts should be sent You can also set up worker group-specific alerts to receive notifications when the number of running workers in a group falls below a specified threshold. This can be configured in the [worker group config](../core_concepts/9_worker_groups/index.mdx#alerts). :::note Instance-wide critical alerts are only visible to users with the [superadmin](../core_concepts/16_roles_and_permissions/index.mdx#superadmin) or [devops](../core_concepts/16_roles_and_permissions/index.mdx#devops) roles. For workspace-specific alerts, users need to have admin privilege over that workspace. ::: ## Troubleshooting ### "The bot is not part of the conversation roster" error This error occurs when Windmill tries to send a message to a Teams channel but the Bot Framework rejects the request. Common causes: 1. **Channel moderation rules**: If the Teams channel has posting restrictions (channel moderation enabled), the bot may not be allowed to post. Check your Teams channel settings and either: - Disable channel moderation - Add the Windmill bot to the list of users allowed to post 2. **Bot not installed in the team**: The Windmill Teams app must be installed in the team where you want to send messages. Verify the app appears in the team's "Apps" section. 3. **Bot ID mismatch** (self-hosted): For self-hosted instances, ensure your Azure AD App Registration client ID matches: - The Microsoft App ID in your Azure Bot Service - The `botId` in your Teams app manifest ### Messages not being delivered If the bot appears to be connected but messages aren't being delivered, check the service URL region. The default service URL is for the Americas region. If your Teams tenant is in a different region, set the `TEAMS_SERVICE_URL` environment variable: - Americas: `https://smba.trafficmanager.net/amer/` (default) - EMEA: `https://smba.trafficmanager.net/emea/` - APAC: `https://smba.trafficmanager.net/apac/` - US Government (GCC): `https://smba.infra.gcc.teams.microsoft.com/` [hub-teams]: https://hub.windmill.dev/?app=teams [hub-script]: https://hub.windmill.dev/scripts/teams/11409/example-of-responding-to-a-microsoft-teams-command-teams [docs-resource]: /docs/core_concepts/resources_and_types --- ## Telegram Source: https://www.windmill.dev/docs/integrations/telegram # Telegram integration [Telegram](https://telegram.org/) is a cloud-based instant messaging and voice over IP service. To integrate Telegram to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). ![Add Telegram Resource](../assets/integrations/add-telegram.png.webp "Add Telegram Resource") | Property | Type | Description | Required | Where to find | | -------- | ------ | ------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | token | string | Bot API token | true | 1. Open the Telegram app on your device or use the web version (https://web.telegram.org/). 2. Search for the "BotFather" bot in the search bar. 3. Start a chat with the BotFather. 4. Send the command "/newbot" to create a new bot. 5. Follow the BotFather's instructions to give your bot a name and username. 6. Once you have successfully created the bot, the BotFather will provide you with the Bot API token. | --- ## Tigris Source: https://www.windmill.dev/docs/integrations/tigris # Tigris integration [Tigris](https://www.tigrisdata.com/) is an S3-compatible object storage service that automatically caches data at the edge closest to where it's accessed. Zero egress fees and a free tier (5 GB). Tigris is used in Windmill through the generic `s3` [resource type](https://hub.windmill.dev/resource_types/42/). The [S3 APIs integrations](./s3.mdx) page is the canonical reference: it covers the resource fields, how to use the resource in scripts, flows and apps, and how to plug a bucket as [workspace or instance object storage](../core_concepts/38_object_storage_in_windmill/index.mdx). ## Where to find the resource details on Tigris Create an access key at [console.tigris.dev](https://console.tigris.dev). Keys are prefixed with `tid_` (access key) and `tsec_` (secret key). | Property | Value for Tigris | | --------- | ---------------------------------------------------------------------------------------- | | bucket | Name of the bucket, from the [Tigris Console](https://console.tigris.dev) | | region | Must be `auto`. Tigris routes requests to the nearest edge automatically | | endPoint | `t3.storage.dev`, or `fly.storage.tigris.dev` on Fly.io | | useSSL | `true` - SSL/TLS is required for Tigris | | pathStyle | `false` - Tigris uses virtual-hosted-style URLs | | accessKey | Required. Access key ID prefixed with `tid_`, from the Tigris Console | | secretKey | Required. Secret access key prefixed with `tsec_`, from the Tigris Console | --- ## Toggl Source: https://www.windmill.dev/docs/integrations/toggl # Toggl integration [Toggl](https://toggl.com/) is a time tracking software. To integrate Toggl to Windmill, you need to save the following elements as a [resource](../core_concepts/3_resources_and_types/index.mdx). ![Add Toggl Resource](../assets/integrations/add-toggl.png.webp) | Property | Type | Description | Required | Where to find | | -------- | ------ | ----------- | -------- | ------------------------------------------------------------ | | token | string | API token | true | 1. Go to https://track.toggl.com/profile 2. Find "API Token" | --- ## Upstash Source: https://www.windmill.dev/docs/integrations/upstash # Upstash integration [Upstash](https://upstash.com/) is a serverless and low-latency Redis-compatible data store for modern applications. Upstash follows the same connection method as [MongoDB](./mongodb.md), providing the database name, TLS settings, server information, and credentials for authentication. ![Add Mongodb](../assets/integrations/add_mongodb.png.webp) Here's a table detailing the properties for Upstash integration using the [MongoDB resource type](https://hub.windmill.dev/resource_types/22/mongodb): | Property | Type | Description | Default | Required | Where to Find | Additional Details | | ----------------- | ------- | -------------------------- | ------- | -------- | -------------------------- | --------------------------------------------------- | | db | integer | Database index | | true | Redis Server Configuration | Index of the Redis database you want to connect to | | tls | boolean | Use TLS for connections | true | false | Your own preference | Set to true for secure connections | | servers | array | Array of server objects | | true | Redis Server Configuration | Each server object should contain `host` and `port` | | host (nested) | string | Server address | | true | Redis Server Configuration | Hostname of the Redis instance | | port (nested) | integer | Port number | 6379 | false | Redis Server Configuration | Default Redis port is `6379` | | credential | object | Authentication information | | true | Redis Server Configuration | Contains `password` | | password (nested) | string | Database password | | true | Redis Server Configuration | Your Redis server's password | --- ## Intro Source: https://www.windmill.dev/docs/intro # What is Windmill? Windmill is a fast, **open-source** workflow engine and developer platform. It's an alternative to the likes of Retool, Superblocks, n8n, Airflow, Prefect, Kestra and Temporal, designed to **build comprehensive internal tools** (endpoints, workflows, UIs). It supports coding in TypeScript, Python, Go, PHP, Bash, C#, SQL, Rust, Ruby and R, or any Docker image, alongside intuitive low-code builders, featuring: - An [execution runtime](./script_editor/index.mdx) for scalable, low-latency function execution across a worker fleet. - An [orchestrator](./flows/1_flow_editor.mdx) for assembling these functions into efficient, low-latency flows, using either a low-code builder or YAML. - A [full-code app](./full_code_apps/index.mdx) builder for custom frontends in React or Svelte connected to Windmill backend runnables. - A [low-code app editor](./apps/0_app_editor/index.mdx) (legacy) for creating data-centric dashboards with drag-and-drop components. Windmill supports both UI-based operations via its webIDE and low-code builders, as well as [CLI](./advanced/3_cli/index.mdx) deployments [from a Git repository](./advanced/11_git_sync/index.mdx), aligning with your preferred development style. Start your project today with our **Cloud App** (no credit card needed) or opt for **self-hosting**. ## Develop faster Focus on code that matters: your critical business logic, from data transformation to internal API calls, starts as scripts and SQL files. Windmill transforms these into scalable microservices and tools without the usual heavy lifting. Boilerplate code is eliminated as Windmill takes care of the repetitive work of building UIs, [handling errors](./core_concepts/10_error_handling/index.mdx), scaling logic, and [managing dependencies](./advanced/6_imports/index.mdx), making these aspects more streamlined. ## Core features **An efficient runtime**: execute code [across languages](./getting_started/0_scripts_quickstart/index.mdx) with minimal overhead and instant starts. **Smart dependency and input management**: automatically generate lockfiles and input specs from your code, ensuring consistent [dependency versions](./advanced/6_imports/index.mdx) and simplified [input handling](./core_concepts/13_json_schema_and_parsing/index.mdx). **Dynamic web IDE and low-code builders**: create [scripts](./script_editor/index.mdx) with advanced editing tools and [auto-generated UIs](./core_concepts/6_auto_generated_uis/index.mdx), build [flows](./flows/1_flow_editor.mdx) with a drag-and-drop interface, build [full-code apps](./full_code_apps/index.mdx) with React or Svelte, or design [low-code apps](./apps/0_app_editor/index.mdx) (legacy) without extensive coding. **Enterprise-ready**: Windmill offers robust [permissioning](./core_concepts/16_roles_and_permissions/index.mdx), [secret management](./core_concepts/2_variables_and_secrets/index.mdx), OAuth, and more, wrapped in an enterprise-grade platform. **Integrations and automations**: with [webhooks](./core_concepts/4_webhooks/index.mdx), an [open API](https://app.windmill.dev/openapi.html), and a [scheduler](./core_concepts/1_scheduling/index.mdx), Windmill fits seamlessly into your infrastructure, allowing for extensive automation capabilities. **Local development**: develop scripts and flows locally using your favorite IDE with the [VS Code extension](./cli_local_dev/1_vscode-extension/index.mdx) and [CLI](./advanced/3_cli/index.mdx), then sync with [Git integration](./advanced/11_git_sync/index.mdx) for version control and deployment workflows. ## Compare While other frameworks offer pieces of what Windmill does, none combine its comprehensive feature set with full open-source accessibility. Whether compared to workflow engines like Temporal and Airflow or UI builders like Retool, Windmill stands out for its scalability, open APIs, and ease of use. Windmill is an open-source, self-hostable platform that marries the flexibility of code with the speed of low-code solutions, enabling seamless automation of repetitive tasks. Learn how Windmill compares to other products like [Retool](./compared_to/retool.mdx), [n8n](./compared_to/peers.mdx#n8n), [Airflow](./misc/3_benchmarks/competitors/airflow/index.mdx), [Prefect](./compared_to/prefect.mdx), [Kestra](./compared_to/kestra.mdx) and [Temporal](./misc/3_benchmarks/competitors/temporal/index.mdx). --- ## Benchmarks Source: https://www.windmill.dev/docs/misc/benchmarks # Benchmarks --- ## Aws lambda Source: https://www.windmill.dev/docs/misc/benchmarks/aws_lambda # AWS Lambda vs Windmill This benchmark compares the performance of [AWS Lambda](https://aws.amazon.com/lambda/) and Windmill for executing a simple but computationally intensive task - calculating the 33rd Fibonacci number using a recursive algorithm. We chose this task because: 1. It's CPU-bound rather than I/O-bound, making it a good test of raw compute performance. 2. The recursive implementation creates significant computational overhead. 3. It's simple enough to implement identically in both platforms. The goal is to measure and compare: - Cold start latency. - Warm execution time. - Consistency of response times. - Cost implications at scale. Both services were configured similarly, using Python 3.11 as the runtime. The same Fibonacci calculation code was deployed to each platform to ensure a fair comparison. ## Setup ### Windmill The setup is the exact same as for the [other benchmarks ](../competitors/index.mdx). We used the same EC2 m4-large instance and deployed Windmill on docker using the docker-compose.yml in Windmill's official GitHub repo (with the same adjustment, i.e. 1 worker only, even though for this use case it would not make a difference). We created a script in Windmill computing a Fibonacci number in Python: ```python N_FIBO = 33 # WINDMILL script: `u/benchmarkuser/fibo_script` def fibo(n: int): if n <= 1: return n else: return fibo(n - 1) + fibo(n - 2) def main(): return fibo(N_FIBO) ``` which we called multiple times using its webhook. We used `siege` benchmark tool to trigger the jobs multiple times using its webhook: ```bash siege -r500 -c1 -v -H "Cookie: token=$WM_TOKEN" "http://$WM_HOST/api/w/benchmarks/jobs/run_wait_result/p/u%2Fbenchmarksuser%2Ffibo_script" ``` ### AWS Lambda We set up a Lambda running Python 3.11 with the following simple script: ```python import json N_FIBO = 33 def fibo(n: int): if n <= 1: return n else: return fibo(n - 1) + fibo(n - 2) def lambda_handler(event, context): result = fibo(N_FIBO) return { 'statusCode': 200, 'body': json.dumps(result) } ``` We gave the Lambda 2048MB of memory, but according to the logs the memory never exceeded 50MB. On AWS, vCPU is proportional to the memory so we can assume it got a decent vCPU. We exposed a trigger through AWS API Gateway and from our EC2 instance, we called it using the same `siege` benchmark tool: ```bash siege -r500 -c1 -v -H "x-api-key: $AWS_API_KEY" "https://$AWS_LAMBDA_HOST/default/fibo_lambda" ``` ## Results In the same vein as the other benchmarks, we ran `fibonacci(10)` 500 times (`--reps=500` as `siege` argument) and `fibonacci(33)` 100 times. The results were the following: | **(in seconds)** | **# reps** | **AWS Lambda (sec)** | **WindmillNormal (sec)** | **WindmillDedicated Worker (sec)** | | :--------------: | ---------: | -------------------: | -----------------------------: | ---------------------------------------: | | **fibo(10)** | 500 | 36.56 | 55.36 | 26.81 | | **fibo(33)** | 100 | 93.95 | 109.06 | 104.5 | Which gives an average duration per job in milliseconds: | **(in milliseconds)** | **AWS Lambda (sec)** | **WindmillNormal (sec)** | **WindmillDedicated Worker (sec)** | | :-------------------: | :------------------: | :----------------------------: | :--------------------------------------: | | **fibo(10)** | 73 | 111 | 54 | | **fibo(33)** | 940 | 1091 | 1045 | Visually, we have the following graphs: ## Conclusion For running a high number of lightweight tasks (`fibonacci(10)`), we can see that Windmill in [dedicated worker](../../../core_concepts/25_dedicated_workers/index.mdx) mode is the fastest. Windmill in normal mode is slower because it runs a cold start for each task. For long running tasks (`fibonacci(33)`), Windmill in normal mode and dedicated worker mode is almost equivalent because the warm-up time needed in normal mode is negligible compared to the duration of tasks. AWS Lambda has slightly better performance for those kind of tasks, likely because it is able to run the core of the Python logic faster than Windmill. --- ## Competitors Source: https://www.windmill.dev/docs/misc/benchmarks/competitors # Benchmarks - Methodology :::tip More context For additional insights about benchmark methodology, refer to our [blog post](/blog/launch-week-1/fastest-workflow-engine). ::: In this benchmark study, we compared six job orchestration engines: [Airflow](https://airflow.apache.org/), [Prefect](https://www.prefect.io/), [Temporal](https://temporal.io/), [Kestra](https://kestra.io/), [Hatchet](https://github.com/hatchet-dev/hatchet), and [Windmill](/), focusing on performance across several scenarios. The aim was to evaluate not just raw task execution time, but also deeper engine-level behaviors such as scheduling efficiency, task dispatch latency, and worker utilization. This study was last run in 2025. The exact engine versions used (e.g. Windmill 1.483.1, Airflow 2.7.3, Prefect 2.14.4, Temporal 2.34.0, Kestra 0.22.3, Hatchet 0.62.0) are listed on each engine's dedicated page; results may differ with newer releases. We chose to compute Fibonacci numbers as a simple task that can easily be run with all six engines. Given that Airflow has first class support for Python, we used Python for every engine that supports it. The function in charge of computing the Fibonacci numbers was very naive: ## Benchmark use cases We defined three categories of workflow scenarios: 1. **Lightweight tasks**: Simulates high-frequency, short-lived operations where engine overhead may dominate. 2. **Long-running tasks**: Designed to surface runtime performance and engine efficiency when task duration is significant. 3. **Multi-worker scenarios**: For engines demonstrating high efficiency and Go support, we ran: - 400 lightweight tasks - 100 long-running tasks These were distributed across multiple workers, examining parallelism, load balancing, and assignment latency. ## Task definition To ensure simplicity, repeatability, and no external dependencies, we used the classic recursive Fibonacci function: ```python def fibo(n: int): if n <= 1: return n else: return fibo(n - 1) + fibo(n - 2) ``` - `fibo(10)` was used for lightweight tasks, with an average execution time of ~10ms. - `fibo(33/38)` was selected for long-running tasks, typically taking several hundred milliseconds. This approach eliminates the need for external libraries, providing a level playing field and highlighting the core performance of the orchestration engines. ## Language and runtime environment Given native Python support in Airflow, Python was used as the primary implementation language for initial benchmarks. For orchestrators supporting multiple runtimes, we expanded testing to: - JavaScript (where supported) - Go for its speed, concurrency features, and lack of warmup latency For Go-enabled engines, we also evaluated multi-worker configurations to explore scaling behavior. ## Infrastructure setup To standardize the environment, each orchestrator was deployed using its recommended docker-compose.yml setup, running on **AWS m4.large instances**. This provides a balanced compute and memory profile while ensuring consistency across platforms. ## Performance evaluation metrics The benchmarking framework was designed to expose both high-level throughput and low-level engine characteristics. Key metrics observed: - Execution time: the time it takes for the orchestrator to execute the task once it has been assigned to an executor - Assignment time: the time it takes for a task to be assigned to an executor once it has been created in the queue - Transition time: the time it takes to create the following task once a task is finished - Worker load distribution: whether tasks were evenly distributed or exhibited contention/idling. Observational expectations: - Short-running tasks: Performance is expected to be dominated by orchestration overhead, making it a strong indicator of engine efficiency in high-frequency workflows. - Long-running tasks: Majority of time should be spent on actual computation, with minimal overhead from task management and worker assignment. ## Extraction of timings The timings were extracted either through the exposed api of the orchestrators or by directly querying the database of the orchestrators. For most of the engines the following timestamps could be extracted: - `Reference time (t0)`: workflow start time - `created_at`: task added to queue / scheduled time - `started_at`: task assigned to worker - `completed_at`: task finished Raw measurements for each engine: - [Airflow](@site/src/data/airflow.json) - [Kestra](@site/src/data/kestra.json) - [Prefect](@site/src/data/prefect.json) - [Temporal](@site/src/data/temporal.json) - [Windmill](@site/src/data/windmill.json) - [Hatchet](@site/src/data/hatchet.json) The scripts used to extract the data can be found [in the windmill-benchmarks repository](https://github.com/windmill-labs/windmill-benchmarks). We did not dive into the source code of each orchestration engine to see if the timestamp generation is consistent across all engines or there are some slight differences. --- ## Airflow benchmark Source: https://www.windmill.dev/docs/misc/benchmarks/competitors/airflow # Airflow benchmarks ## Summary [Airflow](https://airflow.apache.org/) was the slowest in all categories, with high orchestration overhead and poor responsiveness to both lightweight and long-running tasks. It suffers from long assignment delays and inefficient scaling, making it unsuitable for performance-critical workflows. ## Airflow setup We set up Airflow version 2.7.3 using the [docker-compose.yaml](https://airflow.apache.org/docs/apache-airflow/2.7.3/docker-compose.yaml) referenced in Airflows official [documentation](https://airflow.apache.org/docs/apache-airflow/stable/howto/docker-compose/index.html#fetching-docker-compose-yaml). The DAG was the following: ```python ITER = 10 # respectively 40 FIBO_N = 33 # respectively 10 with DAG( dag_id="bench_{}".format(ITER), schedule=None, start_date=datetime(2023, 1, 1), catchup=False, tags=["benchmark"], ) as dag: for i in range(ITER): @task(task_id=f"task_{i}") def task_module(): return fibo(FIBO_N) fibo_task = task_module() if i > 0: previous_task >> fibo_task previous_task = fibo_task ``` --- ## Hatchet benchmark Source: https://www.windmill.dev/docs/misc/benchmarks/competitors/hatchet # Hatchet benchmarks ## Summary [Hatchet](https://hatchet.run/) performed well in long-running tasks, with high execution ratios and low overhead. However, it struggled in multi-worker lightweight scenarios, with high wait and idle times despite perfect task distribution. It's efficient for compute-heavy jobs, but needs refinement in orchestration under load. ## Hatchet setup We set up Hatchet version 0.62.0 using the [docker-compose.yml from the official documentation](https://docs.hatchet.run/self-hosting/docker-compose). The following workflows were used to run the benchmarks: Note that for multi-worker scenarios, we tried both using `RunBulkNoWait` that splits up all individual fibonacci tasks as separate workflows and setting `PARALLEL=true` that runs all fibonacci tasks in parallel in a single workflow with parallel tasks. We did not see significant difference in the results. --- ## Kestra benchmark Source: https://www.windmill.dev/docs/misc/benchmarks/competitors/kestra # Kestra benchmarks ## Summary [Kestra](https://kestra.io/) delivered middle-of-the-pack performance — relatively stable for long-running tasks, but slower than competitors in lightweight workloads. While more modern than [Airflow](../airflow/index.mdx) and [Prefect](../prefect/index.mdx), it falls short of the orchestration efficiency seen in [Temporal](../temporal/index.mdx), [Hatchet](../hatchet/index.mdx), or [Windmill](../windmill/index.mdx). ## Kestra setup We set up Kestra version v0.22.3 using the [docker-compose.yml from their official Documentation](https://kestra.io/docs/installation/docker-compose). We made some adjustments to it to have a similar setup compared to the other orchestrator. The flow we used to run the benchmarks is the following: We executed it once with `n=33`, `iters=10` and once with `n=10` and `iters=40`. Note that we set the concurrency limit to 1 meaning all task will run sequentially on one worker. Furthermore, no extra python dependencies had to be installed during the execution of those flows, , and we use a `Process` task runner to avoid starting a Docker container for each task execution. --- ## Prefect benchmark Source: https://www.windmill.dev/docs/misc/benchmarks/competitors/prefect # Prefect benchmarks ## Summary [Prefect](https://prefect.io/) outperformed [Airflow](../airflow/index.mdx) in most benchmarks, especially in long-running tasks, but still exhibited noticeable assignment delays and orchestration latency. It’s adequate for moderate workloads, but not optimized for high-frequency or highly parallel use cases. ## Prefect setup We set up Prefect version 2.14.4. We wrote our own simple docker compose since we couldn't find a recommended one in Prefect's documentation. We chose to use Postgresql as a database, as it is the recommended option for production usecases. ```yaml version: '3.8' services: postgres: image: postgres:14 restart: unless-stopped volumes: - db_data:/var/lib/postgresql/data expose: - 5432 environment: POSTGRES_PASSWORD: changeme POSTGRES_DB: prefect healthcheck: test: ['CMD-SHELL', 'pg_isready -U postgres'] interval: 10s timeout: 5s retries: 5 prefect-server: image: prefecthq/prefect:2-latest command: - prefect - server - start ports: - 4200:4200 depends_on: postgres: condition: service_started volumes: - ${PWD}/prefect:/root/.prefect - ${PWD}/flows:/flows environment: PREFECT_API_DATABASE_CONNECTION_URL: postgresql+asyncpg://postgres:changeme@postgres:5432/prefect PREFECT_LOGGING_SERVER_LEVEL: INFO PREFECT_API_URL: http://localhost:4200/api volumes: db_data: null ``` The flow was defined using the following Python file. ```python from prefect import flow, task ITER = 10 # respectively 40 FIBO_N = 33 # respectively 10 def fibo(n: int): if n <= 1: return n else: return fibo(n - 1) + fibo(n - 2) @task def fibo_task(): return fibo(FIBO_N) @flow(name="bench_{}".format(ITER)) def benchmark_flow(): for i in range(ITER): fibo_task() if __name__ == "__main__": benchmark_flow.serve(name="bench_{}".format(ITER)) ``` --- ## Results Source: https://www.windmill.dev/docs/misc/benchmarks/competitors/results # Benchmarks of individual engines --- ## Conclusion Source: https://www.windmill.dev/docs/misc/benchmarks/competitors/results/conclusion # Benchmark conclusions ## Conclusion Across all our benchmarks, **[Airflow](../../airflow/index.mdx) consistently demonstrated the weakest performance**, particularly struggling with lightweight or high-concurrency workloads. While reliable and widely adopted, its architecture introduces significant scheduling and task startup overhead that makes it a poor fit for modern, performance-sensitive workflows. **[Prefect](../../prefect/index.mdx)** performed moderately better, especially in long-running task scenarios, but also showed signs of orchestration latency when scaled out or pushed with lightweight workloads. In contrast, **[Temporal](../../temporal/index.mdx)**, **[Hatchet](../../hatchet/index.mdx)**, **[Kestra](../../kestra/index.mdx)**, and **[Windmill](../../windmill/index.mdx)** consistently outperformed Airflow and Prefect across languages and scenarios, with varying strengths depending on the workload and configuration. Among these, **Windmill** stood out for its flexibility and consistent performance profile in both single-worker and multi-worker setups. For **long-running tasks**, where actual computation dominates, **Windmill** delivered the fastest total flow times across Python, JavaScript, and Go, maintaining extremely high execution ratios and minimal orchestration overhead. It's well-suited for CPU-bound or latency-insensitive workflows, where efficient use of compute resources matters most. When dealing with **high-frequency, lightweight workloads**, **Windmill** in [dedicated worker](../../../../../core_concepts/25_dedicated_workers/index.mdx) mode and **Temporal** emerged as the most effective solutions. Temporal achieved top throughput in Go multi-worker scenarios thanks to high worker utilization and low scheduling latency. Windmill, even with lower utilization due to its per-task isolation model, delivered competitive completion times due to its optimized caching and parallel dispatch model. **Hatchet**, while strong in raw execution, showed bottlenecks in multi-worker environments, with high wait times and underutilized workers despite perfect task distribution. **Kestra** remained in the mid-tier throughout — showing decent performance in long-running task setups, but slower orchestration in short task loads. Its behavior was more predictable than Airflow or Prefect, but less optimized than Temporal, Hatchet, or Windmill. Ultimately, **Windmill** proved to be the most versatile and well-balanced engine across all dimensions tested — combining strong execution speed, adaptive behavior between cold and warm task starts, and reliable scaling patterns. **Temporal** was a close contender, particularly in Go environments, offering efficient parallelism and predictable low-latency behavior. **Hatchet** shined in tightly controlled, compute-heavy scenarios but would benefit from improved orchestration in dynamic workloads. ## Future work We plan to extend this study in the following directions: 1. **Scalability testing**: Evaluating how each orchestrator performs as the number of workers scales into the hundreds. This will help determine the elasticity and bottlenecks of each engine under real-world horizontal scaling. 2. **Throughput ceilings**: Measuring the maximum tasks per second each engine can handle, while observing database and queue saturation points. This is especially important for orchestrators with hybrid persistence models (e.g., DB + message queues). 3. **Resilience under network conditions**: Introducing artificial latency, jitter, and packet loss into the orchestration network to simulate real-world infrastructure variance, particularly useful for hybrid or edge-cloud setups. 4. **Engine determinism and stability**: Repeating each benchmark multiple times to measure variance and uncover how consistent each engine's performance is under otherwise identical loads. These next steps aim to paint a fuller picture of not just speed, but **robustness**, **scalability**, and **real-world reliability**. --- ## Go Source: https://www.windmill.dev/docs/misc/benchmarks/competitors/results/go # Results: Go To evaluate orchestrator performance in Go, we tested [Hatchet](/docs/misc/benchmarks/competitors/hatchet), [Temporal](/docs/misc/benchmarks/competitors/temporal), and [Windmill](/docs/misc/benchmarks/competitors/windmill) using a consistent set of benchmark scenarios: 400 lightweight tasks and 100 long-running tasks, both in single-worker and 10-worker configurations. **Go provides an ideal testbed** for orchestration engine performance due to its **low startup** overhead and **tight runtime**, meaning most latency observed can be attributed to the orchestration engine itself. The other orchestration engines either didn't support the Go language natively (e.g Kestra only via Docker) or not at all. ## Single worker ### Long-Running Tasks (10 tasks) In the single-worker configuration for long-running tasks, total flow durations were: Hatchet: 30.715s, Temporal: 28.313s and Windmill: 27.648s. As expected with heavier workloads, execution time dominated the run: Windmill: 93.82%, Temporal: 91.81% and Hatchet: 88.69% This suggests that all three engines handle Go workloads efficiently once tasks are assigned. Assignment times were lowest for Windmill (2.46%) and Temporal (2.60%), with Hatchet slightly higher at 4.95%. Transition times were also well managed, with Windmill again leading at just 3.73% of total time. The results confirm that in long-running tasks, all three engines effectively minimize orchestration overhead in Go, with Windmill slightly outperforming the others in total time and transition latency. ### Lightweight Tasks (400 tasks) For lightweight tasks, engine overhead becomes critical. The results show: Hatchet: 35.845s, Temporal: 39.016s and Windmill: 19.702s. Here, the execution of fibo(10) is negligible (~10ms), so orchestration mechanics dominate performance. Windmill stands out with less than half the total time of either Temporal or Hatchet. Breaking it down: Assignment: Windmill: 18.53%, Temporal: 41.24% and Hatchet: 68.87%. Transition: Windmill: 67.17%, Temporal: 52.62% and Hatchet: 26.75%. The high transition percentage for Windmill reflects its very fast execution and relatively even task distribution. Hatchet and Temporal, in contrast, spend more time assigning tasks and maintaining queues, which stretches the total duration. Overall, Windmill clearly leads in lightweight throughput in Go with a single worker, suggesting minimal orchestration latency and efficient task sequencing. ## Multi-worker ### 10 workers: 100 long running tasks All three engines performed well under this heavier workload, with durations as follows: Temporal: 11.152s, Windmill: 11.899s and Hatchet: 17.753s. Temporal had slightly better worker utilization (94.86%) compared to Windmill (92.19%), with Windmill showing slightly higher avg wait time (5.478s vs. 5.012s). Execution time per worker was nearly identical across engines (roughly 11s), suggesting that the core compute workload was equally distributed. Windmill's advantage, however, lies in its fast scheduling (0.024s) and low transition costs, which helped maintain competitive performance despite a slightly higher wait time. Hatchet showed lower utilization (63.05%) and higher idle times, indicating room for improvement in orchestrating long-running parallel workloads in Go. Note that for multi-worker scenarios, we tried both using `RunBulkNoWait` that splits up all individual fibonacci tasks as separate workflows and setting `PARALLEL=true` that runs all fibonacci tasks in parallel in a single workflow with parallel tasks. We did not see significant difference in the results. ### 10 workers: 400 lightweight tasks In the multi-worker configuration for lightweight tasks, Temporal was the fastest, completing all 400 tasks in just 4.270 seconds. Windmill followed at 7.224 seconds, while Hatchet significantly lagged behind with a total duration of 37.809 seconds. At first glance, Windmill appears notably slower than Temporal, but a deeper look into worker load distribution helps explain the trade-offs in orchestration design. - Worker Utilization: Temporal: 32.41%, Windmill: 11.75%, Hatchet: 3.16% - Avg Wait Time: Windmill: 4.235s, Temporal: 2.115s, Hatchet: 19.187s Despite Hatchet having perfect task distribution (exactly 40 per worker), the wait time and idle time were extremely high, indicating tasks were not being effectively overlapped or queued. Its average task duration was very low (0.030s), but these were not leveraged due to orchestration bottlenecks. Note that for multi-worker scenarios, we tried both using `RunBulkNoWait` that splits up all individual fibonacci tasks as separate workflows and setting `PARALLEL=true` that runs all fibonacci tasks in parallel in a single workflow with parallel tasks. We did not see significant difference in the results. Temporal and Windmill both showed strong parallel execution, but with some differences: Windmill started scheduling tasks faster (first task scheduled at 0.025s), but took longer to start executing the first task (1.663s). This is due to the fact that Windmill, when executing flow iterations in parallel, will create a sub-flow for each iteration containing the fibonacci task. This creates a slight overhead in orchestration compared to the other engines. Temporal, by contrast, started execution almost immediately at 0.108s and maintained lower wait and idle times. --- ## Js Source: https://www.windmill.dev/docs/misc/benchmarks/competitors/results/js # Results: JavaScript ## Long-Running Tasks (10 tasks) When executing long-running tasks (`fibo(33)`), [Windmill](../../windmill/index.mdx) emerged as the **fastest orchestrator** overall, completing the full workflow in 0.935 seconds. [Temporal](../../temporal/index.mdx) followed closely at 0.966 seconds, while Windmill Dedicated took slightly longer at 1.077 seconds. [Kestra](../../kestra/index.mdx), in contrast, lagged behind at 2.919 seconds, more than twice as slow as the top performers. The execution phase dominated the runtime in this scenario, as expected for compute-intensive tasks. Windmill devoted 82.67% of its total time to execution, significantly higher than the others. Temporal also maintained a high execution ratio at 66.05%, while Kestra's share dropped to 60.98%. The overhead introduced by assignment and task transitions remained comparatively low for all orchestrators in this category. This reinforces the expectation that for longer tasks, the engine's overhead fades into the background, and raw computational throughput takes precedence. Windmill's ability to maintain minimal assignment (5.78%) and transition (11.55%) times is a strong indicator of its lightweight orchestration layer. Temporal also performed well, though it incurred slightly more scheduling overhead. Windmill Dedicated's assignment time was higher at 22.19%, likely reflecting startup or handoff costs in a [dedicated worker](../../../../../core_concepts/25_dedicated_workers/index.mdx) configuration, yet the system still completed the workflow with excellent overall timing. ## Lightweight Tasks (40 tasks) When shifting to lightweight tasks using `fibo(10)` — with each task lasting around 10 milliseconds — the dynamics changed dramatically. In this scenario, orchestration efficiency became the bottleneck, as the time spent managing tasks often exceeded the time spent executing them. Windmill Dedicated posted the **fastest total runtime** at 2.125 seconds, with Windmill right behind at 2.973 seconds. Temporal completed the flow in 3.063 seconds, while Kestra required 9.050 seconds—making it the slowest by a substantial margin. Unlike the long-running task case, here the execution phase was only a minor part of the total runtime: Execution time accounted for just 9.72% for Windmill, 5.93% for Windmill Dedicated, 8.81% for Temporal, and 51.19% for Kestra. These proportions confirm that engine overhead dominates lightweight workflows. Windmill and Windmill Dedicated both handled orchestration with high parallelism, but the dedicated setup incurred notably higher assignment overhead —84.05% of total time, compared to Windmill's 48.00%. Despite this, Windmill Dedicated still completed the full workflow quickly, likely due to parallel scheduling and nearly negligible transition time (10.02%). Temporal, while maintaining decent overall timing, exhibited the highest transition overhead (55.17%), which may be attributed to workflow state persistence or slower task chaining under rapid-fire conditions. Kestra underperformed again, consuming 30.69% of time in assignment and 18.12% in transition, along with longer-than-expected task execution durations. This suggests either less responsive workers or more rigid task dispatch intervals that cannot keep up with high-frequency scheduling even though we were using `io.kestra.plugin.core.runner.Process` rather than spawning a new container. Another notable difference is that Kestra's time to schedule the first task was the highest at 0.93 seconds, compared to the other orchestrators that were close to each other (0.08s for Windmill, 0.04s for Windmill Dedicated, 0.1s for Temporal). --- ## Python Source: https://www.windmill.dev/docs/misc/benchmarks/competitors/results/python # Results: Python ## Long-Running Tasks (10 tasks) At a macro level, the total time to complete the long-running task flow varied widely across orchestrators. [Airflow](../../airflow/index.mdx) was by far the slowest, taking 54.668 seconds, followed by [Kestra](../../kestra/index.mdx) at 15.786s, [Prefect](../../prefect/index.mdx) at 15.489s, and [Temporal](../../temporal/index.mdx) at 7.247s. [Hatchet](../../hatchet/index.mdx) and [Windmill](../../windmill/index.mdx) were the fastest in this scenario, with total durations of 7.793s and 8.347s respectively. When Windmill was run in [dedicated worker](../../../../../core_concepts/25_dedicated_workers/index.mdx) mode, it edged ahead slightly with a total of 7.701s. Execution time—defined as the period during which tasks were actively being processed by workers—dominated the total runtime for all engines, especially for Hatchet (96.21%), Temporal (96.56%), and Windmill (93.83%). This is expected given the computationally intensive nature of fibo(33). The higher execution ratios suggest these engines introduce minimal orchestration overhead and keep workers consistently busy. Airflow and Prefect, by contrast, spent significantly more time on assignment, consuming 40.35% and 9.77% of total runtime, respectively. This indicates slower dispatch of the initial tasks, especially noticeable before the parallelism benefits take effect. Despite this, Prefect still maintained decent performance compared to Airflow, though both trail behind more modern orchestrators. Windmill in dedicated mode exhibited slightly higher assignment time (4.80%) than its normal mode (5.13%), suggesting a shift of overhead from task execution to scheduling. Nonetheless, Windmill's transition time—the delay between finishing one task and initiating the next—was remarkably low at 1.04%, demonstrating highly efficient task chaining. Overall, the engines that most closely aligned with Windmill's dedicated worker architecture—namely Hatchet and Temporal—showed similar performance characteristics: tight scheduling, consistent task execution throughput, and minimal orchestration noise. ## Lightweight Tasks (40 tasks) We can exclude Airflow from the previous chart as it was performing much slower than the other orchestrators and focus on the other orchestrators: The lightweight task scenario produced a far starker contrast in performance. As expected, Airflow underperformed dramatically, taking 116.221 seconds to complete the 40-task flow. The next slowest, Kestra, completed in 6.044s, while Prefect followed at 4.872s. Temporal, Windmill, and Hatchet all performed significantly better, with durations of 2.967s, 4.383s, and 1.222s respectively. Hatchet delivered the fastest performance, completing the flow in just 1.222 seconds, followed by Windmill in dedicated mode at 2.092 seconds, and Temporal at 2.967 seconds. In lightweight scenarios, where each task executes in around 10ms, orchestration overhead becomes the dominant factor. Execution accounted for only a small portion of total time—just 11.19% for Temporal, 8.18% for Hatchet, and a mere 5.83% for Windmill in dedicated mode. The implication is that most of the runtime is now spent coordinating tasks, rather than executing them. Windmill, in normal mode, spent more time on task execution (50.54%) compared to other engines. This is due to the way Windmill handles task startup—using isolated, "cold-started" task containers. As a result, each task includes some initialization cost, making Windmill slightly slower than Hatchet and Temporal in this lightweight test. However, this changes when Windmill is run in dedicated worker mode. In this configuration, startup overhead is minimized, and orchestration becomes more efficient. Execution time drops to 5.83%, and assignment jumps to 85.80%, resembling the tight loop style seen in Temporal and Hatchet. Transition times in this scenario further highlight orchestration differences. Windmill again stood out with one of the lowest transition delays—only 7.57% in standard mode and 8.37% in dedicated. Temporal’s transition overhead was higher (53.15%), which may point to internal mechanisms such as durable state recording. Hatchet also showed a relatively high transition cost (54.91%), which is interesting given its otherwise strong performance. These results confirm that Windmill-dedicated, Hatchet, and Temporal are the top performers in lightweight task orchestration, where internal engine latency dominates. Windmill, despite its cold-start architecture in normal mode, holds up well and excels in transition responsiveness. Prefect and Kestra show adequate performance in both cases but are less consistent under varying load. Airflow, though functional, is considerably slower in both test scenarios and appears less suitable for modern, latency-sensitive workflows. --- ## Scaling Source: https://www.windmill.dev/docs/misc/benchmarks/competitors/results/scaling # Scaling Windmill workers We performed those benchmarks with a single worker assuming the capacity to process jobs would scale linearly with the number of workers deployed on the stack (conclusions [here](../conclusion/index.mdx)). We haven't verified this assumption for [Airflow](../../airflow/index.mdx), [Prefect](../../prefect/index.mdx), [Kestra](../../kestra/index.mdx) and [Temporal](../../temporal/index.mdx), but we've scaled Windmill up to a 100 virtual workers to verify. And the conclusion is that it scales pretty linearly. For this test, we've deployed the same docker compose as earlier on an AWS `m4.xlarge` instance (4 vCPU, 16Gb of memory) and to virtually increase the number of workers, we've used the `NUM_WORKERS` environment variable Windmill accepts. Note that it is not strictly equivalent to adding real hardware to the stack, but until we reach the maximum capacities on the instance, both in terms of CPU and memory, we can assume it's a good approximation. The other change we had to make was to bump the `max_connections` to `1000` on Postgresql: as we're adding more and more workers, each worker needs to connect to the database and we need to increase the maximum number of connections Posgtresql allows. The job we ran was a simple sleeping job sleeping for 100ms, which is a good average during for a job running on an orchestrator. ```python import time def main(): time.sleep(0.1) ``` Finally, we've ran it on Windmill Dedicated Worker mode, and we used a specific endpoint to "bulk-create" the jobs before any worker can start pulling them from the queue. For this test to be representative, we had to measure the performance of Windmill processing a large number of jobs (10000 in this case), and we quickly realised that the time it was taking to only _insert_ the jobs one by one in the queue was non negligible and was affecting the real performance of workers. The results are the following: | **Number of workers** | **Throughput (jobs/sec) batch of 10K jobs** | | --------------------- | ------------------------------------------- | | 2 | 19.9 | | 6 | 59.8 | | 10 | 99.6 | | 20 | 198 | | 30 | 298 | | 40 | 391 | | 50 | 496 | | 60 | 591 | | 70 | 693 | | 80 | 786 | | 90 | 887 | | 100 | 981 | This proves that Windmill scales linearly with the number of workers (at least up to 100 workers). We can also notice that the throughput is close to the optimal: given that the job takes 100ms to be executed, N workers processing the jobs in parallel can't go above `N*100` jobs per seconds, and Windmill is pretty close. --- ## Temporal benchmark Source: https://www.windmill.dev/docs/misc/benchmarks/competitors/temporal # Temporal benchmarks ## Summary [Temporal](https://temporal.io/) consistently showed strong performance across the board, particularly in Go and multi-worker scenarios. It handled lightweight tasks efficiently with low wait times and high utilization. A great fit for teams needing low-latency, high-throughput orchestration, especially when running parallel workloads at scale. ## Temporal setup We set up Temporal version 2.34.0 using the [docker-compose.yml from the official GitHub repository](https://github.com/temporalio/docker-compose). --- ## Windmill benchmark Source: https://www.windmill.dev/docs/misc/benchmarks/competitors/windmill # Windmill benchmarks ## Summary [Windmill](/) emerged as the most balanced and versatile orchestrator. In normal mode, it excelled at long-running tasks with minimal overhead. In dedicated worker mode, it matched or surpassed the performance of [Temporal](../temporal/index.mdx) and [Hatchet](../hatchet/index.mdx) for lightweight, parallel workloads. With smart caching, cold-start isolation, and adaptive behavior, it consistently delivered top-tier performance across runtimes and load types. ## Windmill setup We set up Windmill version 1.483.1 using the [docker-compose.yml from the official GitHub repository](https://github.com/windmill-labs/windmill). We made some adjustments to it to have a similar setup compared to the other orchestrators. We executed the Windmill benchmarks in both "normal" and "[dedicated worker](../../../../core_concepts/25_dedicated_workers/index.mdx)" mode. To implement the flows in Windmill, we first created a script simply computing the Fibonacci numbers: ```python # WINDMILL script: `u/benchmarkuser/fibo_script` def fibo(n: int): if n <= 1: return n else: return fibo(n - 1) + fibo(n - 2) def main( n: int, ): return fibo(n) ``` ```javascript // WINDMILL script: `u/benchmarkuser/fibo_script_ts` function fibo(n) { if (n <= 1) { return n } else { return fibo(n-1) + fibo(n-2) } } ``` And then we used this script in a simple flow composed of a For-Loop sequentially executing the scripts. The JSON representation of the flow is as follow: ```yaml summary: Fibonacci benchmark flow description: Flow running 10 (resp. 40) times Fibonacci of 33 (resp. 10) value: modules: - id: a value: type: forloopflow modules: - id: b value: path: u/admin/fibo_script # respectively u/admin/fibo_script_ts or u/admin/fibo_script_go type: script input_transforms: n: type: static value: 33 # respectively 10 iterator: expr: Array(10) # respectively 40 type: javascript parallel: false # respectively true for more than one worker skip_failures: true schema: '$schema': https://json-schema.org/draft/2020-12/schema properties: {} required: [] type: object ``` When using more than one worker, we're also using the `parallel` parameter so that each iteration of the for-loop is executed in parallel. Under the hood, Windmill will spawn a sub-flow with a single job for each iteration. --- ## Contributor guide Source: https://www.windmill.dev/docs/misc/contributing # Contributor guide As an open-source platform, Windmill relies on the developer community for enhancements and support. Your work, from coding to writing docs, directly improves our platform. This guide outlines how you can get involved. Here are some ways you can contribute to our platform: - **Share Your Work**: [Upload your Scripts, Flows, Apps, and Resource types](../1_share_on_hub/index.md) to Windmill Hub. Approved high-quality submissions are accessible for community use. - **Code Contributions**: Enhance the [Windmill codebase](https://github.com/windmill-labs/windmill) by submitting pull requests (PRs) on GitHub. Bug reports and feature requests are valuable—please file an issue to start the conversation. - **Community Engagement**: Join our [Discord community](https://discord.com/invite/V7PM2YHsPB) to offer suggestions, assist others, or discuss your ideas. ## Security bounty program :::caution Temporarily on hold Our security bounty program is paused while we revamp our static analysis methodology around LLMs. We still welcome vulnerability reports at security@windmill.dev, but monetary rewards are suspended in the meantime. ::: We are committed to rewarding white hat hackers who help us by identifying and reporting significant security vulnerabilities. **Eligibility and Rewards**: We offer rewards of up to $2,500 for the discovery and reporting of severe security flaws that could potentially impact the integrity, confidentiality, or availability of our services. The reward amount is determined by the severity and impact of the vulnerability. **Reporting Process**: To report a vulnerability, please send a detailed description, including steps to reproduce it, to security@windmill.dev. Our team will work with you to assess the report and, if validated, make the necessary fixes. **Guidelines**: We ask that you act responsibly, not disclose the vulnerability publicly or to third parties before it is fixed, and give us at least 48 hours to address the issue. Thank you for helping us keep Windmill secure. ## Expanding Windmill's integrations: adding new OAuth providers To enhance Windmill's connectivity and integration capabilities, we welcome contributions that add new OAuth providers. This not only broadens the range of services Windmill can interact with but also directly impacts the platform's functionality and user experience. **How to Contribute a New OAuth Provider**: Submit a Pull Request: Add your new OAuth provider configuration to the [backend/oauth_connect.json](https://github.com/windmill-labs/windmill/blob/main/backend/oauth_connect.json) file with the following format: ```json "": { "auth_url": "", "token_url": "", "scopes": , "extra_params": { "": "", } }, ``` Where `extra_params` is an escape hatch to deal with OAuth provider that need some extra fields to be passed along to the authorization URL. You can iterate without requiring a dev setup. The item accepts an extra optional field: `connect_config` or `login_config` of type OAuthConfig: ``` interface OAuthConfig { auth_url: string, token_url: string, userinfo_url?: string, scopes?: string[], extra_params?: Record, extra_params_callback?: Record, req_body_auth?: bool } ``` `connect_config` is used for resources, and `login_config` for SSO. ## Mapping python imports Python can automatically [infer requirements from imports](../../advanced/15_dependencies_in_python/index.mdx). However it is not always accurate because import can mismatch with the requirement. To handle this case, there is [import map](https://github.com/windmill-labs/windmill/blob/main/backend/parsers/windmill-parser-py-imports/src/mapping.rs). You can help us and others by adding new entries there and opening PR. Let's take a look at simple example **1. Find problematic import** ```python import git def main(): ... ``` It will fail with error indicates either `git` cannot be resolved or `git` module cannot be imported. **2. Pin it** Use one of the [pinning methods](../../advanced/15_dependencies_in_python/index.mdx#pinning-dependencies-and-requirements) to override requirement ```python import git # pin: GitPython def main(): ... ``` **3. Add entry to global map** Navigate to [mappings](https://github.com/windmill-labs/windmill/blob/main/backend/parsers/windmill-parser-py-imports/src/mapping.rs) and add new entry to the `SHORT_IMPORTS_MAP` ```rust pub static SHORT_IMPORTS_MAP: PyMap = phf_map! { ... "git" => "GitPython", }; ``` **4. Open PR** We appreciate every contribution to Windmill! **Special cases** Sometimes dependencies require to be imported separated by `.` Let's take a look at one of those on [azure-storage-blob](https://pypi.org/project/azure-storage-blob/) example ```python import azure.storage.blob # pin: azure-storage-blob ``` As you can see this entire import needs to be mapped and not just `azure` part of it. To finalize map for everyone, add this entry in the [mappings](https://github.com/windmill-labs/windmill/blob/main/backend/parsers/windmill-parser-py-imports/src/mapping.rs). But this time add it to `FULL_IMPORTS_MAP` ```rust pub static FULL_IMPORTS_MAP: PyMap = phf_map! { ... "azure.storage.blob" => "azure-storage-blob", }; ``` --- ## Full text search Source: https://www.windmill.dev/docs/misc/full_text_search # Full text search on jobs and logs Windmill offers the functionality to do full-text search on jobs (across args, logs, results, ...) and service logs. In order to access this functionality, the instance must be running the windmill indexer service, which is powered by the super fast search engine written in rust, [Tantivy](https://github.com/quickwit-oss/tantivy) ## How to run the indexer service ### Setup using docker compose On the Windmill's docker-compose.yml there is an example of how to setup the indexer container to enable full text search, just make sure to change replicas from 0 to 1. :::warning The replicas should be set to exactly one and not more, only one index writer can exist at a time and having multiple will not result in the expected behavior. ::: ```yml # The indexer powers full-text job and log search, an EE feature. windmill_indexer: image: ${WM_IMAGE} pull_policy: always deploy: replicas: 1 # set to 1 to enable full-text job and log search restart: unless-stopped expose: - 8001 environment: - PORT=8001 - DATABASE_URL=${DATABASE_URL} - MODE=indexer depends_on: db: condition: service_healthy volumes: - windmill_index:/tmp/windmill/search ``` The indexer is in charge of both indexing new jobs and answering search queries. Because of this, we also need to redirect search requests to this container instead of the normal windmill server. This is what it looks like if you're using Caddy: ```Caddyfile {$BASE_URL} { bind {$ADDRESS} reverse_proxy /ws/* http://lsp:3001 # reverse_proxy /ws_mp/* http://multiplayer:3002 reverse_proxy /api/srch/* http://windmill_indexer:8001 reverse_proxy /* http://windmill_server:8000 # tls /certs/cert.pem /certs/key.pem } ``` Redirecting requests prefixed by /api/srch to port 8001 (same port as in the docker-compose.yml) ### Setup using Helm charts On Kubernetes, the [Windmill Helm chart](https://github.com/windmill-labs/windmill-helm-charts) deploys the indexer for you. The indexer is enabled by default in the chart but is only deployed on [Enterprise Edition](/pricing) instances, so both of the following must be set in your `values.yaml`: ```yaml enterprise: enabled: true windmill: indexer: enabled: true # default ``` The chart runs a single indexer replica (only one index writer can exist at a time) and its ingress automatically routes search requests (`/api/srch/*`) to the indexer service, so no manual reverse proxy configuration is needed. Resource limits, node selectors, tolerations and extra environment variables for the indexer pod can be configured under `windmill.indexer` in the chart's [values.yaml](https://github.com/windmill-labs/windmill-helm-charts/blob/main/charts/windmill/values.yaml). The indexer pod uses ephemeral storage by default, so the index is rebuilt from scratch on every restart. To avoid full reindexing, set up object storage so the index is backed up to and restored from S3 (see [Index persistence](#index-persistence)). ## Configure the indexer service ### Indexer settings The index can be configured in the Instance Settings. Note that the default values should work for most use cases ![Indexer Settings](./indexer_settings.png) | Setting name | default | description | | -------------------------------------- | -------- | ----------- | | Index writer memory budget (MB) | 300 MB | How much memory the writer can use before writing to disk. Increasing it can improve indexing throughput. | | Commit max batch size | 100000 | How many documents to include at most per commit. This is mostly relevant for the first time indexing. A large value will result in less commits, i.e. faster and more efficient indexing, but results will be available only once their commits are completed. | | Refresh index period (s) | 300s | The indexer will periodically fetch the latest jobs and write them to the index. A shorter period means new jobs/logs are available for search faster, but also results in more and more frequent writes to s3. | | Max indexed job log size (MB) | 1 MB | Job logs bigger than this will be truncated before indexing. | ### Index persistence There are two ways to make the index persistent (and avoid reindexing all jobs at every restart). The recommended way is to setup an object storage such as Amazon S3, and the index will automatically be backed up and pulled from there. This can be done by setting up [S3/Azure for python cache and large logs](../../core_concepts/38_object_storage_in_windmill/index.mdx#large-job-logs-management). It is also possible to store the index in a volume attached to the indexer container. The docker-compose.yml serves as an example of how to set it up (on `/tmp/windmill/search`). ## Using full text search Learn how full text search can be used to find [completed jobs](../../core_concepts/35_search_bar/index.mdx#searching-runs) and [service logs](../../core_concepts/36_service_logs/index.mdx#log-search) --- ## Getting help Source: https://www.windmill.dev/docs/misc/getting_help # Contact / Getting help Getting stuck using Windmill? Your problem can be quickly solved and reporting it is the best way to contribute! First of all, you could join our [Discord](https://discord.com/invite/V7PM2YHsPB), from where the team and even other contributors will be happy to give you a hand. From GitHub, feel free to open an [Issue](https://github.com/windmill-labs/windmill/issues), or if you have a hint, a [Pull Request](https://github.com/windmill-labs/windmill/pulls). Also, you might want to reach out by email at contact@windmill.dev. For more in-depth discussions, reach out at sales@windmill.dev or schedule a [meeting](https://www.windmill.dev/book-demo) with the founder. --- ## Build a Google ADK agent in an AI sandbox Source: https://www.windmill.dev/docs/misc/guides/adk_agent # Build a Google ADK agent in an AI sandbox This guide walks you through running a [Google Agent Development Kit (ADK)](https://google.github.io/adk-docs/) agent on Windmill as a single script. The agent runs inside an [AI sandbox](/docs/core_concepts/ai_sandbox), keeps its session state in a [volume](/docs/core_concepts/volumes), and exposes a clean `main()` entry point you can call from a [flow](/docs/flows/flow_editor), a [trigger](/docs/triggers), or the UI. ADK is a code-first framework for building, evaluating, and orchestrating agents. It is optimized for [Gemini](https://aistudio.google.com/) but model-agnostic, so the same script works with other providers via [LiteLLM](https://google.github.io/adk-docs/agents/models/#non-google-models). The same patterns shown here work with both the Python ([`google-adk`](https://pypi.org/project/google-adk/)) and the TypeScript ([`@google/adk`](https://www.npmjs.com/package/@google/adk)) ports. ## What you'll build A weather assistant agent that: - Defines a single ADK agent with one function tool. - Runs inside an [nsjail](/docs/advanced/security_isolation#nsjail-sandboxing) sandbox so the agent process is isolated from the worker. - Persists session history to a SQLite database stored in a [volume](/docs/core_concepts/volumes), so subsequent runs resume where the conversation left off. - Takes a `prompt` and a `googleai` [resource](/docs/core_concepts/resources_and_types) as script inputs and returns the agent's final response. ## Prerequisites - A Windmill instance with [workspace object storage](/docs/core_concepts/object_storage_in_windmill#workspace-object-storage) configured (required for volumes). - [Workers](/docs/core_concepts/worker_groups) with `nsjail` available (included in standard Windmill Docker images). - A Google AI Studio API key — get one at [aistudio.google.com/apikey](https://aistudio.google.com/apikey). - A `googleai` resource holding the API key. The `googleai` [resource type](https://hub.windmill.dev/resource_types/222/googleai) ships on the Windmill Hub; create a resource of that type and paste your key into `api_key`. ## Step 1: Create the script Create a new script and paste the code for your language. The two annotations at the top are the only Windmill-specific parts — everything else is plain ADK. ```python # sandbox # volume: adk-state .adk #requirements: #google-adk import asyncio import os from pathlib import Path from typing import TypedDict from google.adk.agents import Agent from google.adk.runners import Runner from google.adk.sessions.sqlite_session_service import SqliteSessionService from google.genai import types class googleai(TypedDict): api_key: str base_url: str platform: str def get_weather(city: str) -> dict: """Return the current weather for a given city. Args: city: The city name to look up. """ data = { "paris": {"temp_c": 18, "conditions": "Sunny"}, "tokyo": {"temp_c": 22, "conditions": "Cloudy"}, "new york": {"temp_c": 12, "conditions": "Rainy"}, } info = data.get(city.lower()) if info is None: return {"status": "error", "message": f"No weather data for {city}."} return {"status": "success", "city": city, **info} root_agent = Agent( name="weather_agent", model="gemini-2.5-flash", description="Assistant that answers questions about the current weather.", instruction=( "You are a friendly weather assistant. " "Call the get_weather tool when the user asks about weather. " "If a city is not supported, suggest one of: Paris, Tokyo, New York." ), tools=[get_weather], ) APP_NAME = "windmill_adk_demo" USER_ID = "windmill_user" STATE_DIR = Path(".adk") SESSION_FILE = STATE_DIR / "session-id.txt" DB_FILE = STATE_DIR / "sessions.db" async def _run(prompt: str) -> dict: STATE_DIR.mkdir(parents=True, exist_ok=True) # SQLite-native service (REAL-typed timestamps, plain aiosqlite). ADK's # generic DatabaseSessionService is SQLAlchemy-based and aimed at # postgres/mysql; on SQLite it can fail on resume with # `fromisoformat: argument must be str`. session_service = SqliteSessionService(str(DB_FILE)) saved_id = SESSION_FILE.read_text().strip() if SESSION_FILE.exists() else None session = None if saved_id: session = await session_service.get_session( app_name=APP_NAME, user_id=USER_ID, session_id=saved_id ) is_resume = session is not None if not is_resume: session = await session_service.create_session( app_name=APP_NAME, user_id=USER_ID ) SESSION_FILE.write_text(session.id) runner = Runner( agent=root_agent, app_name=APP_NAME, session_service=session_service, ) new_message = types.Content( role="user", parts=[types.Part.from_text(text=prompt)] ) response = "" async for event in runner.run_async( user_id=USER_ID, session_id=session.id, new_message=new_message ): if event.content and event.content.parts: for part in event.content.parts: if part.text: response += part.text return { "is_resume": is_resume, "session_id": session.id, "prompt": prompt, "response": response, } def main(gemini: googleai, prompt: str = "What's the weather in Paris?") -> dict: # Script inputs are not env vars — set them explicitly. # See "Passing credentials" below. os.environ["GOOGLE_API_KEY"] = gemini["api_key"] if gemini.get("platform") == "google_vertex_ai": os.environ["GOOGLE_GENAI_USE_VERTEXAI"] = "true" if gemini.get("base_url"): os.environ["GOOGLE_GEMINI_BASE_URL"] = gemini["base_url"] return asyncio.run(_run(prompt)) ``` ```typescript // sandbox // volume: adk-state .adk import * as fs from 'node:fs' import * as path from 'node:path' const getWeather = new FunctionTool({ name: 'get_weather', description: 'Get the current weather for a city.', parameters: z.object({ city: z.string().describe('The city name.'), }), execute: async ({ city }: { city: string }) => { const data: Record = { paris: { temp_c: 18, conditions: 'Sunny' }, tokyo: { temp_c: 22, conditions: 'Cloudy' }, 'new york': { temp_c: 12, conditions: 'Rainy' }, } const info = data[city.toLowerCase()] if (!info) return { status: 'error', message: `No weather data for ${city}.` } return { status: 'success', city, ...info } }, }) const rootAgent = new LlmAgent({ name: 'weather_agent', model: 'gemini-2.5-flash', description: 'Assistant that answers questions about the current weather.', instruction: 'You are a friendly weather assistant. ' + 'Call the get_weather tool when the user asks about weather. ' + 'If a city is not supported, suggest one of: Paris, Tokyo, New York.', tools: [getWeather], }) const APP_NAME = 'windmill_adk_demo' const USER_ID = 'windmill_user' const STATE_DIR = '.adk' const SESSION_FILE = path.join(STATE_DIR, 'session-id.txt') const DB_FILE = path.join(STATE_DIR, 'sessions.db') export async function main( gemini: RT.Googleai, prompt: string = "What's the weather in Paris?" ) { // adk-js reads GEMINI_API_KEY (or GOOGLE_GENAI_API_KEY) — script inputs // are not env vars, set them explicitly. See "Passing credentials" below. process.env.GEMINI_API_KEY = gemini.api_key process.env.GOOGLE_GENAI_API_KEY = gemini.api_key if (gemini.platform === 'google_vertex_ai') { process.env.GOOGLE_GENAI_USE_VERTEXAI = 'true' } if (gemini.base_url) { process.env.GOOGLE_GEMINI_BASE_URL = gemini.base_url } fs.mkdirSync(STATE_DIR, { recursive: true }) const sessionService = new DatabaseSessionService(`sqlite://${DB_FILE}`) const savedId = fs.existsSync(SESSION_FILE) ? fs.readFileSync(SESSION_FILE, 'utf-8').trim() : undefined let session = savedId ? await sessionService.getSession({ appName: APP_NAME, userId: USER_ID, sessionId: savedId, }) : undefined const isResume = !!session if (!session) { session = await sessionService.createSession({ appName: APP_NAME, userId: USER_ID, }) fs.writeFileSync(SESSION_FILE, session.id) } const runner = new Runner({ appName: APP_NAME, agent: rootAgent, sessionService, }) let response = '' for await (const event of runner.runAsync({ userId: USER_ID, sessionId: session.id, newMessage: createUserContent(prompt), })) { for (const part of event.content?.parts ?? []) { if (part.text) response += part.text } } return { is_resume: isResume, session_id: session.id, prompt, response, } } ``` ## Step 2: Run it Click **Test** with a prompt like `"What's the weather in Paris?"` and your API key. You should see something like: ```json { "is_resume": false, "session_id": "51b81888-567e-4f46-9190-dff1e1c4eefd", "prompt": "What's the weather in Paris?", "response": "The weather in Paris is Sunny with a temperature of 18 degrees Celsius." } ``` Now run it again with `"And in Tokyo?"`. The script reads the saved session ID from the volume, resumes the same conversation, and the agent answers based on the prior turn: ```json { "is_resume": true, "session_id": "51b81888-567e-4f46-9190-dff1e1c4eefd", "prompt": "And in Tokyo?", "response": "Tokyo is cloudy at 22°C." } ``` ## How it works ### `# sandbox` / `// sandbox` Wraps the job process in [nsjail](/docs/advanced/security_isolation#nsjail-sandboxing). The agent — and any subprocess it spawns (e.g. an MCP server or shell tool) — sees an isolated filesystem and cannot reach the worker's secrets or other jobs. ### `# volume: adk-state .adk` / `// volume: adk-state .adk` Mounts a persistent volume at `./.adk` relative to the job's working directory. Files written there survive across runs and are synced to [object storage](/docs/core_concepts/object_storage_in_windmill). The script uses two files in the volume: - `.adk/session-id.txt` — the active ADK session ID, so the next run resumes the same conversation. - `.adk/sessions.db` — a SQLite database managed by ADK's `DatabaseSessionService`, holding the full session history. If you want a fresh conversation, delete the volume contents (or use a different volume name). ### `DatabaseSessionService` ADK ships several [session services](https://google.github.io/adk-docs/sessions/session/). `InMemorySessionService` loses state at the end of each job — useless across Windmill runs. `DatabaseSessionService` with a SQLite URL inside the volume gives you persistence with zero infrastructure. The two ports do not use the same SQLite implementation: - **Python** — use `SqliteSessionService` (in `google.adk.sessions.sqlite_session_service`). It's a SQLite-native service that stores timestamps as `REAL` and goes through `aiosqlite` directly, sidestepping a SQLAlchemy + aiosqlite type-marshalling bug that surfaces on resume as `fromisoformat: argument must be str`. The generic `DatabaseSessionService` is fine for postgres/mysql but should not be used with SQLite. - **TypeScript** — `DatabaseSessionService` works with a `sqlite://` URL via MikroORM. No extra dependency needed. For multi-user or multi-tenant setups, scope the volume per workspace or input: ``` // volume: $workspace-adk-state .adk ``` See [dynamic volume names](/docs/core_concepts/volumes#dynamic-volume-names) for the full set of placeholders. ### Passing credentials Script inputs are language variables — they are **not** automatically exported as environment variables inside the sandbox. ADK reads its API key from the environment, so the scripts set it explicitly: - **Python** (`google-adk`) reads `GOOGLE_API_KEY`. - **TypeScript** (`@google/adk`) reads `GEMINI_API_KEY` or `GOOGLE_GENAI_API_KEY`. Apply the same pattern for any other credential the agent or its tools need at runtime — for example `OPENAI_API_KEY`, `TAVILY_API_KEY`, or an MCP server token. The script also forwards the resource's optional `platform` and `base_url` fields: - `platform: "google_vertex_ai"` flips ADK to Vertex mode by setting `GOOGLE_GENAI_USE_VERTEXAI=true`. Vertex needs `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` set on the worker (or wired through additional script inputs). - `base_url` is exported as `GOOGLE_GEMINI_BASE_URL` for users running behind a proxy or against a non-default endpoint. Most users leave both fields empty. ## Extending the agent ### Use a different model ADK is model-agnostic. Both ports support [LiteLLM](https://google.github.io/adk-docs/agents/models/#non-google-models) (Python) or compatible model wrappers (TypeScript) to swap the underlying provider. To run the same agent on Anthropic Claude in Python, replace the `model=` line: ```python from google.adk.models.lite_llm import LiteLlm root_agent = Agent( name="weather_agent", model=LiteLlm("anthropic/claude-3-5-sonnet-20241022"), ... ) ``` Then export `ANTHROPIC_API_KEY` (or the relevant provider key) the same way as `GOOGLE_API_KEY`. If you already have a Windmill `anthropic` [resource](/docs/core_concepts/resources_and_types), accept it as a script input: ```python def main(anthropic: dict, prompt: str = "..."): os.environ["ANTHROPIC_API_KEY"] = anthropic["apiKey"] return asyncio.run(_run(prompt)) ``` ### Compose multiple agents ADK supports [multi-agent systems](https://google.github.io/adk-docs/agents/multi-agents/) out of the box — define sub-agents and assign them to a coordinator: ```python greeter = Agent(name="greeter", model="gemini-2.5-flash", instruction="...", description="...") researcher = Agent(name="researcher", model="gemini-2.5-flash", instruction="...", description="...", tools=[...]) root_agent = Agent( name="coordinator", model="gemini-2.5-flash", description="Routes user requests to the right specialist.", sub_agents=[greeter, researcher], ) ``` The runner code stays unchanged — ADK handles the routing and tool calls. ### Add MCP tools ADK can consume any [Model Context Protocol](https://modelcontextprotocol.io/) server via `MCPToolset`. Because the sandbox isolates subprocesses, you can safely launch MCP servers inside the same job: ```python from google.adk.tools.mcp_tool.mcp_toolset import MCPToolset, StdioServerParameters tools = await MCPToolset.from_server( connection_params=StdioServerParameters(command="npx", args=["@modelcontextprotocol/server-filesystem", "."]) ) ``` Pair this with a separate volume for the MCP server's working directory if it needs persistent state. ### Use the agent inside a flow Drop this script into any [flow](/docs/flows/flow_editor) step and chain it with other Windmill primitives — fetch context from a [database](/docs/integrations/postgresql), call the agent, then post the response to [Slack](/docs/integrations/slack). The volume keeps the agent's memory consistent across flow runs. ## Troubleshooting - **`#sandbox` annotation but nsjail is not available** — your worker image does not include nsjail. Use a standard Windmill Docker image or remove the annotation. - **Volume not persisting** — check that workspace object storage is configured under [instance settings](/docs/advanced/instance_settings). Without it, volumes only exist for the duration of a single job. - **`fromisoformat: argument must be str`** (Python, on resume) — you used `DatabaseSessionService` against SQLite, which hits a SQLAlchemy + aiosqlite type-marshalling bug. Switch to `SqliteSessionService` from `google.adk.sessions.sqlite_session_service` (the snippet above already does this) and delete the old `.adk/sessions.db`. - **`API key must be provided via constructor or GOOGLE_GENAI_API_KEY or GEMINI_API_KEY`** (TypeScript) — the JS port reads `GEMINI_API_KEY` / `GOOGLE_GENAI_API_KEY`, not `GOOGLE_API_KEY`. Set both for portability. --- ## Aggrid table Source: https://www.windmill.dev/docs/misc/guides/aggrid_table # AgGrid table guide :::info Legacy This guide uses the legacy low-code app editor. For new apps, we recommend [full-code apps](../../../full_code_apps/index.mdx) with React or Svelte. ::: This is a basic introduction on how to use [AgGrid table](https://www.ag-grid.com/) together with Windmill. It assumes little to no knowledge about AgGrid. ![AgGrid Overview](../../../../static/img/guide/aggrid_overview.png.webp) ## What is AgGrid table The [AgGrid table component](../../../apps/4_app_configuration_settings/aggrid_table.mdx) (called AgGrid from here) is a small wrapper around a fantastic library called [AgGrid](https://www.ag-grid.com/). It provides you with a lot of advanced features. It comes in a free and an Enterprise (paid) version. All features below are part of the free version of AgGrid. :::tip Enterprise If you need the enterprise version of AgGrid, please [contact us](../../6_getting_help/index.mdx). ::: ## AgGrid vs Table component vs Database studio In Windmill there are 3 table components: one simply called [Table](../../../apps/4_app_configuration_settings/table.mdx), [AgGrid](../aggrid_table/index.md) and [Database studio](../../../apps/4_app_configuration_settings/database_studio.mdx). The [Table component](../../../apps/4_app_configuration_settings/table.mdx) covers most use cases. It takes an array of objects as input and uses the keys of the objects as the headers of the table. It also provides you with one or more action buttons to trigger an action for the row or create a dropdown button based on the row data. [Database studio](../../../apps/4_app_configuration_settings/database_studio.mdx) is a web-based database management tool. It allows you to display and edit the content of a database. :::info Transformer If you want to do basic sorting, or edit the column header name from the script you can also use a Transformer script. See the [documentation](../../../apps/3_app-runnable-panel.mdx#transformer) for more information. ::: ## Column definition AgGrid needs two inputs, rowdata and column definitions. By default AgGrid does not show the rowdata. You need to specify the properties of each column. This is done in the Configuration on the right side. ![Column definitions](./../../../../static/img/guide/aggrid-column-definition-menu.png.webp) You can statically set the properties of each column and many properties are available. Let's walk through the ones you are most likely to use: - field (string) - which field to use from the rowdata | [documentation](https://www.ag-grid.com/javascript-data-grid/column-properties/#reference-columns-field) - headerName (string) - rename the column header to something other than the field name | [documentation](https://www.ag-grid.com/javascript-data-grid/column-properties/#reference-header-headerName) - sortable (boolean) - should the column be sortable by the user | [documentation](https://www.ag-grid.com/javascript-data-grid/column-properties/#reference-sort-sortable) - sort (asc|desc) - which order to sort the column | [documentation](https://www.ag-grid.com/javascript-data-grid/column-properties/#reference-sort-sort) - resizable (boolean) - should the column be resizable | [documentation](https://www.ag-grid.com/javascript-data-grid/column-properties/#reference-width-resizable) - rowDrag (boolean) - should the user be able to drag the column | [documentation](https://www.ag-grid.com/javascript-data-grid/column-properties/#reference-row%20dragging-rowDrag) There are _a lot_ more properties; these are just the first few. See [Column Properties](https://www.ag-grid.com/javascript-data-grid/column-properties/) for the complete list of properties. ### Dynamically configure the column definition As with most things, Windmill lets you **dropdown to code** when you want to do more advanced stuff, where the GUI is more in the way than helping. The Windmill way is to first create a background runnable and then connect it to the column definition. ![Column definitions](./../../../../static/img/guide/coldef-connect.png.webp) Here you can provide your own column definition that is more than just JSON and can also include classes and functions. Below is a series of snippets that will help you get started. #### Provide default values for all columns Create a [background runnable](../../../apps/3_app-runnable-panel.mdx#background-runnables) that is a [Frontend JavaScript](../../../apps/3_app-runnable-panel.mdx#frontend-scripts) with the following content: ```js const columnDef = [ { field: 'name', headerName: 'Full name' }, { field: 'age', sortable: false } ]; const defaultColumnProperties = { sortable: true }; return columnDef.map((col) => ({ ...defaultColumnProperties, ...col })); ``` ![Column Def Script](./column_def1.png ) #### Create a select button Here we are using an AgGrid component called `agSelectCellEditor`. There [are more predefined cell components](https://www.ag-grid.com/javascript-data-grid/provided-cell-editors/#select-cell-editor) like this. Create a [Frontend JavaScript](../../../apps/3_app-runnable-panel.mdx#frontend-scripts) with the following content: ```js return [ { field: 'name', headerName: 'Full name' }, { field: 'age', cellEditorParams: function (params) { return { values: [1, params.data.age, 100] }; }, cellEditor: 'agSelectCellEditor', editable: true, useFormatter: true } ]; ``` and [connect it](../../../apps/2_connecting_components/index.mdx) to the column definition. If you want to act on changes in the select dropdown, you may use the components states `newChanges`, or `selectedRow`. ![Aggrid component state](./../../../../static/img/guide/aggrid-state.png.webp) ### Create a button (custom component) Create a [Frontend JavaScript](../../../apps/3_app-runnable-panel.mdx#frontend-scripts) with the following content: ```js class BtnCellRenderer { constructor() {} init(params) { this.params = params; this.eGui = document.createElement('button'); this.eGui.innerHTML = 'Push me!'; this.btnClickedHandler = this.btnClickedHandler.bind(this); this.eGui.addEventListener('click', this.btnClickedHandler); } getGui() { return this.eGui; } destroy() { this.eGui.removeEventListener('click', this.btnClickedHandler); } } BtnCellRenderer.prototype.btnClickedHandler = function () { if (!state.logs) { state.logs = []; } state.logs.push({ rowDataAfterChange: JSON.stringify(this.params.data) }); }; return [ { field: 'name' }, { field: 'age' }, { headerName: 'Send row data to state', cellRenderer: BtnCellRenderer } ]; ``` and [connect it](../../../apps/2_connecting_components/index.mdx) to the column definition. This example demonstrates how to create a custom cell renderer component, where we put the data from the row into our state. From there, we can use other scripts to act on the row data as we like. You can create a lot of components, not only for cell rendering. To understand how, check out [AgGrid's documentation for components](https://www.ag-grid.com/javascript-data-grid/components/). --- ## App send email smtp Source: https://www.windmill.dev/docs/misc/guides/app_send_email_smtp # Build an App that sends email with SMTP :::info Legacy This guide uses the legacy low-code app editor. For new apps, we recommend [full-code apps](../../../full_code_apps/index.mdx) with React or Svelte. ::: Watch this video on how building a Windmill [App](../../../apps/0_app_editor/index.mdx) that uses a [SMTP resource](../../../integrations/smtp.md) to send an email [connecting components](../../../apps/2_connecting_components/index.mdx) ([Button](../../../apps/4_app_configuration_settings/button.mdx) & [Text input](../../../apps/4_app_configuration_settings/text_input.mdx)). --- ## Build an AI Discord bot with WebSocket triggers Source: https://www.windmill.dev/docs/misc/guides/discord_bot # Build an AI Discord bot with WebSocket triggers This guide walks you through building a Discord bot that connects to the Discord Gateway via Windmill's WebSocket trigger, uses the application-level heartbeat feature to maintain the connection, and responds to messages using the [Claude Agent SDK](https://docs.anthropic.com/en/docs/agents/claude-agent-sdk) with tool use. Everything runs inside Windmill — the AI agent can only call the tools you explicitly define, and secrets are managed via [resources](/docs/core_concepts/resources_and_types) (never exposed to the AI). ## Overview The architecture is simple: 1. A **WebSocket trigger** connects to Discord's Gateway (`wss://gateway.discord.gg`) 2. The **heartbeat config** keeps the connection alive by sending periodic heartbeat messages (required by Discord's protocol) 3. A **handler script** processes incoming events — authenticates the bot, and runs a Claude agent with tools 4. The agent decides which tools to call based on the message, and responses are sent back via the Discord REST API ## Prerequisites - A Windmill instance (self-hosted with EE, WebSocket triggers are not available on Cloud Free/Team plans) - A Discord bot token ([create one in the Discord Developer Portal](https://discord.com/developers/applications)) - An Anthropic API key for Claude - The `@anthropic-ai/claude-agent-sdk` package (available on npm) ## Step 1: Create a Discord bot 1. Go to the [Discord Developer Portal](https://discord.com/developers/applications) 2. Click **New Application**, give it a name, and create it 3. Go to the **Bot** tab: - Click **Reset Token** and copy the token — save it, you'll only see it once - Under **Privileged Gateway Intents**, enable **Message Content Intent** 4. Go to the **OAuth2** tab: - Under **OAuth2 URL Generator**, select the `bot` scope - Under **Bot Permissions**, select: **Send Messages**, **Read Message History**, **View Channels** - Copy the generated URL and open it in your browser to invite the bot to your server ## Step 2: Create Windmill resources Create two resources in your Windmill workspace: **Discord bot token** (e.g., `f/bot/discord_token`): ```json { "token": "YOUR_DISCORD_BOT_TOKEN" } ``` **Anthropic API key** (e.g., `f/bot/anthropic`, resource type `anthropic`): ```json { "apiKey": "YOUR_ANTHROPIC_API_KEY" } ``` ## Step 3: Create the handler script Create a new Bun/TypeScript script (e.g., `f/bot/discord_handler`). This script handles all Discord Gateway events and runs a Claude agent with tools when a message is received: ```typescript // Restrict to a specific channel (optional — remove for all channels) const ALLOWED_CHANNEL = "YOUR_CHANNEL_ID"; // Your bot's application ID (same as the user ID for bots) const BOT_USER_ID = "YOUR_BOT_APPLICATION_ID"; // Optional: allow DMs from a specific user (your Discord user ID) const ALLOWED_DM_USER = "YOUR_DISCORD_USER_ID"; // Define tools the AI agent can use via an in-process MCP server const toolServer = createSdkMcpServer({ name: "bot-tools", tools: [ tool( "get_current_time", "Get the current date and time", {}, async () => ({ content: [{ type: "text", text: new Date().toISOString() }], }) ), // Add your own tools here — e.g. query a database, check an API, etc. // The AI agent can only call the tools defined here. // Secrets are fetched inside the tool implementation via wmill.getResource(), // so the AI never sees raw API keys or credentials. ], }); try { event = JSON.parse(msg); } catch { return null; } const op = event.op; // Op 10: Hello — respond with Identify to authenticate if (op === 10) { console.log(`Hello received. heartbeat_interval: ${event.d?.heartbeat_interval}ms`); const token = (await wmill.getResource("f/bot/discord_token")).token; return JSON.stringify({ op: 2, d: { token, intents: 37377, // GUILDS + GUILD_MESSAGES + MESSAGE_CONTENT + DIRECT_MESSAGES properties: { os: "linux", browser: "windmill", device: "windmill" } } }); } // Op 11: Heartbeat ACK — ignore (heartbeat is handled by the trigger) if (op === 11) return null; // Op 0: Dispatch events if (op === 0) { const t = event.t; if (t === "READY") { console.log(`Connected as ${event.d?.user?.username}`); return null; } if (t === "MESSAGE_CREATE") { const d = event.d; // Ignore bot messages if (d.author?.bot) return null; // Handle DMs and channel messages differently const isDM = !d.guild_id; if (isDM) { // Only accept DMs from the allowed user (remove this check to accept all DMs) if (d.author?.id !== ALLOWED_DM_USER) return null; } else { // In channels: only respond in the allowed channel and when @mentioned if (d.channel_id !== ALLOWED_CHANNEL) return null; const mentioned = d.mentions?.some((m: any) => m.id === BOT_USER_ID); if (!mentioned) return null; } const channelId = d.channel_id; const content = d.content .replace(new RegExp(`<@!?${BOT_USER_ID}>`, "g"), "") .trim(); const username = d.author?.username; console.log(`[#${channelId}] ${username}: ${content}`); // Set up Anthropic API key for the agent SDK const anthropicRes = await wmill.getResource("f/bot/anthropic"); process.env.ANTHROPIC_API_KEY = anthropicRes.apiKey; // Run the Claude agent with tools let response = ""; for await (const agentMsg of query({ prompt: `${username} says: ${content}\n\nRespond concisely in Discord style. Use your tools when relevant.`, options: { model: "sonnet", pathToClaudeCodeExecutable: "/usr/bin/claude", permissionMode: "bypassPermissions", allowDangerouslySkipPermissions: true, mcpServers: { tools: toolServer }, }, })) { if (agentMsg.type === "assistant") { const text = agentMsg.message.content .filter((b: any) => b.type === "text") .map((b: any) => b.text) .join(""); if (text) response = text; } } if (!response.trim()) return null; // Send reply via Discord REST API (split long messages) const token = (await wmill.getResource("f/bot/discord_token")).token; const chunks = splitMessage(response.trim(), 2000); for (const chunk of chunks) { const resp = await fetch( `https://discord.com/api/v10/channels/${channelId}/messages`, { method: "POST", headers: { "Content-Type": "application/json", Authorization: `Bot ${token}`, }, body: JSON.stringify({ content: chunk }), } ); if (!resp.ok) { console.log(`Discord send failed: ${resp.status} ${await resp.text()}`); break; } } console.log(`Replied (${response.length} chars): ${response.slice(0, 100)}`); } } return null; } function splitMessage(text: string, maxLen: number): string[] { if (text.length <= maxLen) return [text]; const chunks: string[] = []; let remaining = text; while (remaining.length > 0) { if (remaining.length <= maxLen) { chunks.push(remaining); break; } let splitAt = remaining.lastIndexOf("\n", maxLen); if (splitAt < maxLen / 2) splitAt = maxLen; chunks.push(remaining.slice(0, splitAt)); remaining = remaining.slice(splitAt).trimStart(); } return chunks; } ``` Key points: - **Op 10 (Hello)**: When Discord sends the Hello event, the script returns an Identify payload to authenticate the bot. This is sent back through the WebSocket because "Send runnable result" is enabled. - **Op 11 (Heartbeat ACK)**: Ignored — the heartbeat itself is handled by the trigger's heartbeat configuration, not the script. - **MESSAGE_CREATE**: The script filters for messages in the allowed channel (or DMs from an allowed user), then runs a Claude agent with access to the tools you define. The agent decides which tools to call based on the user's message. - **Tool use**: Tools are defined via `createSdkMcpServer` from the Claude Agent SDK. The AI can only call tools you explicitly register — it has no access to environment variables, the filesystem, or arbitrary network calls. Secrets are fetched inside tool implementations via `wmill.getResource()`, so the AI never sees raw credentials. - **Message splitting**: Discord has a 2000-character limit per message, so long responses are split at newline boundaries. ## Step 4: Create the WebSocket trigger Create a new WebSocket trigger with the following configuration: | Setting | Value | |---------|-------| | **URL** | `wss://gateway.discord.gg/?v=10&encoding=json` | | **Script** | `f/bot/discord_handler` | | **Send runnable result** | Enabled | | **Initial messages** | *(none)* | ### Heartbeat configuration This is the key part. Discord's Gateway requires clients to send a periodic heartbeat message containing the last received sequence number. Without it, Discord closes the connection after ~41 seconds. | Setting | Value | |---------|-------| | **Enable heartbeat** | Yes | | **Interval (seconds)** | `41` | | **Message** | `{"op": 1, "d": {{state}}}` | | **State field** | `s` | ![Heartbeat configuration for Discord](./heartbeat.png 'Heartbeat configuration for Discord Gateway') This tells Windmill to: 1. Extract the `s` (sequence number) field from every incoming Discord message 2. Every 41 seconds, send `{"op": 1, "d": }` through the WebSocket The heartbeat runs at the Rust level with zero job overhead. :::info Why not use initial messages for Identify? Discord requires the client to receive a Hello (op 10) event before sending Identify (op 2). Since initial messages are sent immediately on connection (before any messages are read), sending Identify as an initial message would fail. Instead, the handler script detects the Hello event and returns the Identify payload via the "Send runnable result" feature. ::: ## Step 5: Test it 1. Save and enable the trigger 2. Check the trigger status — it should show no errors and an active server ID 3. Send a message in your Discord channel (@ the bot if you added the mention filter) 4. The bot should respond with a Claude-generated reply You can verify the heartbeat is working by checking completed jobs — you should see handler jobs with empty results at ~41-second intervals (these are the Heartbeat ACK events from Discord). ## Discord Gateway intents The Identify payload uses `intents: 37377`, which is a bitmask combining: | Intent | Value | Purpose | |--------|-------|---------| | `GUILDS` | `1 << 0` = 1 | Receive guild/channel metadata | | `GUILD_MESSAGES` | `1 << 9` = 512 | Receive message events in guild channels | | `DIRECT_MESSAGES` | `1 << 12` = 4096 | Receive DM events | | `MESSAGE_CONTENT` | `1 << 15` = 32768 | Access message content (privileged — must be enabled in the Developer Portal) | **Total: 1 + 512 + 4096 + 32768 = 37377** Without `MESSAGE_CONTENT`, the `content` field in message events will be empty for messages from other users. Without `DIRECT_MESSAGES`, the bot won't receive DM events. ## Adding AI sandbox with volumes You can enhance the bot with persistent context by using [volumes](/docs/core_concepts/volumes). Add a volume annotation to the script: ```typescript // volume: my-bot-volume .claude ``` This mounts a persistent volume at `.claude/` where you can store persona files, conversation history, or learned preferences that persist across executions: ```typescript // Load persona from volume let systemPrompt = "You are a helpful assistant."; if (fs.existsSync(".claude/PERSONA.md")) { systemPrompt = fs.readFileSync(".claude/PERSONA.md", "utf-8"); } ``` ## Next steps - Add conversation history by storing recent messages in a database or volume - Use [filters](/docs/triggers/websocket_triggers#filters) on the WebSocket trigger to only trigger the handler for specific event types - Connect multiple bots by creating additional WebSocket triggers with different tokens --- ## Local development with AI Source: https://www.windmill.dev/docs/misc/guides/local_dev_with_ai # Local development with AI Windmill provides tools to enhance local development with AI coding assistants like [Claude Code](https://docs.anthropic.com/en/docs/claude-code) or [Cursor](https://www.cursor.com): the CLI `init` command for generating AI context files, the [VS Code extension](../../../cli_local_dev/1_vscode-extension/index.mdx)'s YAML linter for flow validation, and the [MCP server](../../../core_concepts/51_mcp/index.mdx) for direct interaction between LLMs and your workspace. ## Prerequisites - [Windmill CLI](../../../advanced/3_cli/index.mdx) installed (`npm install -g windmill-cli`) - A workspace set up locally with `wmill workspace add` and [`wmill sync pull`](../../../advanced/3_cli/sync.mdx) - Optionally, VS Code with the [Windmill extension](../../../cli_local_dev/1_vscode-extension/index.mdx) installed - An AI coding assistant (Claude Code, Cursor, etc.) ## CLI init command Running `wmill init` in a synced workspace folder generates context files for AI assistants: ```bash cd myworkspace wmill init ``` This creates: - `AGENTS.cli.md` - a wmill-managed context file describing the Windmill file structure and conventions - `AGENTS.md` and `CLAUDE.md` - user-owned files that include the managed `AGENTS.cli.md` and hold your own project-specific instructions - Skill files under `.claude/skills/` and `.agents/skills/` - per-task guidance for AI coding agents `AGENTS.cli.md` and the skill files are refreshed on every run (and via `wmill refresh prompts`); `AGENTS.md` and `CLAUDE.md` are created once and never overwritten. `wmill init` also generates [TypeScript editor config and resource-type definitions](../../../advanced/4_local_development/index.mdx#project-files-generated-by-wmill-init). These files describe the Windmill file structure, how [scripts](../../../script_editor/index.mdx) and [flows](../../../flows/1_flow_editor.mdx) are organized, how to create and edit them, and how to manage resources like [triggers](../../../triggers/index.mdx) and [schedules](../../../core_concepts/1_scheduling/index.mdx). AI tools that read these files automatically get this context when working in the workspace. This is the expected file tree: ``` ├── .agents │ └── skills # mirrors .claude/skills ├── .claude │   └── skills │   ├── cli-commands │   ├── raw-app │   ├── resources │   ├── schedules │   ├── triggers │   ├── write-flow │   ├── write-script-bash │   ├── write-script-bigquery │   ├── write-script-bun │   ├── write-script-bunnative │   ├── write-script-csharp │   ├── write-script-deno │   ├── write-script-duckdb │   ├── write-script-go │   ├── write-script-graphql │   ├── write-script-java │   ├── write-script-mssql │   ├── write-script-mysql │   ├── write-script-nativets │   ├── write-script-php │   ├── write-script-postgresql │   ├── write-script-powershell │   ├── write-script-python3 │   ├── write-script-rust │   └── write-script-snowflake ├── AGENTS.cli.md # managed ├── AGENTS.md # user-owned ├── CLAUDE.md # user-owned ├── rt.d.ts # resource type RT namespace ├── tsconfig.wmill.json # managed ├── tsconfig.json # user-owned, extends tsconfig.wmill.json ├── wmill-lock.yaml └── wmill.yaml ``` Some examples of prompts you can use to guide the AI: - "Write a script that fetches data from my Supabase resource" - "Write a mail triage flow using an AI Agent that labels emails as spam, important, or junk, and schedule it to run every day at 9am." - "Add 4 HTTP triggers to create, delete, update and list users in my MongoDB resource" ## Visual preview with `wmill dev` `wmill dev` runs a local live-reload server that renders the Windmill dev page (flow graph, script preview UI, or raw app) for your local files. Edits on disk reload the page; edits in the page round-trip back to `flow.yaml` and inline scripts. The skills generated by `wmill init` include a `preview` skill that drives this from inside Claude Code or Claude Desktop. After scaffolding a flow or app, the agent offers the visual preview as a one-sentence next step — confirming opens the dev page directly: - In **Claude Code** (when the `mcp__Claude_Preview__*` MCP tools are available), the agent adds a `windmill: ` entry to the per-target `.claude/launch.json` and routes the preview through the proxy port. - In **Claude Desktop** or any other terminal-based agent, the agent runs `wmill dev --no-open` itself and hands you the URL. - In **VS Code / Cursor** with the [Windmill extension](../../../cli_local_dev/1_vscode-extension/index.mdx), the same dev page renders inside the extension's iframe. ## VS Code extension YAML linter The Windmill [VS Code extension](../../../cli_local_dev/1_vscode-extension/index.mdx) validates `flow.yaml` files in real time. AI coding assistants that check IDE diagnostics can read these validation errors and auto-correct schema mistakes in flow definitions. This creates a feedback loop: the AI edits a flow YAML file, the linter catches structural errors immediately, and the AI can fix them without manual intervention. The extension also allows you to run your scripts and flows locally from the editor. ## LLM-friendly documentation The Windmill documentation is published in formats designed for AI agents: - [`https://www.windmill.dev/llms.txt`](https://www.windmill.dev/llms.txt): a curated [llms.txt](https://llmstxt.org/) index of the documentation, with a one-line description per page. Point your agent at it to find the right page for any Windmill concept. - Every docs page is available as raw markdown by appending `.md` to its URL, e.g. [`https://www.windmill.dev/docs/core_concepts/scheduling.md`](https://www.windmill.dev/docs/core_concepts/scheduling.md). Agents should prefer these over the HTML pages. - [`https://www.windmill.dev/llms-full.txt`](https://www.windmill.dev/llms-full.txt): the entire documentation as a single file (~2.3 MB). Intended for bulk indexing or RAG pipelines, not for loading directly into an agent's context. To make your agent aware of these, add a line to your project's `AGENTS.md`, for example: "For Windmill concepts, fetch https://www.windmill.dev/llms.txt and follow the relevant `.md` links." ## Context7 plugin If you use the [Context7](https://context7.com) plugin in Claude Code, Codex, Cursor, or any other Context7-aware client, you can point it at the dedicated Windmill CLI docs index so the agent has up-to-date `wmill` command reference available inline: ``` https://context7.com/windmill-labs/windmill-cli-docs ``` This is useful when iterating on shell scripts or CI pipelines that call `wmill sync`, `wmill script run`, `wmill flow run`, etc. The agent can resolve flags and subcommands without you copy-pasting from the docs. A broader Context7 index covering the full Windmill documentation site is also available at [`context7.com/websites/windmill_dev`](https://context7.com/websites/windmill_dev). ## MCP server Windmill's [MCP server](../../../core_concepts/51_mcp/index.mdx) allows AI tools to interact directly with a Windmill workspace. It supports: - Accessing Windmill documentation - Executing [scripts](../../../script_editor/index.mdx) and [flows](../../../flows/1_flow_editor.mdx) - Creating, updating, listing, and deleting resources ([schedules](../../../core_concepts/1_scheduling/index.mdx), [variables](../../../core_concepts/2_variables_and_secrets/index.mdx), etc.) - Note that for writing and deploying scripts, flows, and apps, we highly recommend using the CLI along with the AGENTS.md file, for better guidance and syntax validation. To set up the MCP server with Claude Code: ```bash claude mcp add --transport http windmill https://app.windmill.dev/api/mcp/w//mcp?token= ``` To set up with Cursor, add the following to your MCP configuration: ```json { "mcpServers": { "windmill-mcp": { "url": "https://app.windmill.dev/api/mcp/w//mcp?token=" } } } ``` See the [MCP documentation](../../../core_concepts/51_mcp/index.mdx) for generating your token and full setup instructions. ## Putting it all together A typical AI-assisted local development workflow: 1. Run `wmill init` in your workspace folder to generate AI context files 2. Open the workspace in VS Code with the Windmill extension for running scripts and flows locally, and real-time YAML validation 3. Connect your AI tool to the Windmill MCP server for direct workspace interaction 4. Start building scripts, flows, and apps with your AI assistant 5. Run `wmill dev` (or accept the agent's offer to open the visual preview) to see edits live, and round-trip changes from the dev page back to disk 6. Push the changes back to the Windmill workspace with `wmill sync push` --- ## Tracing & logging with OpenTelemetry Source: https://www.windmill.dev/docs/misc/guides/otel # Windmill tracing & logging with OpenTelemetry [OpenTelemetry (OTEL)](https://opentelemetry.io/) is an open-source observability framework that provides a set of APIs, libraries, agents, and instrumentation to capture and export telemetry data such as traces, metrics, and logs. It is designed to help developers and operators gain insights into the performance and behavior of their applications and infrastructure. ## Core components and vocabulary - **Traces**: Represent the execution path of a request as it traverses through various services. A trace is composed of multiple spans. - **Spans**: The building blocks of a trace, representing a single operation within a trace. Each span contains metadata such as operation name, start and end timestamps, and attributes. - **Metrics**: Quantitative data that measures the performance of a system, such as CPU usage, memory consumption, or request count. - **Logs**: Records of events that occur within a system, providing context and details about operations and errors. - **Instrumentation**: The process of adding code to applications to collect telemetry data. OpenTelemetry provides SDKs and APIs for this purpose. - **Collector**: A component that receives, processes, and exports telemetry data to various backends. It acts as a central hub for telemetry data. While Windmill offers internal service logs and alerts accessible via the Windmill UI, integrating OpenTelemetry can be beneficial for users who wish to centrally aggregate and manage traces and logs. This centralized approach allows for enhanced alerting, monitoring, and analysis capabilities, enabling users to proactively address issues and optimize their systems. ## Tracing with Jaeger [Jaeger](https://www.jaegertracing.io/) Jaeger is an open-source distributed tracing system for monitoring and debugging microservices. Originally developed by Uber, it helps track requests across services, analyze latency, identify bottlenecks, and diagnose failures. Key use cases include debugging production issues, monitoring performance, visualizing service dependencies, and optimizing system reliability. As Jaeger supports the OpenTelemetry protocol, it can be used to collect traces from Windmill. ### Setting up Jaeger If you do not have an existing Jaeger instance, you can start a container running Jaeger with by adding the following to your `docker-compose.yml` file or use this [docker-compose.yml](https://github.com/windmill-labs/windmill/tree/main/examples/deploy/otel-tracing-jaeger) file as a starting point. ```yaml jaeger: image: jaegertracing/jaeger:latest ports: - "16686:16686" expose: - 4317 ``` this will expose Jaegers UI on port 16686 and the OpenTelemetry collector on port 4317. ### Configuring Windmill to use Jaeger In the Windmill UI, go to the "Instances Settings" and "OTEL/Prom" tab and fill in the Jaeger endpoint and service name and toggle the Tracing option to send traces to Jaeger. If you are using the Jaeger container, the endpoint will be `http://jaeger:4317`. ![Jaeger Endpoint](./jaeger_endpoint.png) ### Open the Jaeger UI The Jaeger UI if hosted with the `docker-compose.yaml` file above will be available at `http://localhost:16686`. When running a script or workflow with Windmill, you will be able to see the traces in the Jaeger UI and investigate them. This can be useful to understand the performance of a workflow and identify bottlenecks in the Windmill server or client. ![Jaeger Timeline](./jaeger_timeline.png) ![Jaeger Flamegraph](./jaeger_flamegraph.png) ![Jaeger Trace](./jaeger_trace_graph.png) ### Searching for specific traces To search/filter for a specific trace, for example a workflow, you can use the search function in the Jaeger UI by filtering by tags set by Windmill. The following tags are useful to filter for specific traces: - `job_id`: The ID of the job - `root_job`: The ID of the root job (flow) - `parent_job`: The ID of the parent job (flow) - `flow_step_id`: The ID of the step within the workflow - `script_path`: The path of the script - `script_hash`: Hex hash of the deployed script version (lets you correlate failures with a specific deploy) - `workspace_id`: The name of the workspace - `worker_id`: The ID of the worker - `language`: The language of the script - `tag`: The queue tag of the workflow - `job_kind`: Job type — `script`, `flow`, `appscript`, `aiagent`, `preview`, `flowscript`, etc. - `trigger_kind`: How the job was triggered — `schedule`, `webhook`, `kafka`, `http`, `sqs`, etc. - `trigger`: Trigger identifier, e.g. the schedule path `f/elt/crm_ingestion_schedule` - `created_by`: User or system that started the job (`u/admin`, `schedule`, …) These attributes are recorded on both the `job` and `job_postprocessing` spans, so you can build dashboards in Sentry/Honeycomb/Datadog that break down execution by trigger source or deploy version without joining to the Windmill DB. ![Jaeger Search](./jaeger_search.png) ### OTEL trace context in jobs When OTEL tracing is enabled, Windmill exposes the trace context of each job as environment variables inside the job's runtime, so user scripts can propagate trace context to downstream services: | Env var | Description | |---------|-------------| | `TRACEPARENT` | W3C Trace Context header value: `00-{trace_id}-{span_id}-01` | | `OTEL_TRACE_ID` | Hex-encoded trace ID (derived from the job UUID) | | `OTEL_SPAN_ID` | Hex-encoded span ID (derived from the job UUID) | These variables are set for Python, Bash, Bun, Deno, Go, TypeScript, Rust, C#, Ruby, Nu, Java, and PHP jobs. You can, for example, forward `TRACEPARENT` on outbound HTTP calls so that downstream services create child spans under the Windmill job span. When OTEL tracing is disabled (or on Community Edition), these env vars are not set. ### Connecting jobs to an inbound distributed trace When a caller that is already part of a distributed trace runs a job through a [webhook](../../../core_concepts/4_webhooks/index.mdx) or REST run endpoint with a [W3C `traceparent`](https://www.w3.org/TR/trace-context/) header, Windmill connects the resulting job to that trace. The job span, along with the spans of its flow steps and script subprocess, is relocated under the caller's span, so the whole execution shows up as part of the same end-to-end trace in your tracing backend instead of as a disconnected one. This requires OTEL tracing to be enabled and is an [Enterprise Edition](/pricing) feature. The inbound `traceparent` is also exposed to the job as its [`TRACEPARENT` environment variable](#otel-trace-context-in-jobs) and propagated to every flow step, so scripts that forward `TRACEPARENT` on their own outbound calls keep the trace unbroken across the whole chain. A few details: - It applies to the run endpoints that accept a request body: running a script or flow by path, hash or version, including the synchronous `run_wait_result` variants. [HTTP routes](../../../triggers/2_http_routing/index.mdx) and other native triggers are not picked up as inbound trace sources. - An invalid or malformed `traceparent` header is ignored: the job keeps its own trace derived from the job UUID, so the `job_id` / `root_job` lookup and dashboards built on it keep working. ### Monitoring metrics with Jaeger Jaeger can be used to generate time series for metrics of the collected traces. These time series can be used to compare the performance of individual steps within a workflow and their overall performance and relative contribution over time as well as identify and troubleshoot issues and anomalies. To set this up, Jaeger needs to store the generated metrics in PromQL-compatible storage such as Prometheus. If you do don't have an existing Prometheus instance, you can start one with the following in your `docker-compose.yml` file or uncomment the prometheus service in the [docker-compose.yml](https://github.com/windmill-labs/windmill/tree/main/examples/deploy/otel-tracing-jaeger) file in the Windmill repo. ```yaml prometheus: image: prom/prometheus:latest expose: - 9090 volumes: - ./prometheus-config.yaml:/etc/prometheus/prometheus.yml command: - "--config.file=/etc/prometheus/prometheus.yml" ``` with the following `prometheus-config.yaml` file: ```yaml global: scrape_interval: 15s evaluation_interval: 15s scrape_configs: - job_name: aggregated-trace-metrics static_configs: - targets: ['jaeger:8889'] ``` In order to use prometheus with Jaeger, you need to configure Jaeger to send metrics to Prometheus. For this we will mount a config file to the Jaeger container that will enable the metrics export. In the docker-compose.yml file, add the following to the Jaeger service: ```yaml jaeger: image: jaegertracing/jaeger:latest ports: - "16686:16686" expose: - 4317 - 8889 volumes: - ./jaeger-config.yaml:/etc/jaeger/config.yml command: ["--config", "/etc/jaeger/config.yml"] ``` and the following `jaeger-config.yaml` file: ```yaml service: extensions: [jaeger_storage, jaeger_query] pipelines: traces: receivers: [otlp] processors: [batch] exporters: [jaeger_storage_exporter, spanmetrics] metrics/spanmetrics: receivers: [spanmetrics] exporters: [prometheus] telemetry: resource: service.name: jaeger metrics: level: detailed address: 0.0.0.0:8888 logs: level: DEBUG extensions: jaeger_query: storage: traces: some_storage metrics: some_metrics_storage jaeger_storage: backends: some_storage: memory: max_traces: 100000 metric_backends: some_metrics_storage: prometheus: endpoint: http://prometheus:9090 normalize_calls: true normalize_duration: true connectors: spanmetrics: receivers: otlp: protocols: grpc: endpoint: "0.0.0.0:4317" processors: batch: exporters: jaeger_storage_exporter: trace_storage: some_storage prometheus: endpoint: "0.0.0.0:8889" ``` In the Jaeger UI, you will now be able to see metrics time series for the traces in the "Monitor" tab. ![Jaeger Metrics](./jaeger_metrics.png) ### Exporting Windmill metrics via OTLP In addition to traces and logs, Windmill can export its own operational metrics to any OTLP-compatible collector. Enable the **Metrics** toggle in **Instance settings > OTEL/Prom** — metrics are then exported alongside traces and logs to the same collector endpoint. These metrics complement the [Prometheus `/metrics` endpoint](../../../advanced/18_instance_settings/index.mdx#prometheus) on port 8001: the Prometheus endpoint is scraped by your monitoring system, while the OTEL metrics are pushed to the collector you configured for tracing. Flipping the toggle triggers a delayed worker restart so the OTLP meter provider can be initialized. ### Exported metrics | Metric | Type | Attributes | |---|---|---| | `windmill.queue.push_count` | Counter | — | | `windmill.queue.delete_count` | Counter | — | | `windmill.queue.pull_count` | Counter | — | | `windmill.queue.zombie_restart_count` | Counter | — | | `windmill.queue.zombie_delete_count` | Counter | — | | `windmill.queue.count` | Gauge | `tag` | | `windmill.queue.running_count` | Gauge | `tag` | | `windmill.worker.started` | Counter | — | | `windmill.worker.uptime` | Gauge | `worker` | | `windmill.worker.execution_count` | Counter | `tag` | | `windmill.worker.execution_duration` | Histogram | `tag` | | `windmill.worker.execution_failed` | Counter | `tag` | | `windmill.worker.busy` | Gauge | `worker` | | `windmill.worker.pull_duration` | Histogram | `worker`, `has_job` | | `windmill.db.pool.active` | Gauge | — | | `windmill.db.pool.idle` | Gauge | — | | `windmill.db.pool.max` | Gauge | — | | `windmill.health.db_latency` | Gauge | — | | `windmill.health.db_unresponsive` | Gauge | — | | `windmill.health.status` | Gauge | `phase` | Metrics export is an [Enterprise Edition](/pricing) feature. ### Selecting the exporter protocol The **OTEL/Prom** instance settings tab exposes a **Protocol** dropdown that applies to all three signal types (traces, logs, metrics): - **grpc** (default): uses the tonic gRPC client against the collector's OTLP gRPC endpoint (typically port `4317`). - **http/protobuf**: uses an HTTP client against the collector's OTLP HTTP endpoint (typically port `4318`). This is equivalent to setting `OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf`. Use `http/protobuf` when your network or collector does not support gRPC (for example, some managed observability backends only expose HTTP endpoints). ## Tracing with Tempo and Grafana [Tempo](https://grafana.com/docs/tempo/) is a distributed tracing system that is part of the Grafana stack. It is designed to collect, store, and query traces from distributed systems. Tempo is a scalable and efficient solution for tracing, providing a comprehensive view of the performance and behavior of your applications and infrastructure. ### Setting up an OpenTelemetry collector and Tempo Use the [docker-compose.yml](https://github.com/windmill-labs/windmill/tree/main/examples/deploy/otel-tracing-grafana) file in the Windmill repo as a starting point. Compared to the Jaeger setup, this setup also includes a Loki instance to store the logs as well as a dedicated OpenTelemetry collector that will be used to collect traces from the Windmill instance and distribute them to Tempo and Loki. It you already have a Tempo instance and/or Loki instance, make sure to update the OpenTelemetry collector configuration to point to your existing Loki and Tempo instances. OpenTelemetry collector configuration: ```yaml receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 processors: batch: timeout: 5s exporters: otlphttp/loki: endpoint: http://loki:3100/otlp tls: insecure: true otlp/tempo: endpoint: http://tempo:4317 tls: insecure: true service: pipelines: traces: receivers: [otlp] processors: [batch] exporters: [otlp/tempo] logs: receivers: [otlp] processors: [batch] exporters: [otlphttp/loki] ``` In the Tempo configuration we'll enable the metrics pipeline and set the metrics storage to point to the Prometheus instance. As with Jaeger, this will allow us to query metrics in Grafana. Tempo configuration: ```yaml stream_over_http_enabled: true server: http_listen_port: 3200 log_level: info query_frontend: search: duration_slo: 5s throughput_bytes_slo: 1.073741824e+09 metadata_slo: duration_slo: 5s throughput_bytes_slo: 1.073741824e+09 trace_by_id: duration_slo: 5s distributor: receivers: otlp: protocols: grpc: endpoint: "tempo:4317" ingester: max_block_duration: 5m compactor: compaction: block_retention: 1h metrics_generator: registry: external_labels: source: tempo cluster: windmill storage: path: /var/tempo/generator/wal remote_write: - url: http://prometheus:9090/api/v1/write send_exemplars: true traces_storage: path: /var/tempo/generator/traces storage: trace: backend: local wal: path: /var/tempo/wal local: path: /var/tempo/blocks overrides: defaults: metrics_generator: processors: [service-graphs, span-metrics, local-blocks] generate_native_histograms: both ``` ### Setting up Loki Loki is a log storage system that is part of the Grafana stack. It is designed to store and query logs from distributed systems. Loki is a scalable and efficient solution for logging, providing a comprehensive view of the performance and behavior of your applications and infrastructure. Learn more about Loki and how to configure it [here](https://grafana.com/docs/loki/latest/). For Windmill, we'll use the in-memory storage backend and set the HTTP listen port to 3100 but you can configure it to use a persistent storage backend and change the port and other settings as needed. Loki configuration: ```yaml auth_enabled: false server: http_listen_port: 3100 common: ring: instance_addr: 0.0.0.0 kvstore: store: inmemory replication_factor: 1 path_prefix: /tmp/loki schema_config: configs: - from: 2020-05-15 store: tsdb object_store: filesystem schema: v13 index: prefix: index_ period: 24h storage_config: filesystem: directory: /tmp/loki/chunks limits_config: allow_structured_metadata: true ``` ### Setting up Prometheus Prometheus is a monitoring and observability platform that is part of the Grafana stack. In this context it is designed to store and query metrics generated by Tempo. Learn more about Prometheus and how to configure it [here](https://prometheus.io/docs/introduction/overview/). Prometheus configuration: ``` global: scrape_interval: 15s evaluation_interval: 15s scrape_configs: - job_name: 'prometheus' static_configs: - targets: [ 'localhost:9090' ] - job_name: 'tempo' static_configs: - targets: [ 'tempo:3200' ] ``` ### Setting up Grafana Grafana is a monitoring and observability platform that is part of the Grafana stack. It is designed to visualize and query metrics, traces and logs from distributed systems. Learn more about Grafana and how to configure it [here](https://grafana.com/docs/grafana/latest/). For Windmill, we'll use the Tempo datasource for traces and Loki as a log datasource. Furthermore, metrics can be visualized in Grafana by using the Prometheus datasource and the metrics generated by Tempo. ### Configure Windmill to use Tempo and Loki In the Windmill UI, go to the "Instances Settings" and "OTEL/Prom" tab and fill in the OpenTelemetry collector endpoint `http://otel-collector:4317` and toggle both the Tracing and Logs options. ![OTEL Endpoint](./otel_endpoint.png) ### Open the Grafana UI The Grafana UI if hosted with the `docker-compose.yaml` file above will be available at `http://localhost:3000`. #### Traces As with Jaeger, you can search for specific traces by filtering by tags set by Windmill. ![Grafana Traces](./tempo_traces.png) #### Logs For logs, select the Loki datasource and you will be able to see the logs in the Loki UI. ![Grafana Logs](./loki_logs.png) #### Metrics For metrics, select the Prometheus datasource and you will be able to see the metrics in the Prometheus UI. The metrics generated by Tempo are labeled as: - traces_spanmetrics_calls_total - traces_spanmetrics_latency - traces_spanmetrics_latency_bucket - traces_spanmetrics_latency_count - traces_spanmetrics_latency_sum - traces_spanmetrics_size_total ![Grafana Metrics](./grafana_metrics.png) --- ## Build on APIs with SQL using Sequin Source: https://www.windmill.dev/docs/misc/guides/sequin # Use SQL to build on external APIs using Sequin This guide is provided by [Sequin](https://sequin.io). With Sequin, developers can build on top of third-party services like Salesforce or HubSpot using SQL. Sequin runs a real-time sync process that pulls data from external APIs into your Postgres database. Any time a record changes in the API, that change is synced to your database. Likewise, Sequin intercepts mutations you make to records in your database. It applies them to the API first before committing them to your database. You can use Sequin to build Windmill apps on top of third-party services using Windmill's PostgreSQL [resource](../../../core_concepts/3_resources_and_types/index.mdx). This means all your scripts can be written in SQL, even if they’re reading data from [Salesforce](https://www.salesforce.com/) or writing data to [HubSpot](https://www.salesforce.com/). You can `join` your API data with your internal data. And because your API data is cached in Postgres, you don't need to worry about rate limits or pagination. ## Setup a Sequin sync Before you can use Sequin with Windmill, you'll need to create a Sequin sync to your Postgres database: **Step 1**: After [signing up for Sequin](https://app.sequin.io), you'll connect Sequin to the API you want to sync: ![Sequin Console, showing an established connection between Sequin and Airtable](./sequin_1.png 'Sequin Console, showing an established connection between Sequin and Airtable') **Step 2**: Configure your schema by selecting the tables and columns you want to sync. **Step 3**: Then, Sequin will prompt you to connect to your Postgres database. Alternatively, you can use a free demo Postgres instance that Sequin hosts. **Step 4**: After you click "Create", Sequin will begin syncing your data. Sequin will provide you with the connection instructions for your Postgres database. Keep this tab open, as you'll need it to configure Windmill: ![Sequin Console, showing the connection instructions page which lists the credentials for connecting to the Postgres database](./sequin_2.png 'Sequin Console, showing the connection instructions page which lists the credentials for connecting to the Postgres database') For more details on setting up a Sequin sync, [see this guide](https://docs.sequin.io/getting-started#create-a-sync). ## Create a Windmill resource Windmill provides integrations with many different apps and services with [resources](../../../core_concepts/3_resources_and_types/index.mdx). Each Resource has a Resource Type (PostgreSQL, MySQL, MS SQL, BigQuery, Snowflake) that defines the schema that the resource of this type needs to implement. Sequin uses a [Postgres Proxy](https://docs.sequin.io/writes#configuration) to interface with your Sequin-synced tables. The Proxy lets Sequin capture inserts, updates, and deletes you make in your database and commit them to the API. To add Sequin's Postgres Proxy as a Windmill Resource, you can treat it as a regular Postgres Resource and enter the connection details in the Resource configuration: **Step 1**: Go to your Windmill dashboard and find "Resources" on the left sidebar. Click on the “Add Resource” button **Step 2**: Select PostgreSQL type. **Step 3**: Fill out the form with the information. Give it a name and paste the values for host, database name, database username, and database password from the _Connection instructions_ tab of your Sequin dashboard. Using the Postgres Resource in Windmill Now, Sequin is syncing your API data to Postgres. You've also connected Windmill to Postgres via Sequin's Proxy. To query this data in your Windmill app, go back to your Windmill dashboard and find “Home”: **Step 1**: From the Home page, click **New** and select **Script**. **Step 2**: Name the Script, give it a summary, and select your “PostgreSQL” as language. **Step 3**: In the list of Resources, select the Postgres connection to Sequin. **Step 4**: Compose your query. The schema for your Sequin-synced tables is available if you click “Explore schema” button right under the database name. **Step 5**: Click the "Test" button on the top right to make sure your query runs as expected. Here's an example Airtable query that returns all the product names in the Products Inventory table, which are of the type “Bag”. ```sql -- $1 type = Bag /*Assumes the default value Bag but can be changed from the menu.*/ SELECT product_name from airtable.product_inventory WHERE type = $1::TEXT ``` The type is an argument with a default value of “Bag” but you can always pass a different value from the input field right below the “Test” button. ![Sequin 3](./sequin_3.png.webp) ## Writing back to the API With Sequin, you can also make [mutations](https://docs.sequin.io/writes) via your database as well. Inserts, updates, and deletes you make to Sequin-synced tables are first applied to the API. If they pass validation, they're committed to your database. To write your first mutation query, navigate to "Home" on the left sidebar, click **New** and select **Script**. In the list of Resources, select the Postgres connection to Sequin. You can compose an `insert` query by populating `values` with various input fields in your application. For example, if you have a form with inputs named `first_name`, `last_name`, and `email`, the corresponding `insert` query would look like this: ```sql insert into salesforce.contact (first_name, last_name, email) values ($1::TEXT, $2::TEXT, $3::TEXT); ``` Provide argument values and click on the "Test" button in the top right to execute the insert query. ![Sequin 4](./sequin_4.png.webp) ## Errors When Sequin's Proxy encounters an error trying to apply your mutation in the upstream API, the Proxy returns a standard Postgres error. You can configure your app to display this as an alert notification. Let’s say you make a mutation to a Salesforce Contact with an invalid email. Salesforce will return a validation error. You can configure your Windmill app to display this as a helpful error message. As an example, Windmill has a Submit button with an **onFailure** property. You can display an error message to the user if the script throws an error: ![Sequin 5](./sequin_5.png.webp) By setting "Append Error" to true, Windmill will append the error returned by Salesforce to the toast notification: ![Sequin 6](./sequin_6.png.webp) ## Next steps With Sequin, you can build Windmill apps on top of sources like Salesforce, Airtable, and HubSpot. To learn more, check out our docs on [building SQL scripts](../../../getting_started/0_scripts_quickstart/5_sql_quickstart/index.mdx) and read more about [how Sequin works](https://docs.sequin.io/). --- ## Snowflake app with user roles Source: https://www.windmill.dev/docs/misc/guides/snowflake_app_with_user_roles # Build an app accessing Snowflake with end-user roles :::info Legacy This guide uses the legacy low-code app editor. For new apps, we recommend [full-code apps](../../../full_code_apps/index.mdx) with React or Svelte. ::: This guide walks you through building an application that accesses Snowflake data based on the end-user’s role, using OAuth in Windmill. By leveraging dynamic role-based credentials from Snowflake’s OAuth integration, we avoid static credentials and enable secure data access customized for each user. This can be particularly useful for organizations with strict data access policies and multiple user roles where [row access policies](https://docs.snowflake.com/en/user-guide/security-row-intro) are set up. The tutorial includes steps to set up Snowflake OAuth, configure user roles, and create UI components in Windmill for a seamless, role-specific data experience. Note that the mechanism of using the end-user's role demonstated here with Snowflake can be used for any [OAuth-supported resource in Windmill](../../../advanced/27_setup_oauth/index.mdx#oauth) such as GitHub, Slack, or Google Workspace. --- ## Video tutorial For a visual walkthrough of building this app, watch the tutorial below: - 00:00 [Create a New Snowflake OAuth User Resource](https://www.youtube.com/watch?v=9r17_ABP4Xk&t=0s) - 00:37 [Background Runnable to query Available Tables](https://www.youtube.com/watch?v=9r17_ABP4Xk&t=21s) - 02:04 [Display Table Content](https://www.youtube.com/watch?v=9r17_ABP4Xk&t=83s) - 02:47 [Test the App](https://www.youtube.com/watch?v=9r17_ABP4Xk&t=147s) ## Prerequisites 1. **Set up Snowflake OAuth**: Follow the [Snowflake OAuth guide](../../../advanced/27_setup_oauth/index.mdx#snowflake). 2. **Configure User Roles in Snowflake**: In the [Snowflake console](https://app.snowflake.com/), ensure that user roles connected through OAuth have access to the relevant tables. ## Sample app setup For this example, we created a new Snowflake organization with a `WINDMILL` database, a `PUBLIC` schema, and two user roles: - **hr_user** with the role `PRIVILEGED` - **support_user** with the role `RESTRICTED` The database contains the following tables: - **SALARIES** – accessible only to the `PRIVILEGED` role - **LIMITED_SALARIES** – accessible to both roles The goal is to use OAuth to dynamically retrieve the credentials for the end-user connecting to the app rather than using static credentials. ### Step 1: Create a new Snowflake OAuth user resource 1. After creating a new app in your Windmill workspace, add a **User Resource Input** component. 2. For "Resource Type," enter `snowflake_oauth`. 3. Enable **Express OAuth Setup** by toggling the option. ![Settings for User Resource Input](./user_resource_input.png) 4. In the UI editor, click the plus icon (+) to authenticate with your Snowflake account and test the connection. This component allows the app to use end-user credentials via an interactive OAuth connection rather than relying on static resources defined in the workspace. ### Step 2: Background runnable to query available tables Next, create a **Background Runnable** to retrieve the available tables based on the user’s Snowflake role. 1. Create a new **Background Runnable** of type "Snowflake." ![Settings for Background Runnable](./background_runnable_1.png) 2. Enter a Snowflake query to list available tables: ```sql SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = 'PUBLIC'; ``` 3. Connect the **Background Runnable** to the **User Resource Input** component from Step 1 by clicking the connect icon and selecting the result field of the **User Resource Input**. 4. Enable the toggle **resource from users allowed** to grant access to user-linked resources. Note that this resource is passed as a reference and won’t be accessible to the app publisher. ![Settings for Background Runnable](./background_runnable_2.png) 5. Click the **Run** button to test the query and view the results. 6. Create a **Select** component and connect it to the **Background Runnable** output to populate the dropdown menu. Map the output to `label` and `value` fields as follows: ```js bg_0.result.map(_ => ({ value: _.TABLE_NAME, label: _.TABLE_NAME })) ``` ### Step 3: Display table content Now, add a **Rich result** component to show the table content based on the selected table. 1. Create a **Rich result** component. 2. In the "Data Source" setting, Select "Create Inline Script" and select "Snowflake". In the code editor enter your snowflake query such as: ```sql -- ? table_name (varchar) = default arg select * from TABLE(?) ``` 2. Connect the **Rich result**’s "database" field to the **User Resource Input** component from Step 1. 3. Connect the **Rich result**’s "table_name" field to the **Select** component created in Step 2. 4. Enable the toggle **resource from users allowed** to grant access to user-linked resources. Note that this resource is passed as a reference and won’t be accessible to the app publisher. ![Settings for Rich result](./rich_result.png) 4. The component will automatically populate with data from the selected table. ### Step 4: Test the app Now, we’ll see how the displayed data changes based on the logged-in user’s role. 1. Click the **Preview** button to switch to the end-user preview mode. 2. Use the plus icon (+) to log in as a privileged user (e.g., **hr_user**) and view the content of the `SALARIES` table. ![View as HR user](./hr_view.png) 3. Log out by clicking the logout button next to the plus icon, then log in as the restricted user (e.g., **support_user**). You should now see only the `LIMITED_SALARIES` table. ![View as Support user](./support_view.png) --- ## Table Source: https://www.windmill.dev/docs/misc/guides/table # Table :::info Legacy This guide uses the legacy low-code app editor. For new apps, we recommend [full-code apps](../../../full_code_apps/index.mdx) with React or Svelte. ::: This is an introduction on how to use the Table component in Windmill. ![Table API](../../../assets/apps/4_app_component_library/table.png.webp) ## AgGrid vs Table component vs Database studio In Windmill there are 3 table components: one simply called Table, [AgGrid](../aggrid_table/index.md) and [Database studio](../../../apps/4_app_configuration_settings/database_studio.mdx). The Table component covers most use cases. In its simplest form, it takes an array of objects as input and uses the keys of the objects as the headers of the table. See the bottom of this document for the current limitations. The [AgGrid](../aggrid_table/index.md) component provides many advanced features. [Database studio](../../../apps/4_app_configuration_settings/database_studio.mdx) is a web-based database management tool. It allows you to display and edit the content of a database. ### Examples :::info [Table Showcase](https://hub.windmill.dev/apps/19/table-component-showcase) - See the Hub for an app that showcases all Table options as working code. You can copy/paste from it directly, giving you both working code to look at and a fast starting point. ::: ## Table data (data source) As with almost all fields in the app, the data can be `static`, `connected` or `eval`. **static** - a static JSON you define. **connected** - connect to the result of another script or component, or to the state of the app. **eval** - run an inline eval that can refer to the state or to scripts, like connected. Eval lets you type instead of clicking. You can also choose to refer to a script from your workspace or create an inline script. :::tip You cannot edit the result of a script, so if you want to later change the data based on another component, the recommendation is to store that data as a state. ::: ## Referring to data from the row to create a new row in the table Sometimes you want to refer to the table data within the table itself, e.g. if you want to have a select that is specific to each row using another field. This can be achieved with the `row.value` and `row.index` properties. You will find examples of using `row.value` and `row.index` in the showcase app above. ### Initial state :::info The initial state is from https://tanstack.com/table/v8/docs/api/core/table#initialstate but not all states work, currently `columnVisibility`, `columnOrder`, `columnPinning` and `columnSizing` are implemented. Pagination and search/filter are supported in the GUI, but not through this config. Grouping, row selection, sorting and expand are currently not supported. ::: #### Hide columns By default the Table component shows all columns. You can hide columns with the following syntax. In the `Initial State` field in the table config add: ```tsx { "columnVisibility": { "id": false } } ``` Here we hide the `id` column. ### Reorder columns By default the Table component shows columns in the order they are added to the array. You can rearrange the order of the columns with the following config. In the `Initial State` field in the table config add: ```tsx { "columnOrder": [ "name", "age", "id" ] } ``` Here we rearrange the order so we have the "name" column first and then "age", "id". ### columnPinning In the `Initial State` field in the table config add: ```tsx { "columnPinning": { "left": [ "name" ], "right": [ "id" ] } } ``` ### columnSizing In the `Initial State` field in the table config add: ```tsx { "columnSizing": { "id": 10, "name": 750, "age": 150 } } ``` Leave out the fields you do not need: you only specify the desired behavior, and it is still TanStack logic that does the final calculations. ## Search **by component** - a nice feature of the Table component is that it can do the search for you based on the data in the component **by runnable** - if you want programmatic control over the search. You need to use a script as the data source and connect the table's "search" key to an input of the script. Please see the [Examples](#examples) for working code. ### Limits - Button and Select/Dropdown are always in the last column, called actions ## Not supported features - Resizable by the user - Grouping - Sorting (can be done by a transformer, but not by the user) :::info Transformer If you want to do basic sorting, or edit the column header name from the script you can use a Transformer script. See the [documentation](../../../apps/3_app-runnable-panel.mdx#transformer) for more information. ::: If some of these features are important, we recommend using the [AgGrid component](../aggrid_table/index.md) --- ## Share on hub Source: https://www.windmill.dev/docs/misc/share_on_hub # Share on Windmill Hub [Windmill Hub][wm-hub] is the community website of Windmill where you can find and share your Scripts, Flows, Apps and Resource types with every Windmill user. The best submissions get approved by the Windmill Team and get integrated directly in the app for everyone to reuse easily. Scripts can be written in TypeScript, Go, Python and Bash, however, similarly to the Windmill app, TypeScript is the recommended language. With the Hub, we aim to create a trusted support for users to save time and find inspirations to solve problems they didn't even know Windmill could crack! It is therefore also made for less technical users to get familiar with Windmill and find ways to improve their daily work. The Hub is complementary to our [Discord][wm-discord] where community members give mutual support & kudos. Below you will find guides on how to contribute to the Hub, thank you for being part of the community! ## Scripts Currently [Windmill Hub][wm-hub] supports TypeScript (Deno and Bun), Python 3, Go or Bash scripts. You can add Common, Error handler, Approval and Trigger scripts by going to the New Script page. The Summary will be the title of the Script, Integration should have the name of the app it integrates with (if there is one), and Description should be a short description of the script - it supports Markdown. Then you can do your magic and write your script for the community: ![Add new script](./add_new_script.png.webp "Add new script on Hub") Once approved by the Windmill Team, the Script will be available for everyone to use directly on Windmill cloud or [Self-Hosted](../../advanced/1_self_host/index.mdx) instances synced with Hub. ![Pick a hub script](./pick_a_hub_script.png.webp) ## Flows Using the [OpenFlow](../../openflow/index.mdx) portable format, one can simply copy the JSON from the Flow editor and paste it on the [New Flow](https://hub.windmill.dev/flows/add) page to upload it to the Hub. Then you can do your magic and share your flow for the community. ![Copy OpenFlow JSON](./export_flow.png.webp) ![New Flow page](./new_flow.png.webp) Once a Flow is approved by the Windmill Team, it will be directly integrated into every workspace of every instance of Windmill. ![Approved Flows on Windmill](./approved_flows.png.webp) ## Apps Using the [Hub Compatible JSON](../../apps/0_toolbar.mdx#hub-compatible-json) of an app, just paste the JSON of your app to [Windmill Hub](https://hub.windmill.dev/). ![Export App JSON to Hub](../../assets/apps/1_app_toolbar/export_hub.png.webp "Export App JSON to Hub") ![Submit App to Hub](../../assets/apps/1_app_toolbar/submit_app.png.webp "Submit App to Hub") Once an App is approved by the Windmill Team, it will be directly integrated into every workspace of every instance of Windmill. ## Resource types [Resource types](../../core_concepts/3_resources_and_types/index.mdx) are simply [JSON Schemas](../../core_concepts/13_json_schema_and_parsing/index.mdx) which create a Type to Resources by constraining the properties or fields that the Resource can have. In addition, they serve two main purposes on Windmill: - Filter Resources by Resource types for the generated UI. - Allow to have a way to manually create Resources of the specified Resource Type using the autogenerated UI from their [JSON Schema](../../core_concepts/13_json_schema_and_parsing/index.mdx). ![Add a PG resource](./add_resource_pg.png.webp "Add a PG resource") To add a Resource Type to the Windmill Hub, go to the [New Resource Type](https://hub.windmill.dev/resource_types/add) page. You can then add your arguments one by one or use the monaco editor to edit it as a JSON directly. Adding a Resource Type to the [Hub][wm-hub] will be available for every Windmill user, once it is approved by the Windmill Team. If it gets approved, the windmill-gh-action-deploy will deploy it in the starter workspace of Windmill cloud. Being deployed on the starter workspace means that it will be available to all workspaces. --- Thank you for your interest in sharing your work on the Hub. Community is essential to our work to build a tool that is useful, powerful and fun to use. [wm-hub]: https://hub.windmill.dev [wm-discord]: https://discord.com/invite/V7PM2YHsPB --- ## White labelling Source: https://www.windmill.dev/docs/misc/white_labelling # White labeling Windmill Windmill provides a library to embed the entire Windmill app or specific components - such as the [Flow editor](../../flows/1_flow_editor.mdx) or the [low-code app editor](../../apps/0_app_editor/index.mdx) (legacy) - with a simplified UI into your own application or website. This enables you to provide Windmill's services to your clients while maintaining your brand's identity. Windmill offers an SDK compatible with any framework, simplifying its integration across various platforms. It can be built in collaboration with us using React/Svelte and our full SDK. In particular, for React, the [React SDK](#react-sdk) below contains all components from the Windmill frontend. The App Viewer and Flow Builder are already available, and we maintain a [webpack example repository](https://github.com/windmill-labs/windmill-whitelabelling-react-webpack) showing how to embed them. Check our [demo](https://windmill-sdk-example.com/) of using the Windmill SDK backed by app.windmill.dev to white label Windmill's Flow Builder and App Viewer in a React app using the default create-react-app template. Also, [Private Hub](../../core_concepts/32_private_hub/index.mdx) is available for white labeling. It allows you to have your own platform and approval process for scripts, flows, apps and resource types suggested within the app. White labeling requires a special license and the package @windmill-labs/windmill-react-sdk is not public. Please contact us at sales@windmill.dev, on [Discord](https://discord.com/invite/V7PM2YHsPB), or schedule a [meeting](https://www.windmill.dev/book-demo) with the founder to get started. Example of Windmill's [flow editor](../../flows/1_flow_editor.mdx) being white labeled by [Premote](https://www.premote.nl/): ![Flow editor Premote](./premote_windmill.png.webp 'Flow editor Premote') ## React SDK The Windmill React SDK provides a suite of tools and components to integrate Windmill applications (scripts editor, flows editor, app editor and its deployed apps) into React-based projects. If you're looking to build a standalone React app connected to Windmill backend runnables, see [full-code apps](../../full_code_apps/index.mdx) instead. ### Installation Add the following to your project: ```js 'windmill-react-sdk': 'file:windmill-react-sdk-X.XXX.X.tgz' ``` :::tip Downloading the SDK The SDK is not available on NPM. The SDK will be provided as a `.tgz` file. ::: ### Configuration As Windmill is built with Svelte, you will need to add the Svelte compiler to your project. #### Using Vite Add the following to your `vite.config.js`: ```js ``` An example is provided directly in the `windmill-react-sdk` repository. #### Using webpack 5 (Next.js) You need to install `svelte-loader` and add the following to your `next.config.js`: ```js const nextConfig = { webpack: (config) => { config.module.rules.push({ test: /\.(svelte)$/, use: [ { loader: 'svelte-loader', options: { emitCss: true, hotReload: true } } ] }); return config; } }; module.exports = nextConfig; ``` ### Usage #### Authentication ```js UserService.login({ requestBody: { email: YOUR_EMAIL, password: YOUR_PASSWORD } }) .then(() => { // Handle successful login }) .catch((error) => { // Handle login errors }); ``` Replace YOUR_EMAIL and YOUR_PASSWORD with the corresponding values. #### App preview ```jsx function MyApp() { return ; } ``` Replace YOUR_WORKSPACE and YOUR_APP_PATH with the corresponding values. --- ## Why windmill Source: https://www.windmill.dev/docs/misc/why_windmill # Why Windmill All code is not made equal and can be split in 2 categories: - **Code that matters**: high-value code containing your business logic, data transformation, internal API calls, and all the logic of your internal long-running services and workflows. This is the crux of the value-added of your engineering. Usually that code is prototyped and started under the form of scripts and SQL files, until it is turned at great expense into micro-services and hard to maintain custom internal tools. - **Boilerplate**: all the rest is boilerplate. Be it UI and frontends that allow you to call the code above, API calls to external services, error handling, retries, logic to make your code scalable, dependency management, CI/CD, managing secrets, schedules, permissions, authentication, etc. That code is boilerplate because it _feels_ like you shouldn't have to reinvent the wheel, over and over again. Many services label themselves as no-code or low-code: they do address the challenge of getting rid of the boilerplate and are accessible to all members of a diverse organization, not solely engineers. However, we believe they lack the full power and flexibility of code, as they either hide it completely or only allow it under restricted forms. Windmill is different: - Windmill is an [open-source](https://github.com/windmill-labs/windmill) developer platform and infra to turn scripts (TypeScript, Python, Go, PHP, Bash, C#, Java, SQL and Rust, among [others](../../getting_started/0_scripts_quickstart/index.mdx)) into endpoints, workflows and UIs. In that respect, Windmill is an alternative to Retool, Prefect, Temporal and n8n. - It empowers semi-technical users to access and edit that code without being overwhelmed by the usual barriers to entry (git, IDE, local environments, secrets management, etc). - It meets the standards of senior/staff software engineers for production-grade infrastructure, while staying flexible and customizable with code. Concretely, Windmill combines: - A **fast, scalable runtime** for [scripts](../../script_editor/index.mdx) with a self-managed [job queue](../../core_concepts/20_jobs/index.mdx), [dependency management](../../advanced/6_imports/index.mdx) inferred from the code itself, and [auto-generated UIs](../../core_concepts/6_auto_generated_uis/index.mdx) derived from your script parameters. - A **workflow engine** with a low-code builder: build and run complex [flows](../../flows/1_flow_editor.mdx) with [retries](../../flows/14_retries.md), [error handling](../../flows/8_error_handling.mdx), [for loops](../../flows/12_flow_loops.md), [branches](../../flows/13_flow_branches.md), [approval steps](../../flows/11_flow_approval.mdx) and [suspended executions](../../flows/15_sleep.md) that consume no resources while waiting. - **App builders**: a [low-code UI builder](../../getting_started/7_apps_quickstart/index.mdx) for internal apps, admin panels and dashboards, and a [full-code app builder](../../full_code_apps/index.mdx) for custom React or Svelte frontends connected to Windmill backend runnables. - **[Triggers](../../triggers/index.mdx)** for every script and flow: [webhooks](../../core_concepts/4_webhooks/index.mdx), [schedules](../../core_concepts/1_scheduling/index.mdx), HTTP routes, queues and more, plus an open API to embed Windmill into your existing infrastructure. - An **enterprise-grade platform**: [permissions and RBAC](../../core_concepts/16_roles_and_permissions/index.mdx), [secrets](../../core_concepts/2_variables_and_secrets/index.mdx), [OAuth and SSO](../../advanced/27_setup_oauth/index.mdx), [audit logs](../../core_concepts/14_audit_logs/index.mdx), and a [CLI](../../advanced/3_cli/index.mdx) with [Git sync](../../advanced/11_git_sync/index.mdx) for [local development](../../advanced/4_local_development/index.mdx) and version control. The central tenet is: make building automation fast and easy, and everybody will automate repetitive tasks and save a lot of time. Scripts become widely useful tools, with an agreed-upon way to run them, UIs that cost nothing to build, and production-grade infrastructure that you don't have to maintain. Windmill is fully open source and can be [self-hosted](../../advanced/1_self_host/index.mdx) with a simple `docker compose up`, or used through the Cloud App. You can find examples and inspiration on [Windmill Hub](https://hub.windmill.dev) or on our [Blog](/blog), and a detailed view of how Windmill compares to other tools on the [Windmill compared to competitors](../../compared_to/peers.mdx) page. --- ## Windows workers Source: https://www.windmill.dev/docs/misc/windows_workers # Windows workers Windows workers enable you to run Windmill scripts and flows directly on Windows machines without requiring Docker or WSL, supporting Python, Bun, PowerShell, C#, and Nu executors for native Windows execution. Windows Native Workers are a [Self-Hosted Enterprise](/pricing) feature. In terms of [billing](/pricing), they count as 1 Compute Unit, unless you are using them for compute on a large machine, in which case they count as 2 Compute Units. You can connect Windows workers to your existing Dockerized or cloud self-hosted PostgreSQL database and Windmill server. ### Setting up Windmill worker executable 1. **Set up a working directory**: - Create a directory from where you want to run the Windmill worker, e.g., `C:\Users\Alex\windmill`. 2. **Download Windmill executable**: - Download the `windmill-ee.exe` file into the newly created directory from the [releases page](https://github.com/windmill-labs/windmill/releases). 3. **Set basic environment variables**: - Set the following [environment variables](../../core_concepts/47_environment_variables/index.mdx) (replace the placeholders with your specific values): ```powershell # Replace these variables with your specific configuration $env:MODE="worker" $env:DATABASE_URL="postgres://postgres:changeme@172.12.0.1:5432/windmill?sslmode=disable" $env:SKIP_MIGRATION="true" ``` More environment variables and worker settings can be found [here](https://github.com/windmill-labs/windmill?tab=readme-ov-file#environment-variables). 4. **Run windmill-ee.exe**: ```powershell PS C:\Users\Alex\windmill> .\windmill-ee.exe ``` We recommend running Windmill as a service on your Windows environment using `sc` or `NSSM` to monitor the Windmill worker, start it at system boot, and manage the restart policy. After the basic setup, follow these steps for each language your worker should support. ### Python executor 1. **Install uv**: ```powershell powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex" ``` You can also check other installation methods in uv`s [official documentation](https://docs.astral.sh/uv/getting-started/installation/). ### Bun executor 1. **Install Bun**: Follow the official [documentation](https://bun.sh/docs/installation) for Windows. 2. **Locate Bun installation**: - Find where Bun is installed by running: ```powershell where.exe bun ``` Example output: ```plaintext C:\Users\Alex\.bun\bin\bun.exe ``` 3. **Set environment variables**: - Add the following environment variables (replace the placeholders with your specific values): ```powershell # Replace these variables with your specific configuration $env:BUN_PATH="C:\Users\Alex\.bun\bin\bun.exe" ``` ### PowerShell executor 1. **Install PowerShell 7+ (stable)**: Ensure you have the latest stable release of PowerShell by following the [official documentation](https://learn.microsoft.com/en-us/powershell/scripting/install/installing-powershell-on-windows). - Start PowerShell 7 and verify you're running PowerShell 7 by checking `$PSVersionTable`: ```powershell $PSVersionTable PSVersion 7.4.5 ``` 2. **Locate PowerShell 7 installation**: - Find where PowerShell 7 is installed by running: ```powershell where.exe pwsh.exe # Note: previous versions used powershell.exe ``` Example output: ```plaintext C:\Program Files\PowerShell\7\pwsh.exe ``` 3. **Set environment variables**: - Add the following environment variables (replace the placeholders with your specific values): ```powershell # Replace these variables with your specific configuration $env:POWERSHELL_PATH="C:\Program Files\PowerShell\7\pwsh.exe" ``` ### C# executor 1. **Install .NET 9.0 SDK**: Follow [Microsoft instructions](https://learn.microsoft.com/en-us/dotnet/core/install/windows) and make sure you have .NET 9.0 installed. - You can check it by listing the installed SDKs: ```powershell dotnet --list-sdks ``` 2. **Locate your .NET installation**: - Find where .NET is installed by running: ```powershell where.exe dotnet ``` 3. **Set environment variables**: - Add the following environment variables (replace with your values if needed): ```powershell # Replace these variables with your specific configuration $env:DOTNET_ROOT="C:\Program Files\dotnet" $env:DOTNET_PATH="C:\Program Files\dotnet\dotnet.exe" ``` ### Nu executor 1. **Nu**: Ensure you have installed Nu by following the [official documentation](https://www.nushell.sh/book/installation.html#package-managers). - Start PowerShell and verify you can enter Nushell: ```powershell nu ``` 2. **Locate Nu installation**: - Find where Nu is installed by running: ```powershell where.exe nu.exe ``` 3. **Set environment variables**: - Add the following environment variables (replace the placeholders with your specific values): ```powershell # Replace these variables with your specific configuration $env:NU_PATH="C:\..\..\nu.exe" ``` ### Java executor 1. **Install Java**: Ensure you have installed Java. You can use any Java version, but OpenJDK-22 is recommended and tested by Windmill. - Start PowerShell and verify you have working `java` and `javac`: ```powershell java --version && javac --version ``` 2. **Install Coursier**: - Open PowerShell and fetch .jar: ```powershell Start-BitsTransfer -Source https://github.com/coursier/launchers/raw/master/coursier -Destination coursier ``` 3. **Set environment variables**: - Add the following environment variables (replace the placeholders with your specific values): ```powershell # Replace these variables with your specific configuration. # Make sure you provide **full** path! $env:COURSIER_PATH="C:\..\..\coursier" ``` ### Ruby executor 1. **Install Ruby**: For Ruby you will need to have `ruby.exe`, `bundler.bat` and `gem.cmd` in `PATH`. You can use [RubyInstaller](https://rubyinstaller.org/) for this. - Start PowerShell and verify you have working executables: ```powershell ruby --version && bundler --version && gem --version ``` ### R executor 1. **Install R**: Download and install R from the [official CRAN website](https://cran.r-project.org/bin/windows/base/). During installation, ensure R is added to the system PATH. - Start PowerShell and verify you have a working R installation: ```powershell Rscript --version ``` 2. **Install Pak and Renv**: - Run: ```powershell Rscript -e "install.packages(c('pak', 'renv'), lib=Sys.getenv('R_LIBS_USER'), repos='https://cloud.r-project.org')" ``` ## Running as a Windows service For production environments, it's strongly recommended to run `windmill-ee.exe` as a Windows service. This ensures the Windmill process starts automatically at system boot, restarts on failure, and runs reliably in the background. Windmill supports three different modes of operation: - **Worker mode** (`MODE=worker`): Executes jobs from the queue - **Server mode** (`MODE=server`): Runs the API server and web interface - **Agent mode** (`MODE=agent`): Connects to a remote Windmill server to execute jobs ### Understanding Windmill modes #### Worker mode Worker mode is used to execute jobs from the Windmill queue. Workers connect directly to the PostgreSQL database and pull jobs to execute. **Required environment variables:** - `MODE=worker` - `DATABASE_URL`: PostgreSQL connection string #### Server mode Server mode runs the Windmill API server and web interface. This mode handles all HTTP requests, serves the UI, and manages the job queue. **Required environment variables:** - `MODE=server` - `DATABASE_URL`: PostgreSQL connection string #### Agent mode Agent mode allows workers to connect to a remote Windmill server via HTTP instead of directly to the database. This is useful for running workers in isolated networks or when you want centralized control over worker authentication. See the [Agent workers documentation](../../core_concepts/28_agent_workers/index.mdx) for more details. **Required environment variables:** - `MODE=agent` - `BASE_INTERNAL_URL`: URL of the Windmill server (e.g., `http://your-windmill-server:8000`) - `AGENT_TOKEN`: JWT token for authenticating with the server (obtain this from your Windmill server's worker management page) The agent token encodes the worker group and optionally worker tags, allowing fine-grained control over which jobs the agent can execute. ### Setting up the Windows service Windows natively supports running executables as services. The `windmill-ee.exe` binary automatically detects when it's running as a Windows service and adjusts its behavior accordingly. #### Step 1: Create the service Use the Windows `sc` command to create a service. Run PowerShell as Administrator: **For Worker mode:** ```powershell sc.exe create WindmillWorker ` binPath= "C:\Users\Alex\windmill\windmill-ee.exe" ` start= auto ` DisplayName= "Windmill Worker" # Set environment variables for the service $regPath = "HKLM:\SYSTEM\CurrentControlSet\Services\WindmillWorker" $envVars = @( "MODE=worker", "DATABASE_URL=postgres://postgres:changeme@172.12.0.1:5432/windmill?sslmode=disable", ) Set-ItemProperty -Path $regPath -Name "Environment" -Value $envVars -Type MultiString ``` **For Server mode:** ```powershell sc.exe create WindmillServer ` binPath= "C:\Users\Alex\windmill\windmill-ee.exe" ` start= auto ` DisplayName= "Windmill Server" # Set environment variables for the service $regPath = "HKLM:\SYSTEM\CurrentControlSet\Services\WindmillServer" $envVars = @( "MODE=server", "DATABASE_URL=postgres://postgres:changeme@172.12.0.1:5432/windmill?sslmode=disable" ) Set-ItemProperty -Path $regPath -Name "Environment" -Value $envVars -Type MultiString ``` **For Agent mode:** ```powershell sc.exe create WindmillAgent ` binPath= "C:\Users\Alex\windmill\windmill-ee.exe" ` start= auto ` DisplayName= "Windmill Agent" # Set environment variables for the service $regPath = "HKLM:\SYSTEM\CurrentControlSet\Services\WindmillAgent" $envVars = @( "MODE=agent", "BASE_INTERNAL_URL=http://your-windmill-server:8000", "AGENT_TOKEN=jwt_agent_your_token_here" ) Set-ItemProperty -Path $regPath -Name "Environment" -Value $envVars -Type MultiString ``` :::tip Add any additional environment variables (like `WORKER_GROUP`, executor paths, etc.) to the `$envVars` array before setting them. ::: #### Step 2: Configure service recovery Configure the service to restart automatically on failure: ```powershell # Replace WindmillWorker with WindmillServer or WindmillAgent as appropriate sc.exe failure WindmillWorker reset= 86400 actions= restart/60000/restart/60000/restart/60000 ``` This configures the service to restart after 60 seconds on each of the first three failures. #### Step 3: Start the service ```powershell # Replace WindmillWorker with WindmillServer or WindmillAgent as appropriate sc.exe start WindmillWorker ``` #### Step 4: Verify the service is running ```powershell # Check service status sc.exe query WindmillWorker # View service logs in Event Viewer # Navigate to: Event Viewer > Windows Logs > Application # Look for events from source "WindmillWorker" ``` ### Managing the service **Stop the service:** ```powershell sc.exe stop WindmillWorker ``` **Restart the service:** ```powershell sc.exe stop WindmillWorker sc.exe start WindmillWorker ``` **Update environment variables:** ```powershell # Stop the service first sc.exe stop WindmillWorker # Update environment variables $regPath = "HKLM:\SYSTEM\CurrentControlSet\Services\WindmillWorker" $envVars = @( "MODE=worker", "DATABASE_URL=postgres://postgres:changeme@172.12.0.1:5432/windmill?sslmode=disable", "SKIP_MIGRATION=true", "WORKER_GROUP=my-group" # Add or modify variables as needed ) Set-ItemProperty -Path $regPath -Name "Environment" -Value $envVars -Type MultiString # Start the service sc.exe start WindmillWorker ``` **Delete the service:** ```powershell # Stop the service first sc.exe stop WindmillWorker # Delete the service sc.exe delete WindmillWorker ``` ### Alternative: Using NSSM If you prefer a GUI-based approach or need more advanced service management features, you can use [NSSM (Non-Sucking Service Manager)](https://nssm.cc/): 1. **Download and install NSSM** 2. **Run NSSM GUI:** ```powershell nssm install WindmillWorker ``` 3. **Configure in the GUI:** - **Application tab:** Set path to `windmill-ee.exe` - **Details tab:** Set display name and description - **Environment tab:** Add your environment variables (one per line, format: `KEY=VALUE`) - **I/O tab:** Optionally configure log file paths - **Exit actions tab:** Configure restart behavior 4. **Start the service:** ```powershell nssm start WindmillWorker ``` ### Troubleshooting **Service fails to start:** - Check Event Viewer (Windows Logs > Application) for error messages - Verify all environment variables are set correctly - Ensure the `windmill-ee.exe` path in `binPath` is correct and accessible - For agent mode, verify `BASE_INTERNAL_URL` is reachable and `AGENT_TOKEN` is valid **Checking service status and logs:** - Check service status: `sc.exe query WindmillWorker` or open Services GUI with `services.msc` - **Windmill writes logs to**: `C:\tmp\windmill\logs\` by default - Check this directory for log files generated by the Windmill service - Make sure this directory exists and the service has write permissions to it - For **stdout/stderr console output** (startup errors, etc.), the output is not captured by default when using `sc.exe`. To capture console output: - Use NSSM instead of `sc.exe` and configure log file paths in the I/O tab (recommended - see [Alternative: Using NSSM](#alternative-using-nssm) section) **Getting agent tokens:** - Agent tokens must be generated from your Windmill server. See the [Agent workers documentation](../../core_concepts/28_agent_workers/index.mdx) for detailed instructions - Navigate to the worker management section in your Windmill instance - Create a new agent token with the appropriate worker group and tags - The token format is `jwt_agent__` --- ## OpenFlow Source: https://www.windmill.dev/docs/openflow # OpenFlow Spec OpenFlow is an open standard for defining "Flows". Flows are directed graphs - [directed acyclic graphs](https://en.wikipedia.org/wiki/Directed_acyclic_graph) to be exact - in which every node represents a step of computation. In other words, it is a declarative model for chaining scripts. Windmill is the open-source reference implementation for it, providing a UI to build Flows and highly scalable executors. However, everyone is welcome to build upon it and to develop new UIs that target OpenFlow, or create new executors. Flows can be shared and showcased on [Windmill Hub](https://hub.windmill.dev/flows). To see an example of an OpenFlow in practice, go to the Hub and pick a Flow (e.g [Upon new user sign up, check for existence in postgres, hash password, add record to postgres and Airtable, send an email to new user](https://hub.windmill.dev/flows/23/)), then select the JSON tab to see its specification. ## OpenFlow We provide an OpenAPI/Swagger definition file for the spec, it is hosted within the GitHub repository [here.](https://github.com/windmill-labs/windmill/blob/main/openflow.openapi.yaml) It is the source of truth; the TypeScript equivalent below is a simplified version of it for ease of readability and omits some optional fields. OpenFlow is portable and its root object is defined as follows: ```typescript type OpenFlow = { // a one-liner summary-line summary: string; // optional description description?: string; // the actual logic of the flow value: FlowValue; // the input spec of the flow as defined by a json schema schema?: any; }; ``` It contains a short line summary, a description, a schema which is the JSON Schema that constraints the JSON it takes as an input and the FlowValue type is where the logic of the Flow is actually defined. ### FlowValue ```typescript type FlowValue = { // a sequence of modules, some of which are containers // for other modules, like a for-loop or a branch modules: FlowModule[]; // the error handler to call in case of an unrecoverable error failure_module?: FlowModule; // a special module that runs before the first step when the flow // is invoked from an external trigger preprocessor_module?: FlowModule; // force this flow to be executed entirely on the same worker // and share a mounted folder to pass heavy data same_worker?: boolean; // the spec also defines optional flow-level settings not detailed // here: concurrency limits, caching, early stop and early return // expressions, debouncing, flow env variables, chat input, notes // and groups (see the OpenAPI definition) }; ``` A Flow is just a sequence of modules, an optional failure module that will be triggered to handle a failure at any point of the Flow (think `try/catch` in terms of programming languages) and an optional [preprocessor](../core_concepts/43_preprocessors/index.mdx) module. See an example of modules represented in a graph below - this visualization is built-in on the Windmill Flow editor. ![Flow Modules](./modules_graph.png 'Flow Modules') ### FlowModule An OpenFlow module is defined as follows: ```typescript type FlowModule = { // unique identifier of the step, used to reference its result // via 'results.' in later steps id: string; // a module can be one of many kinds, see below for more details value: | RawScript | PathScript | PathFlow | ForloopFlow | WhileloopFlow | BranchOne | BranchAll | Identity | AiAgent; // an optional summary line summary?: string; // stop the flow at this step if condition is met stop_after_if?: StopAfterIf; // for loops only: stop after all iterations if condition is met stop_after_all_iters_if?: StopAfterIf; // skip this step if condition is met skip_if?: { expr: string }; // sleep for a static or dynamic number of seconds after this step sleep?: InputTransform; // cache the results of this step for a number of seconds cache_ttl?: number; // custom timeout for this step, static or dynamic timeout?: InputTransform; // return a mocked value without actually executing the step mock?: { enabled?: boolean; return_value?: any }; // suspend the flow until it is resumed by receiving a certain number // of events before a timeout: this is how approval steps are built suspend?: { required_events?: number; timeout?: number; resume_form?: { schema?: any }; user_auth_required?: boolean; user_groups_required?: InputTransform; self_approval_disabled?: boolean; hide_cancel?: boolean; continue_on_disapprove_timeout?: boolean; }; // continue the flow even if this step fails continue_on_error?: boolean; // number of times to retry this module before passing it to the error handler retry?: Retry; }; type InputTransform = StaticTransform | JavascriptTransform; type StaticTransform = { type: 'static'; value: any; }; type JavascriptTransform = { type: 'javascript'; expr: string; }; type RawScript = { type: 'rawscript'; input_transforms: Record; content: string; language: | 'bun' | 'deno' | 'python3' | 'go' | 'bash' | 'powershell' | 'nu' | 'postgresql' | 'mysql' | 'mssql' | 'bigquery' | 'snowflake' | 'oracledb' | 'duckdb' | 'graphql' | 'nativets' | 'php' | 'rust' | 'csharp' | 'java' | 'ruby' | 'rlang' | 'ansible'; path?: string; lock?: string; tag?: string; }; type PathScript = { type: 'script'; input_transforms: Record; path: string; hash?: string; }; type PathFlow = { type: 'flow'; input_transforms: Record; path: string; }; type ForloopFlow = { type: 'forloopflow'; modules: FlowModule[]; iterator: InputTransform; skip_failures?: boolean; parallel?: boolean; parallelism?: InputTransform; }; type WhileloopFlow = { type: 'whileloopflow'; modules: FlowModule[]; skip_failures?: boolean; }; type BranchOne = { type: 'branchone'; default: FlowModule[]; branches: Array<{ summary?: string; expr: string; modules: FlowModule[]; }>; }; type BranchAll = { type: 'branchall'; parallel?: boolean; branches: Array<{ summary?: string; skip_failure?: boolean; modules: FlowModule[]; }>; }; type Identity = { type: 'identity'; }; // AI agent step that can call its configured tools (scripts, flows, // MCP servers, web search) to accomplish a task, see the OpenAPI // definition for the full type type AiAgent = { type: 'aiagent'; input_transforms: Record; tools: AgentTool[]; }; type StopAfterIf = { expr: string; skip_if_stopped?: boolean; error_message?: string; }; type Retry = { constant?: { attempts: number; seconds: number; }; exponential?: { attempts: number; multiplier: number; seconds: number; random_factor?: number; }; retry_if?: { expr: string }; }; ``` ### FlowModule value The `value` field of the `FlowModule` type can be one of these 9 kinds: - `identity`: the most simple one, it passes its input as output. Useful for debugging. - `rawscript`: embed a full script (in any of the [supported languages](../getting_started/0_scripts_quickstart/index.mdx)) inside the Flow. Useful for custom logic and ad-hoc scripts. - `script`: a reference to a Script by its path (including a path to the Hub using the `hub/` prefix). - `flow`: a reference to another Flow by its path, run as a subflow. - `forloopflow`: run a [for-loop](../flows/12_flow_loops.md), which iterates over an iterator - a list in general - that is constructed by evaluating the JavaScript expression in: `iterator`. The result of this module is the results of the iterations collected as a list. Iterations can run in parallel with a configurable parallelism. - `whileloopflow`: run a [while-loop](../flows/22_while_loops.mdx) that repeats its modules until a `stop_after_if` condition inside the loop is met. - `branchone`: run exactly one branch out of many, based on a predicate. Predicates are evaluated in-order and the first one that matches, gets to run. In case none matches, the default branch is run. The result of this module is the result of the branch that was run. - `branchall`: run many branches with all their modules, sequentially or in parallel. One can decide to skip failure of a particular branch. The result of this module is the results of the branches collected as a list. - `aiagent`: run an [AI agent](../flows/1_flow_editor.mdx) configured with a provider, a prompt and a set of tools (scripts, flows, MCP servers) that it can call to accomplish its task. ### Input transforms `RawScript`, `PathScript` and `PathFlow` modules contain `input_transforms`, which is a mapping between fields (i.e. input of the module) to either a static JSON value, or a raw JavaScript expression. The `input_transforms` is the way to do the piping from any other previous steps, variable, or resources to one of the inputs of your script/module. Since it is actual JavaScript (although a restricted JavaScript, for example, fetch is limited to getting secrets and variables), it is very flexible. One interesting pattern that this allows is that you can **compose complex strings directly**, so you could imagine composing your email body or SQL query using string interpolation and populating it with previous results. The Windmill Editor makes it very easy to do so, using the properties picker: ![Prop picker](./prop_picker.png 'Prop picker') More details at: ### Conditional stop after There's also the `stop_after_if` optional object: ```typescript type StopAfterIf = { expr: string; skip_if_stopped?: boolean; error_message?: string; }; ``` If present: - `stop_after_if.expr`: evaluate a JavaScript expression that takes the result as an input to decide if the Flow should stop there. Useful to stop a Flow that is meant to watch for changes if there are no changes. - `stop_after_if.skip_if_stopped`: used to flag stopped runs as skippable. It is useful in the context of Flows being triggered very often to watch for changes as you might want to ignore the runs that have been skipped. - `stop_after_if.error_message`: if set, stop with an error carrying this message instead of a success. ### Suspend and resume The module-level `suspend` object determines the number of events (resume messages) needed to progress to the next step in the Flow. This is useful for inserting user inputs during the execution of a Flow, such as approving (resume) or disapproving (cancel) a Flow. See [Approval/Suspend steps](../flows/11_flow_approval.mdx). Resume messages are sent to `https://app.windmill.dev/w//jobs/resume/` as POST or GET requests. Requests must have JSON payload, either as the request body (with `Content-Type: application/json` header) for POST requests, or as the value to the `payload` query parameter as a **base64url** encoded JSON value (`?payload=${base64url_encoded_json}`) for GET requests. When enough resume messages are received, the next job starts with its `input_transforms` evaluated with two notable variables in scope: - `resume`: The payload from the most recent resume message. - `resumes`: A list of payloads from all resume messages in the order they were received - most recent at the end of the list. Alternatively, a job can be immediately canceled by a request to a similar endpoint at `../jobs/cancel/..`. In this case, the Flow will quit, with the cancellation payload as the result and without retrying or running further steps or the failure modules. ### Retries `Retry` sets the retry policy for a module. It is optional and is reset on every successful run: - `constant`: Retry `attempts` times with `seconds` delay after each try. - `exponential`: Applies the **exponential backoff** strategy to the retries - meaning the delay will be multiplied after each unsuccessful attempt, with an optional `random_factor` jitter percentage. If all the retries are exhausted, the failure module - if any - is called. - `retry_if`: an optional JavaScript expression over the error to decide whether a failure should be retried at all. Et voilà, we have completed our tour of OpenFlow. --- ## Script editor Source: https://www.windmill.dev/docs/script_editor # Script editor In Windmill, Scripts are the basis of all major features (they are the steps of [flows](../getting_started/6_flows_quickstart/index.mdx), [linked to apps components](../apps/3_app-runnable-panel.mdx), used as [backend runnables](../full_code_apps/2_backend_runnables/index.mdx) in full-code apps, or can be [run as standalone](../triggers/index.mdx)). A Script can be written in: [TypeScript (Bun & Deno)](../getting_started/0_scripts_quickstart/1_typescript_quickstart/index.mdx), [Python](../getting_started/0_scripts_quickstart/2_python_quickstart/index.mdx), [Go](../getting_started/0_scripts_quickstart/3_go_quickstart/index.mdx), [Bash](../getting_started/0_scripts_quickstart/4_bash_quickstart/index.mdx), [PowerShell](../getting_started/0_scripts_quickstart/4_bash_quickstart/index.mdx), [Nu](../getting_started/0_scripts_quickstart/4_bash_quickstart/index.mdx), [SQL](../getting_started/0_scripts_quickstart/5_sql_quickstart/index.mdx) (PostgreSQL, MySQL, MS SQL, BigQuery, Snowflake, Oracle, DuckDB), [REST & GraphQL](../getting_started/0_scripts_quickstart/6_rest_grapqhql_quickstart/index.mdx), [PHP](../getting_started/0_scripts_quickstart/8_php_quickstart/index.mdx), [Rust](../getting_started/0_scripts_quickstart/9_rust_quickstart/index.mdx), [C#](../getting_started/0_scripts_quickstart/11_csharp_quickstart/index.mdx), [Java](../getting_started/0_scripts_quickstart/13_java_quickstart/index.mdx), [Ruby](../getting_started/0_scripts_quickstart/14_ruby_quickstart/index.mdx), [Ansible](../getting_started/0_scripts_quickstart/10_ansible_quickstart/index.mdx) or [R](../getting_started/0_scripts_quickstart/15_rlang_quickstart/index.mdx). Any other language can run through [Docker](../getting_started/0_scripts_quickstart/7_docker_quickstart/index.mdx). Its two most important components are the input [JSON Schema](../core_concepts/13_json_schema_and_parsing/index.mdx) specification and the [code content](../code_editor/index.mdx). Scripts in languages with dependencies (TypeScript, Python, Go, PHP, Rust, C#, Java, Ruby and more) also have an auto-generated [lockfile](../advanced/6_imports/index.mdx) that ensures that executions of the same Script always use the exact same set of versioned dependencies. To fit Windmill's execution model, the code must always have a main function, which is its entrypoint when executed as an individual serverless endpoint or a [Flow](../flows/1_flow_editor.mdx) module and typed parameters used to infer the script's inputs and [auto-generated UI](../core_concepts/6_auto_generated_uis/index.mdx): ```typescript async function main(param1: string, param2: { nested: string }) { ... } ``` ```python def main(param1: str, param2: dict, ...): ... ``` ```go func main(x string, nested struct{ Foo string \`json:"foo"\` }) (interface{}, error) { ... } ``` For scripts with numerous lines of code (+1,000), we recommend splitting the logic into [Flows](../flows/1_flow_editor.mdx) or [Sharing common logic](../advanced/5_sharing_common_logic/index.mdx). ## Script editor features The Script editor is made of the following features: ## Workflows as code One way to write distributed programs that execute distinct jobs is to use [flows](../flows/1_flow_editor.mdx) that chain scripts together. Another approach is to write a program that defines the jobs and their dependencies, and then execute that program directly in your script. This is known as [workflows as code](../core_concepts/31_workflows_as_code/index.mdx). Use `workflow()` and `task()` to define checkpoint-based orchestration where each task runs as a separate job with its own logs, and the workflow suspends between tasks. ![Workflows as code](../core_concepts/31_workflows_as_code/wac-editor-1.png 'Workflows as code') ## Code editor features For features specific to the [Code editor](../code_editor/index.mdx), check: --- ## Concurrency limit Source: https://www.windmill.dev/docs/script_editor/concurrency_limit # Concurrency limits The Concurrency limits feature allows you to define concurrency limits for scripts, flows and inline scripts within flows. Its primary goal is to prevent exceeding the API Limit of the targeted API, eliminating the need for complex workarounds using worker groups. ![Concurrency limit](../assets/code_editor/concurrency_limit.png) Concurrency limit is a [Cloud plans and Self-Hosted Enterprise Edition](/pricing) feature. Concurrency limit can be set from the Settings menu. When jobs reach the concurrency limit, they are automatically queued for execution at the next available optimal slot given the time window. The Concurrency limit operates globally and across flow runs. It involves three key parameters: ## Max number of executions within the time window The maximum number of executions allowed within the time window. If the number of executions exceeds this limit, the job is queued for execution at the next available optimal slot. ## Time window in seconds Set in seconds, the time window defines the period within which the maximum number of executions is allowed. ## Custom concurrency key This parameter is optional. Concurrency keys are global, you can have them be workspace specific using the variable `$workspace`. You can also use an argument's value using `$args[name_of_arg]`. Jobs can be filtered from the [Runs menu](../core_concepts/5_monitor_past_and_future_runs/index.mdx) using the Concurrency Key. --- ## Custom environment variables Source: https://www.windmill.dev/docs/script_editor/custom_environment_variables # Custom environment variables In a self-hosted environment, Windmill allows you to set custom [environment variables](../core_concepts/47_environment_variables/index.mdx) for your scripts. This feature is useful when a script needs an environment variable prior to the main function executing itself. For instance, some libraries in Go do some setup in the 'init' function that depends on environment variables. To add a custom environment variable to a script in Windmill, you should follow this format: `=`. Where `` is the name of the environment variable and `` is the corresponding value of the environment variable. --- ## Customize ui Source: https://www.windmill.dev/docs/script_editor/customize_ui # Generated UI Main function's arguments can be given advanced settings that will affect the inputs' [auto-generated UI](../core_concepts/6_auto_generated_uis/index.mdx) and [JSON Schema](../core_concepts/13_json_schema_and_parsing/index.mdx). From the script's [Settings](./settings.mdx), pick "Generated UI" tab. Here is an example on how to define a [Python](../getting_started/0_scripts_quickstart/2_python_quickstart/index.mdx) list as an enum of strings using the `Generated UI` menu. Each argument has the following settings: - **Name**: the name of the argument (defined in the main Function). - **Type**: the type of the argument (defined in the main Function): Integer, Number, String, Boolean, Array, Object, or Any. - **Description**: the description of the argument. - **Custom Title**: will be displayed in the UI instead of the field name. - **Placeholder**: will be displayed in the input field when the field is empty. If not set, the default value (directly set from the script code) will be used. The placeholder is disabled depending on the field type, format, etc. - **Field settings**: advanced settings depending on the type of the field. Below is the list of advanced settings for each type of field: | Type | Advanced Configuration | | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Integer | Min and Max. Currency. Currency locale. | | Number | Min and Max. Currency. Currency locale. | | String | Min textarea rows. Disable variable picker. Is Password (will create a [variable](../core_concepts/2_variables_and_secrets/index.mdx) when filled). Field settings: - File (base64) | Enum | Format: email, hostname, uri, uuid, ipv4, yaml, sql, date-time | Pattern (Regex) | | Boolean | No advanced configuration for this type. | | Resource | No advanced configuration for this type. | | Object | Object properties, or a Template (path to a [`json_schema` resource](../core_concepts/3_resources_and_types/index.mdx#json-schema-resources)) that contains a JSON schema with the properties. | | Array | - Items are strings | Items are strings from an enum | Items are objects (JSON) | Items are numbers | Items are bytes | | Any | No advanced configuration for this type. | --- ## Debugger Source: https://www.windmill.dev/docs/script_editor/debugger # Script debugger Windmill's [script editor](./index.mdx) includes a built-in debugger for [Python](../getting_started/0_scripts_quickstart/2_python_quickstart/index.mdx) and [Bun (TypeScript)](../getting_started/0_scripts_quickstart/1_typescript_quickstart/index.mdx) scripts. It lets you set breakpoints, step through code, inspect variables, view the call stack, and evaluate expressions in an interactive console. ## Starting a debug session 1. Open a Python or TypeScript (Bun) script in the script editor. 2. Click the **Debug** button in the toolbar (next to the run controls). 3. The editor enters debug mode, indicated by the **Exit Debug** button and debug controls appearing in the toolbar. ## Breakpoints Click the gutter (left margin) next to a line number to toggle a breakpoint. A red dot appears to indicate an active breakpoint. When the script execution reaches a breakpoint, it pauses and the current line is highlighted. ## Debug controls Once paused at a breakpoint, use the toolbar controls to: - **Continue** — resume execution until the next breakpoint or end of script. - **Step over** — execute the current line and move to the next one. - **Step into** — enter a function call on the current line. - **Step out** — run until the current function returns. - **Stop** — terminate the debug session. ## Variables The **Variables** panel displays all local and global variables in the current scope. Values update in real time as you step through code. Use the filter input to search for specific variables. ## Call stack The **Call Stack** panel shows the current execution stack, listing each function frame with its file and line number. Click a frame to navigate to that location in the editor. ## Console The **Console** panel at the bottom of the editor allows you to evaluate expressions in the current scope while paused. Type an expression and press Enter to see its result. Use the up arrow to recall previous expressions. --- ## Multiplayer Source: https://www.windmill.dev/docs/script_editor/multiplayer # Multiplayer The Multiplayer feature allows you to collaborate with team members on scripts simultaneously. Multiplayer is a [Cloud & Enterprise Self-Hosted](/pricing) feature. --- ## Perpetual scripts Source: https://www.windmill.dev/docs/script_editor/perpetual_scripts # Running services with perpetual scripts Perpetual scripts restart upon ending unless canceled. ## How to enable perpetual scripts In the script's [Settings](./settings.mdx), go to the Runtime tab and enable "Perpetual Script", then [Deploy](../core_concepts/0_draft_and_deploy/index.mdx) the script. ## How to disable perpetual scripts Canceling one [job](../core_concepts/20_jobs/index.mdx) from a perpetual script is enough to disable it. You can do it from "Cancel" button. ![Cancel perpetual script](../assets/script_editor/cancel.png 'Cancel perpetual script') You can also click on "Scale down to zero" in the "Current runs" tab. ![Scale down to zero](../assets/script_editor/scale_down_to_zero.png 'Scale down to zero') ## Tutorial To learn more about Perpetual Scripts, you can visit our tutorial on how to use a perpetual script to implement a service in Windmill leveraging [Apache Kafka](https://kafka.apache.org/): --- ## Script kinds Source: https://www.windmill.dev/docs/script_editor/script_kinds # Script kind You can attach additional functionalities to Scripts by specializing them into specific Script kinds. From the [Settings](./settings.mdx) of a script, the "Metadata" tab lets you define the following Script kinds: ## Actions Actions are the basic building blocks for the flows. ## Trigger scripts These are used as the first step in flows, most commonly with an internal state and a schedule to watch for changes on a external system, and compare it to the previously saved state. If there are changes,it _triggers_ the rest of the flow, i.e. subsequent Scripts. ## Approval scripts Suspend a flow until it's approved. An Approval Script will interact with the Windmill API using any of the Windmill clients to retrieve a secret approval URL and resume/cancel endpoints. Most common scenario for Approval scripts is to send an external notification with an URL that can be used to resume or cancel a flow. ## Error handlers Handle errors for Flows after all retries attempts have been exhausted. If it does not return an exception itself, the Flow is considered to be "recovered" and will have a success status. So in most cases, you will have to rethrow an error to have it be listed as a failed flow. ## Preprocessors Preprocessors are used to preprocess the data before it is used in the flow. --- ## Settings Source: https://www.windmill.dev/docs/script_editor/settings # Settings Each script has settings associated with it, enabling it to be defined and configured in depth. ![Script settings](../../static/images/script_languages.png 'Script settings') ## Metadata Metadata is used to define the script's path, summary, description, language and kind. ### Summary Summary (optional) is a short, human-readable summary of the Script. It will be displayed as a title across Windmill. If omitted, the UI will use the `path` by default. It can be pre-filled automatically using [Windmill AI](../core_concepts/22_ai_generation/index.mdx): ### Path Path is the Script's unique identifier that consists of the [script's owner](../core_concepts/16_roles_and_permissions/index.mdx#permissions-and-access-control), and the script's name. The owner can be either a user, or a group of users ([folder](../core_concepts/8_groups_and_folders/index.mdx#folders)). ### Description This is where you can give instructions to users on how to run your Script. It supports markdown. ### Language Language of the script. Windmill supports: - [TypeScript](../getting_started/0_scripts_quickstart/1_typescript_quickstart/index.mdx) (Bun & Deno) - [Python](../getting_started/0_scripts_quickstart/2_python_quickstart/index.mdx) - [Go](../getting_started/0_scripts_quickstart/3_go_quickstart/index.mdx) - [Bash & Powershell & Nu](../getting_started/0_scripts_quickstart/4_bash_quickstart/index.mdx) - [SQL](../getting_started/0_scripts_quickstart/5_sql_quickstart/index.mdx) (PostgreSQL, MySQL, MS SQL, BigQuery, Snowflake) - [Rest & GraphQL](../getting_started/0_scripts_quickstart/6_rest_grapqhql_quickstart/index.mdx) - [Docker](../getting_started/0_scripts_quickstart/7_docker_quickstart/index.mdx) You can configure the languages that are visible and their order. The setting applies to scripts, flows and apps and is global to all users within a workspace but only configurable by [admins](../core_concepts/16_roles_and_permissions/index.mdx#admin). ![Configurable Default Languages](../assets/script_editor/configurable-languages.png 'Configurable Default Languages') ### Script kind You can attach additional functionalities to Scripts by specializing them into specific Script kinds (Actions, Trigger, Approval, Error handler, Preprocessor). ## Runtime Runtime settings allow you to configure how your script is executed. ![Script runtime](../../static/images/script_runtime.png "Script runtime") ### Concurrency limits The Concurrency limit feature allows you to define concurrency limits for scripts and inline scripts within flows. ### Debouncing Job debouncing prevents redundant job executions by canceling pending jobs with identical characteristics when new ones are submitted within a specified time window. ### Worker group tag Scripts can be assigned custom [worker groups](../core_concepts/9_worker_groups/index.mdx) for efficient execution on different machines with varying specifications. For scripts saved on the script editor, select the corresponding worker group tag in the [settings](../script_editor/settings.mdx) section. ![Worker group tag](../core_concepts/9_worker_groups/select_script_builder.png.webp) ### Cache Caching a script step means caching the results for a certain duration. If the script is triggered with the same inputs during the given duration, it will return the cached result. ### Timeout Add a custom timeout for this script, for a given duration. If enabled to execution will be stopped after the timeout. ### Perpetual script Perpetual scripts restart upon ending unless canceled. ### Dedicated workers In this mode, the script is meant to be run on [dedicated workers](../core_concepts/9_worker_groups/index.mdx) that run the script at native speed. Can reach >1500rps per dedicated worker. Only available on enterprise edition and for Python3, Deno and Bun. For other languages, the efficiency is already on par with deidcated workers since they do not spawn a full runtime. ### Delete after use Delete [logs](../core_concepts/14_audit_logs/index.mdx), arguments and results after use. :::warning This settings ONLY applies to [synchronous webhooks](../core_concepts/4_webhooks/index.mdx#synchronous) or when the script is used within a [flow](../flows/1_flow_editor.mdx). If used individually, this script must be triggered using a synchronous endpoint to have the desired effect. The logs, arguments and results of the job will be completely deleted from Windmill once it is complete and the result has been returned. The deletion is irreversible. ::: You can also configure a **retention period** instead of immediate deletion: toggle the setting and specify a number of seconds. After job completion, logs, arguments and results will be scheduled for deletion after the specified delay. This is available per-script and per-flow-step. [Enterprise](/pricing) only. ### High priority script Jobs within a same job queue can be given a [priority](../core_concepts/20_jobs/index.mdx#high-priority-jobs) between 1 and 100. Jobs with a higher priority value will be given precedence over jobs with a lower priority value in the job queue. ### Runs visibility When this [option](../core_concepts/5_monitor_past_and_future_runs/index.mdx#invisible-runs) is enabled, manual [executions](../core_concepts/5_monitor_past_and_future_runs/index.mdx) of this script are invisible to users other than the user running it, including the [owner(s)](../core_concepts/16_roles_and_permissions/index.mdx). This setting can be overridden when this script is run manually from the advanced menu (available when the script is [deployed](../core_concepts/0_draft_and_deploy/index.mdx)). ## Generated UI main function's arguments can be given advanced settings that will affect the inputs' [auto-generated UI](../core_concepts/6_auto_generated_uis/index.mdx) and [JSON Schema](../core_concepts/13_json_schema_and_parsing/index.mdx). Here is an example on how to define a [Python](../getting_started/0_scripts_quickstart/2_python_quickstart/index.mdx) list as an enum of strings using the `Generated UI` menu. ## Triggers Triggers allow you to automate the execution of your scripts based on various events or conditions. ![Script triggers](../../static/images/script_triggers.png "Script triggers") ### Webhooks Each Script and Flow created in Windmill gets autogenerated webhooks. The webhooks depend on how they are triggered, and what their return values are. ### Schedules Schedules let you run your script at specified intervals or times, perfect for recurring tasks or periodic data updates. ### Routes Windmill supports custom HTTP routes to trigger a script or flow. ### Websocket Windmill can connect to WebSocket servers and trigger runnables (scripts, flows) when a message is received. ### Postgres Windmill can connect to a [Postgres](https://www.postgresql.org/) database and trigger runnables (scripts, flows) in response to database transactions (INSERT, UPDATE, DELETE) on specified tables, schemas, or the entire database. ### Kafka Windmill can connect to Kafka brokers and trigger scripts or flows when messages are received on specific topics. This enables real-time processing of events from your Kafka ecosystem. ### NATS Windmill can connect to NATS brokers and trigger scripts or flows when messages are received on specific subjects. This enables real-time processing of events from your NATS ecosystem. ### SQS triggers Windmill can connect to Amazon SQS queues and trigger scripts or flows when messages are received. This enables event-driven processing from your AWS ecosystem. Preprocessors can transform the SQS message data before it reaches your script or flow. ### MQTT triggers Windmill can connect to an MQTT broker, subscribe to specific topics, and trigger scripts or flows when messages are received, enabling event-driven processing. Preprocessors can transform the MQTT message data before it reaches your script or flow. ### GCP Pub/Sub triggers Windmill can connect to Google Cloud Pub/Sub subscriptions and trigger scripts or flows when messages are published to a topic. Preprocessors can transform the Pub/Sub message data before it reaches your script or flow. ### Email Scripts and flows can be triggered by email messages sent to a specific email address, leveraging SMTP. --- ## Vs code scripts Source: https://www.windmill.dev/docs/script_editor/vs_code_scripts # Run scripts in VS Code The Windmill VS Code extension allows you to run your scripts and preview the output within VS Code. This feature lets you preview your work without leaving your code editor. ![VS Code extension](../../blog/2023-11-20-vscode/vscode_extension.png 'VS Code extension') --- ## Triggers Source: https://www.windmill.dev/docs/triggers # Triggers Windmill scripts and flows can be triggered in various ways. On-demand triggers: - [Auto-generated UIs](./index.mdx#auto-generated-uis) - [Full-code apps](#full-code-apps) - [Customized UIs with the low-code app editor (legacy)](#customized-uis-with-the-low-code-app-editor-legacy) - [Trigger from flows](#trigger-from-flows) - [Workflows as code](#workflows-as-code) (scripts only) - [Schedule](#schedule) - [Command-line interface (CLI)](#cli-command-line-interface) - [Trigger from API](#trigger-from-api) - [Trigger from LLM clients with MCP](#trigger-from-llm-clients-with-mcp) Triggers from external events: - [Webhooks](#webhooks), including from [Slack](#webhooks-trigger-scripts-from-slack) - [Emails](#emails) - [Custom HTTP routes](#custom-http-routes) - [WebSocket triggers](#websocket-triggers) - [Postgres triggers](#postgres-triggers) - [Kafka triggers](#kafka-triggers) - [NATS triggers](#nats-triggers) - [SQS triggers](#sqs-triggers) - [MQTT triggers](#mqtt-triggers) - [GCP triggers](#gcp-triggers) - [Azure Event Grid triggers](#azure-event-grid-triggers) - [Native triggers](#native-triggers) (Nextcloud, Google Drive, Google Calendar) - [Scheduled polls](#scheduled-polls-scheduling--trigger-scripts) :::info Scripts and Flows in Windmill [Scripts](../getting_started/0_scripts_quickstart/1_typescript_quickstart/index.mdx) are sequences of instructions that automate tasks or perform specific operations. [Flows](../flows/1_flow_editor.mdx) are sequences of scripts that execute one after another or in parallel. Both are hosted in workspaces. ::: ## On-demand triggers ### Auto-generated UIs Windmill automatically generates user interfaces (UIs) for scripts and flows based on their parameters. By analyzing the main function parameters, it creates an input specification in the JSON Schema format, which is then used to render the UI. Users do not need to interact with the JSON Schema directly, as Windmill simplifies the process and allows for optional UI customization. This feature is also usable directly in the script editor to test a script in the making: ### Customized UIs with the low-code app editor (legacy) Windmill also provides a legacy WYSIWYG low-code app editor. It allows you to build your own UI with drag-and-drop components and to connect your data to scripts and flows in minutes. For new apps, we recommend [full-code apps](#full-code-apps) instead. You can also [automatically generate](../core_concepts/6_auto_generated_uis/index.mdx) a dedicated app to execute your script. ### Full-code apps For full control over your UI, build [full-code apps](../full_code_apps/index.mdx) with React or Svelte. Your frontend calls backend runnables that run on Windmill workers, giving you framework-level flexibility while keeping Windmill's execution, permissions and logging. ![Full-code app demo](../getting_started/9_full_code_apps_quickstart/full_code_app_demo.png 'Full-code app demo') ### Trigger from flows Flows are basically sequences of scripts that execute one after another or [in parallel](../flows/13_flow_branches.md#branch-all). Flows themselves can be triggered from other flows. This is called inner flows. ![Inner flows](./inner_flow.png "Inner flows") ### Workflows as code Flows are not the only way to write distributed programs that execute distinct jobs. Another approach is to write a program that defines the jobs and their dependencies, and then execute that program within a [Python](../getting_started/0_scripts_quickstart/2_python_quickstart/index.mdx) or [TypeScript](../getting_started/0_scripts_quickstart/1_typescript_quickstart/index.mdx) script. This is known as workflows as code. ![Flow as code](../core_concepts/31_workflows_as_code/wac-editor-1.png) ### Schedule Windmill allows you to schedule scripts using a user-friendly interface and control panel, **similar to [cron](https://crontab.guru/)** but with more features. You can create schedules by specifying a script or flow, its arguments, and a CRON expression to control the execution frequency, ensuring that your tasks run automatically at the desired intervals. ### CLI (Command-line interface) The `wmill` CLI allows you to interact with Windmill instances right from your terminal. ### Trigger from API Windmill provides a RESTful API that allows you to interact with your Windmill instance programmatically. In particular, the operation [run script by path](https://app.windmill.dev/openapi.html#/operations/runScriptByPath) is designed to trigger a script by its path, passing the required arguments. It's an efficient way to run a script from another script. ### Trigger from LLM clients with MCP Windmill supports the [Model Context Protocol (MCP)](https://modelcontextprotocol.io/introduction) to trigger scripts and flows from LLM clients. All you need is an MCP URL to connect your LLM client to Windmill. ## Triggers from external events ### Webhooks In Windmill, webhooks are autogenerated for each Script and Flow, providing either asynchronous or synchronous execution modes. You can use preprocessors to transform the incoming webhook payload before it reaches your script or flow. These webhooks accept incoming HTTP requests, allowing users to easily trigger their Windmill scripts and flows from external services by simply sending a POST request to the appropriate authenticated webhook URL (requires passing a [token](../core_concepts/4_webhooks/index.mdx#webhook-specific-tokens) as Bearer or query arg). Their purpose is to have the script run when it receives an input from its associated webhook. Using webhooks, you could also trigger a script from other scripts. #### Webhooks: trigger scripts from Slack One use case of webhooks is [building a Slackbot with Windmill](/blog/handler-slack-commands). Windmill uses Slack to trigger scripts and flows by establishing Slackbots and creating specific commands. By connecting Slack with Windmill, parsing incoming Slack commands, and leveraging Windmill workflows, operational teams can trigger complex automations directly from Slack. ### Emails Scripts and flows can be triggered by email messages sent to a specific email address, leveraging [SMTP](https://www.cloudflare.com/learning/email-security/what-is-smtp/). Preprocessors can be used to parse and transform the email content before it's processed by your script or flow. ### Custom HTTP routes Windmill allows you to define custom HTTP routes for your scripts and flows. You can use preprocessors to transform incoming HTTP requests, handling headers, query parameters, and path parameters before they reach your script or flow. ### WebSocket triggers Windmill can connect to WebSocket servers and trigger runnables (scripts, flows) when a message is received. Preprocessors can transform the websocket message data before it's passed to your script or flow. ### Postgres triggers Windmill can connect to a [Postgres](https://www.postgresql.org/) database and trigger runnables (scripts, flows) in response to database transactions (INSERT, UPDATE, DELETE) on specified tables, schemas, or the entire database. ### Kafka triggers Windmill can connect to Kafka brokers and trigger scripts or flows when messages are received on specific topics. This enables real-time processing of events from your Kafka ecosystem. Preprocessors can transform the Kafka message data before it reaches your script or flow. ### NATS triggers Windmill can connect to NATS servers and trigger scripts or flows when messages are received on specific subjects. This enables real-time processing of events from your NATS ecosystem. Preprocessors can transform the NATS message data before it reaches your script or flow. ### SQS triggers Windmill can connect to an Amazon SQS queues and trigger scripts or flows when messages are received. This enables event-driven processing from your AWS ecosystem. Preprocessors can transform the SQS message data before it reaches your script or flow. ### MQTT triggers Windmill can connect to an MQTT broker, subscribe to specific topics, and trigger scripts or flows when messages are received, enabling event-driven processing. Preprocessors can transform the MQTT message data before it reaches your script or flow. ### GCP triggers Windmill can connect to Google Cloud Pub/Sub, subscribe to specific topics, and trigger scripts or flows when messages are published, enabling event-driven processing. Preprocessors can transform the Pub/Sub message data before it reaches your script or flow. ### Azure Event Grid triggers Windmill can subscribe to Azure Event Grid topics, system topics, domains and Event Grid Namespace topics and trigger scripts or flows when events are delivered. Supports classic push, CloudEvents 1.0 push, and CloudEvents pull with lock-token ack. Preprocessors can transform the event data before it reaches your script or flow. ### Native triggers Native triggers allow external services like Nextcloud, Google Drive, and Google Calendar to push events directly to Windmill, triggering scripts or flows in real-time via webhooks or watch channels. ### Scheduled polls (Scheduling + Trigger scripts) A particular use case for schedules are [Trigger scripts](../flows/10_flow_trigger.mdx). Trigger scripts are used in [Flows](../flows/1_flow_editor.mdx) and are designed to pull data from an external source and return all of the new items since the last run, without resorting to external webhooks. A trigger script is intended to be used as scheduled poll with [schedules](../core_concepts/1_scheduling/index.mdx) and [states](../core_concepts/3_resources_and_types/index.mdx#states) (rich objects in JSON, persistent from one run to another) in order to compare the execution to the previous one and process each new item in a [for loop](../flows/12_flow_loops.md). If there are no new items, the flow will be skipped. For more complex objects or when you need structured relational data instead of simple state tracking, consider using [Data tables](../core_concepts/11_persistent_storage/data_tables.mdx) which provide a workspace-scoped SQL database for storing and querying data. You could set your script in a flow after a Trigger script to have it run only when new data is available. ## Test triggers You can test your triggers in test mode: ## Suspended mode When a trigger is in suspended mode, it continues to accept payloads and queue jobs, but those jobs won't run automatically. This is useful for debugging your runnable or trigger logic without disabling the trigger entirely. To enable suspended mode, toggle the "Suspend job execution" option in the trigger settings: ![Enable suspended mode](./enable_suspended_mode.png) ### Managing suspended jobs You can review all suspended jobs by clicking the "See suspended jobs" button: ![Open suspended jobs](./open_suspended_jobs_button.png) This opens a table showing all queued jobs for the trigger: ![Suspended jobs table](./suspended_jobs_table.png) From this table, you can: - Resume individual jobs to execute them - Discard jobs that are no longer needed - Resume all jobs at once - Discard all jobs at once ### Updating trigger configuration If you modify the trigger's configuration (such as changing the runnable, retry settings, or error handler) and save, resumed jobs will run using the updated configuration: ![Reassigned suspended jobs](./reassigned_suspended_jobs.png) :::warning If your old runnable had a preprocessor, the new one should have one too (and vice versa), as the arguments format differs based on whether a preprocessor is present. ::: --- ## Azure triggers Source: https://www.windmill.dev/docs/triggers/azure_triggers # Azure Event Grid triggers Windmill can connect to [**Azure Event Grid**](https://learn.microsoft.com/en-us/azure/event-grid/overview) and trigger runnables (scripts, flows) when events are delivered from custom topics, system topics, domains, or Event Grid Namespace topics. Azure Event Grid triggers are a [self-hosted Enterprise](/pricing) feature. They are disabled on the [Cloud](/pricing). ## Trigger modes A single Azure trigger covers three delivery modes, selected via the **Edition** (Basic / Namespace) and **Delivery** (Push / Pull) toggles in the editor: | Mode | Edition | Delivery | Use when | |------|---------|----------|----------| | `basic_push` | Basic | Push | Reacting to first-party Azure events from custom topics, [system topics](https://learn.microsoft.com/en-us/azure/event-grid/system-topics) (Storage, Resource Manager, Key Vault, Service Bus, IoT Hub control-plane, etc.) or [domains](https://learn.microsoft.com/en-us/azure/event-grid/event-domains). | | `namespace_push` | Namespace | Push | Pushing [CloudEvents 1.0](https://learn.microsoft.com/en-us/azure/event-grid/cloud-event-schema) events from [Event Grid Namespace topics](https://learn.microsoft.com/en-us/azure/event-grid/concepts-event-grid-namespaces) over HTTP. | | `namespace_pull` | Namespace | Pull | Pulling events from an Event Grid Namespace topic with lock-token ack/reject — enables dead-lettering and batched consumption. | All three modes subscribe with the **CloudEvents 1.0** schema (`eventDeliverySchema: CloudEventSchemaV1_0`), so basic, namespace push and namespace pull deliveries share the same payload parser. ## How to use ### Configure the Azure Service Principal Select an existing **Azure Service Principal** resource or create a new one. The resource provides the credentials Windmill uses to manage subscriptions and (for pull mode) to receive events: - `azureTenantId` - `azureClientId` - `azureClientSecret` The Azure subscription ID is not a field on the resource — Windmill extracts it at runtime from the ARM path of the topic or namespace you pick as the trigger scope. The service principal must have enough permissions on the target scope (topic, namespace, or domain) to: - List topics, system topics and namespaces - Create and delete Event Grid subscriptions - For `namespace_pull`: call the `:receive`, `:acknowledge`, and `:reject` data-plane endpoints on the namespace topic The built-in role **EventGrid Contributor** covers the control-plane operations; **EventGrid Data Sender/Receiver** covers the namespace data-plane calls. Refer to the [Azure Event Grid security and authentication](https://learn.microsoft.com/en-us/azure/event-grid/security-authorization) documentation for the complete list. ### Select the scope resource The editor auto-loads the resources the service principal can access: - **Basic edition**: pick a custom topic or system topic from the list (system topics are tagged with `(system)`). - **Namespace edition**: pick an Event Grid Namespace, then pick a topic inside it. Click **Refresh** if you just created a topic or namespace and it hasn't appeared yet. ### Subscription name Windmill creates (or reuses) an Event Grid subscription named according to the **Subscription name** field. The name must be 3–50 characters, letters, digits, and hyphens only. If left empty, Windmill auto-generates one in the form `windmill-{workspace}-{trigger_path}` (truncated to 50 chars). > A subscription name is unique per `(subscription_name, scope_resource_id, workspace_id)` — two triggers cannot claim the same subscription on the same scope. ### Push endpoint (push modes only) For `basic_push` and `namespace_push`, Windmill registers the subscription's webhook URL as: ``` {base_endpoint}/api/azure/w/{workspace_id}/{trigger_path} ``` Example: a trigger at `u/alice/cool_trigger` in workspace `demo` becomes: ``` {base_endpoint}/api/azure/w/demo/u/alice/cool_trigger ``` Windmill handles both Event Grid handshakes automatically: - The classic `SubscriptionValidationEvent` handshake (basic Event Grid) - The CloudEvents 1.0 `OPTIONS` abuse-protection handshake (namespace push) Push deliveries are authenticated with a server-managed shared secret. Windmill stores only the sha256 hash; the plaintext is sent to Azure once during subscription create/update (Azure stores it as `isSecret: true` on the delivery attribute and attaches it to each delivery in an `X-Windmill-Secret` header). The secret is regenerated on every save of the trigger. ### Event type filters (optional) Restrict the trigger to specific event types (one per line) — for example `Microsoft.Storage.BlobCreated` or `Microsoft.Resources.ResourceWriteSuccess`. Leave empty to receive every event delivered to the subscription. ### Choose the runnable Select the [script](../../script_editor/index.mdx) or [flow](../../flows/1_flow_editor.mdx) to execute when events are received. ### Delete behavior When you delete a trigger, the editor offers an **Also delete Azure subscription** toggle. Leave it on to clean up the Event Grid subscription in Azure; turn it off to keep the Azure side in place (for example, when handing the subscription over to another trigger). ## Implementation examples Windmill delivers the event's CloudEvent `data` field to your runnable as `payload`, passed through unchanged. For binary CloudEvents (where `data_base64` is set instead of `data`), `payload` is the base64 string and decoding is the script's job. ### Basic script ```typescript export async function main(payload: any) { // `payload` is the CloudEvent `data` field — typically a JSON object whose // shape depends on the event source (Storage, Resource Manager, Key Vault, // a custom topic, etc.). See the Azure docs for per-source schemas. console.log('Event data:', payload); return { processed: true }; } ``` ### Using a preprocessor If you configure a [preprocessor](../../core_concepts/43_preprocessors/index.mdx), you can extract fields before they reach the main function. The preprocessor receives an `event` object with the full CloudEvent envelope alongside the `payload`. #### Azure Event Grid trigger object - `payload`: CloudEvent `data` field (usually a JSON object), or the `data_base64` string for binary CloudEvents - `id`: CloudEvents `id` - `source`: CloudEvents `source` - `type`: CloudEvents `type` (e.g. `Microsoft.Storage.BlobCreated`) - `subject`: CloudEvents `subject` - `time`: CloudEvents `time` - `specversion`, `datacontenttype`, `dataschema`: optional CloudEvents attributes when the event sets them - `delivery_type`: `"push"` or `"pull"` - `headers`: HTTP request headers (push modes); minimal map in pull mode - `lock_token`: lock token used to ack/reject the message (pull mode only) - `trigger_path`: path of the trigger that received the event (push modes only) ```typescript export async function preprocessor( event: { kind: 'azure', payload: any, id: string, source: string, type: string, subject?: string, time?: string, specversion?: string, datacontenttype?: string, dataschema?: string, delivery_type: 'push' | 'pull', headers?: Record, lock_token?: string, trigger_path?: string, } ) { if (event.kind !== 'azure') { throw new Error(`Expected azure trigger kind got: ${event.kind}`); } return { eventType: event.type, subject: event.subject, data: event.payload, }; } console.log('Subject:', subject); console.log('Data:', data); } ``` ## Testing with capture The trigger editor includes a **Capture** button that listens for real events without running the runnable. Windmill creates a companion subscription suffixed with `-wm-capture` (subscription names are truncated to 39 chars before appending the suffix, so they stay within the 50-char Azure limit). The captured payloads can then be applied as script or flow arguments, or used to generate a schema. ## Troubleshooting - **Permission denied on topic/namespace listing**: the service principal is missing `Microsoft.EventGrid/*/read` on the subscription or resource group. Grant **EventGrid Contributor** or equivalent and refresh. - **Handshake fails on push**: verify the webhook URL matches `{base_endpoint}/api/azure/w/{workspace}/{trigger_path}` and that the instance is reachable from Azure. For namespace push, the CloudEvents `OPTIONS` abuse-protection handshake must land on the same route. - **`namespace_pull` not receiving events**: confirm the service principal has the **EventGrid Data Receiver** role on the namespace, and that the subscription actually exists (check the Azure portal). - **Subscription name validation error**: names must match `[A-Za-z0-9-]{3,50}`. ## Error handling Azure triggers support local error handlers that override workspace error handlers for specific triggers. See the [error handling documentation](../../core_concepts/10_error_handling/index.mdx#trigger-error-handlers) for configuration details and examples. --- ## Email triggers Source: https://www.windmill.dev/docs/triggers/email_triggers # Email triggers Scripts and flows can be triggered by email messages sent to a specific email address, leveraging [SMTP](https://www.cloudflare.com/learning/email-security/what-is-smtp/). Email triggers on Windmill [Community Edition](/pricing) are limited to 100 emails per day. ## Configuration Email triggers is available on both [cloud](#cloud) and [self-hosted](#self-hosted) instances. ### Cloud On [cloud](https://app.windmill.dev/) instances, Email triggers is already configured. You can try it from `demo` workspace. ![Email triggers from cloud](./email_triggers_cloud.png "Email triggers from cloud") ### Self-hosted First, make sure that the port 25 is exposed either on your instance public IP or a separate IP and that it redirects to the Windmill app on port 2525. The Caddyfile already contains the necessary configuration for this. For Kubernetes, you will find example configurations for some providers on the [Windmill helm charts repository](https://github.com/windmill-labs/windmill-helm-charts). In addition, you will need to create one or two records in your DNS provider depending on your setup. If the port 25 is exposed on the same IP as the Windmill instance (e.g. [docker-compose](https://github.com/windmill-labs/windmill/blob/main/docker-compose.yml) with Caddy): - An MX record from `mail.` to ``. If the port 25 is exposed through a different IP (e.g. Kubernetes): - An A/CNAME record that points to the IP of the Windmill instance with port 25 exposed (for example `mail_server.`). - An MX record from `mail.` to the record defined above (`mail_server.` if following the example). You can choose any email domain, we suggest using `mail.`. Once you have defined the DNS settings, set the email domain in the [instance settings](../../advanced/18_instance_settings/index.mdx#email-domain) under the "Core" tab. ![Instance settings](./instance_settings.png "Instance settings") ## How to use There are two kind of email triggers: default runnable emails and custom email triggers. ### Default runnable emails Each script and flow has a default trigger email address. You will find the specific email address to use in the triggers panels. The email address takes the form `+@`. ### Custom email triggers In addition to default runnable emails, you can also create email triggers with a custom address to trigger a script or flow. The local part can only contain lowercase letters, numbers, underscores, and dots (no dashes allowed). Only workspace admins can create custom email triggers or edit the local part of existing ones. You have the option to prefix the email address with the workspace id. This is useful for avoiding conflicts when you have a staging and production workspace and you are deploying between the two (see [Deploy to prod](/docs/advanced/deploy_to_prod)). The format is `{workspace_id}-{local_part}@yourdomain.com`. On Cloud, the option is always enabled. ### Email extra arguments You can pass additional arguments to your script by adding them to the email address, formatted as query parameters. **For default runnable emails**, add them after the base32 encoded part, separated by a `+`: ``` ++env=prod&debug=true®ion=us-west@ ``` **For custom email triggers**, add them before the `@` symbol, separated by a `+`: ``` alerts+env=prod&debug=true®ion=us-west@yourdomain.com ``` These extra arguments are available in your script as `email_extra_args`: ```json { "env": "prod", "debug": "true", "region": "us-west" } ``` Your script will receive the following arguments: - `raw_email`: the raw email as a string - `parsed_email` the parsed email with the following attributes: - `headers` a dictionary with the email headers (e.g. `From`, `To`, `Subject`, `Date`) - `text_body` the text body of the email (or textified html body if none) - `html_body` the html body of the email (or htmlified text body if none) - `attachments` list of attachments with the following attributes: - `headers` a dictionary with the attachment headers - `body` the attachment data - `email_extra_args`: (optional) a dictionary containing extra arguments from the email address Attchments are uploaded to the [workspace object storage (s3)](../../core_concepts/38_object_storage_in_windmill/index.mdx#workspace-object-storage) and are replaced in `parsed_email` by s3 objects (`{ s3: "path/to/key" }`). A workspace object storage is required for attachments to be handled. Here's an example script: ```TypeScript if (email_extra_args) { console.log("Environment:", email_extra_args.env); console.log("Debug mode:", email_extra_args.debug === "true"); } // do something with the email } ``` And if you use a [preprocessor](../../core_concepts/43_preprocessors/index.mdx), the script could look like this: ```TypeScript } // Access extra args from the email address const environment = event.email_extra_args?.env || "production"; const debugMode = event.email_extra_args?.debug === "true"; // return what you want to pass to the main function return { sender_address: event.parsed_email.headers["From"][0].address, email_body: event.parsed_email.text_body, environment, debug_enabled: debugMode } } export async function main( sender_address: string, email_body: string, environment: string, debug_enabled: boolean ) { console.log(`Processing email from ${sender_address} in ${environment} environment`); if (debug_enabled) { console.log("Debug mode enabled, email body:", email_body); } // do something with the processed data } ``` From a script or flow [deployed](../../core_concepts/0_draft_and_deploy/index.mdx) page, you will find on the "Details & Triggers" - "Email" tab the email address to use. ![Trigger panel](./trigger_panel.png "Trigger panel") ### Enable/disable triggers Custom email triggers can be enabled or disabled from the trigger editor without having to delete and recreate them. Disabled triggers reject incoming emails for their address, making it easy to temporarily pause a trigger (e.g. during maintenance) and re-enable it later. Only workspace admins can toggle the enabled state. ## Custom TLS certificate By default, the self-hosted SMTP server generates a self-signed TLS certificate on startup. For production deployments you can supply your own certificate so that senders can verify the server over STARTTLS. The certificate is configured via environment variables on the server container. Two modes are supported: - **Direct PEM content**: set `SMTP_TLS_CERT_PEM` and `SMTP_TLS_KEY_PKCS8_PEM` to the full PEM contents of the certificate and private key. - **File paths**: set `SMTP_TLS_CERT_PEM_PATH` and `SMTP_TLS_KEY_PKCS8_PEM_PATH` to paths inside the container (e.g. mounted from Kubernetes secrets or Docker volumes). The private key must be in PKCS#8 format. Convert an existing key with: ```bash openssl pkcs8 -topk8 -nocrypt -in key.pem -out key_pkcs8.pem ``` When file paths are used, the certificate is reloaded from disk every 12 hours so that renewed certificates are picked up without restarting the server. If loading or validating the custom certificate fails, Windmill logs an error and falls back to the self-signed certificate. This feature is only available on [Enterprise Edition](/pricing). ## Git sync Custom email triggers are included in [Git sync](../../advanced/11_git_sync/index.mdx) alongside other event triggers (HTTP routes, WebSocket, Kafka, etc.). When the **Triggers** resource type is enabled on a git sync repository, email triggers are pushed to the repository as `*.email_trigger.yaml` files and pulled back into the workspace the same way. This lets you version, review, and promote email trigger definitions across workspaces using the regular Windmill [CLI](../../advanced/3_cli/index.mdx) / git workflow. ## Error handling Email triggers support local error handlers that override workspace error handlers for specific triggers. See the [error handling documentation](../../core_concepts/10_error_handling/index.mdx#trigger-error-handlers) for configuration details and examples. --- ## Gcp triggers Source: https://www.windmill.dev/docs/triggers/gcp_triggers # GCP Pub/Sub triggers Windmill can connect to [**Google Cloud Pub/Sub**](https://cloud.google.com/pubsub/docs/overview) and trigger runnables (scripts, flows) when messages are published on topics.\ You can configure Windmill to either **pull** messages from subscriptions or **receive pushed** messages via auto-generated endpoints. Google Cloud Pub/Sub triggers is a [self-hosted Enterprise](/pricing) feature. --- ## How to use ### Configure GCP connection - Select an existing [GCP resource](https://hub.windmill.dev/resource_types/154/gcloud) (service account credentials) or create a new one. > The service account used must have enough permissions for Windmill to fully manage Pub/Sub resources. Specifically: > > - **Pub/Sub Viewer** (`roles/pubsub.viewer`): to check if topics or subscriptions exist, list them. > - **Pub/Sub Subscriber** (`roles/pubsub.subscriber`): to attach to subscriptions and consume messages. > - **Pub/Sub Editor** (`roles/pubsub.editor`): needed to create or update subscriptions, and to optionally delete the subscription in the cloud when deleting the associated trigger if the user chooses to do so. > > If you prefer not to assign these three individually, you can simply grant the **Pub/Sub Admin** role (`roles/pubsub.admin`). > > Additionally, if you want to create **authenticated push delivery subscriptions**, the service account must also have **Service Account User** (`roles/iam.serviceAccountUser`) permission. See [Authenticate Push Subscriptions](https://cloud.google.com/pubsub/docs/authenticate-push-subscriptions) for more details. ### Subscription setup #### Select topic and subscription - **Choose a topic** from your GCP project. You can refresh the list if needed. - Decide how to set up your subscription: - **Create or update a subscription**: Windmill will create a new subscription or update an existing one. - **Use an existing subscription**: Link an existing subscription from your GCP project. ##### When creating/updating a subscription: - Specify a **Subscription ID**, or leave it empty to auto-generate one. - Choose the **delivery type**: - **Pull**: Windmill sets the subscription as a **Pull** subscription. - **Push**: Windmill sets the subscription as a **Push** subscription. - For **push delivery**, Windmill sets the subscription's push endpoint URL to match the path of the trigger.\ The format is:\ `{base_endpoint}/api/gcp/w/{workspace_id}/{trigger_path}` - Example: if the trigger path is `u/test/fabulous_trigger`, the endpoint will be:\ `{base_endpoint}/api/gcp/w/myworkspace/u/test/fabulous_trigger` - When creating or updating a **push** subscription, Windmill allows you to configure: - Whether **authentication** is enabled or disabled. Refer to [Google Cloud Pub/Sub - Managing Subscriptions](https://cloud.google.com/pubsub/docs/subscriber) for more details about delivery types. ##### When using an existing subscription: - Select an existing subscription ID **among the subscriptions fetched from the selected topic**. - Windmill will automatically detect the subscription's **delivery type** based on the cloud configuration. - If the subscription is of **push delivery** type: - The subscription's endpoint URL must match the path of the trigger that will be bound to it. - The expected format is:\ `{base_endpoint}/api/gcp/w/{workspace_id}/{trigger_path}` > **Note:** You must not have multiple subscriptions pointing to the same trigger URL (for example, two subscriptions targeting `{base_endpoint}/api/gcp/w/myworkspace/u/test/fabulous_trigger`). ### Choose the runnable - Select the **script** or **flow** to trigger when Pub/Sub messages are received. --- ## Implementation examples Below are examples for handling GCP Pub/Sub messages in Windmill. > Windmill provides the Pub/Sub message as the argument `payload` (a base64-encoded string) to your runnable. ### Basic script ```typescript try { const jsonData = JSON.parse(decoded); console.log("Received JSON data:", jsonData); // Process structured data } catch (e) { console.log("Received plain text:", decoded); // Process raw text } return { processed: true }; } ``` --- ### Using a preprocessor If you configure a [preprocessor](../../core_concepts/43_preprocessors/index.mdx), you can extract fields before they reach the main function. > Windmill provides the Pub/Sub message as the argument `payload` (a base64-encoded string) to the preprocessor. #### GCP Pub/Sub trigger object - `subscription`: Subscription ID - `topic`: Topic ID - `message_id`: Unique message ID - `publish_time`: Publish timestamp (RFC 3339 format with `Z`, e.g., `"2024-04-07T12:34:56Z"`) - `attributes`: Key-value metadata - `delivery_type`: `"push"` or `"pull"` (the type of delivery) - `ordering_key`: Ordering key (optional, if message ordering is enabled) - `headers`: HTTP headers for push delivery (only present for push) Example preprocessor: ```typescript const attributes = event.attributes || {}; const contentType = attributes['content-type'] || attributes['Content-Type']; const isJson = contentType === 'application/json'; let parsedMessage: any = decodedString; if (isJson) { try { parsedMessage = JSON.parse(decodedString); } catch (err) { throw new Error(`Invalid JSON payload: ${err}`); } } return { messageAsDecodedString: decodedString, contentType, parsedMessage, attributes }; } throw new Error(`Expected gcp trigger kind got: ${event.kind}`); } ``` Then your `main` function can simply receive the extracted arguments: ```typescript console.log("Content-Type:", contentType); console.log("Parsed Message:", parsedMessage); console.log("Attributes:", attributes); } ``` --- ## Troubleshooting - **Permission issues**: Verify the service account has required Pub/Sub permissions. If the correct permissions are set but you still encounter `unauthorized` or `permission denied` errors, it might indicate that Google has updated required permissions. Please contact Windmill support so we can investigate and assist. - **Push delivery failures**: If using existing subscription ensure the push endpoint URL matches the required format (`{base_endpoint}/api/gcp/w/{workspace_id}/{trigger_path}`) and is unique across the workspace. - **Topic or subscription not found**: Refresh the list to fetch the latest available resources. ## Error handling GCP triggers support local error handlers that override workspace error handlers for specific triggers. See the [error handling documentation](../../core_concepts/10_error_handling/index.mdx#trigger-error-handlers) for configuration details and examples. --- --- ## Http routing Source: https://www.windmill.dev/docs/triggers/http_routing # HTTP routes Windmill supports HTTP Routes as triggers to execute runnables (scripts or flows) whenever the route is hit by an external HTTP request, and it can also serve static files or websites. This feature is ideal for integrating with third-party services, custom webhooks, or internal systems where events are sent via HTTP. --- ## How it works You define a custom HTTP route with a specific method (GET, POST, PUT, PATCH, DELETE). When the route is called, Windmill triggers the selected script or flow. Each route can be protected with various authentication mechanisms, ranging from simple API keys to advanced HMAC signature validation or even fully custom logic. Among the supported authentication mechanisms, there's also **Windmill Auth**, which uses a JWT token to authenticate requests and ensure you have read access to the route and the runnable. You can generate your personal Windmill JWT token directly from your user settings and use it to securely access your HTTP routes. You can configure the route to run: - **Synchronously**: Wait for the script to complete and return the result. - **Asynchronously**: Return a job ID immediately; the script runs in the background. - **Sync SSE**: Return a [Server-Sent Events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events) stream. Useful when the runnable returns a [stream](../../core_concepts/20_jobs/index.mdx#result-streaming). Works identically to the `/run_and_stream` endpoint of [SSE stream webhooks](../../core_concepts/4_webhooks/index.mdx#sse-stream-webhooks). For more on streaming in Windmill, see [Streaming](../../core_concepts/58_streaming/index.mdx). --- ## Creating an HTTP route Windmill supports two ways to create HTTP routes: - **Manual creation**, where you define a path, method, and bind it to a runnable. - **Automatic generation** from an OpenAPI specification, enabling batch creation. #### To create a route manually: - Navigate to the **Custom HTTP routes** page. - Click the **New route** button. - Fill in the route configuration fields - Click **Save** to create the route. ### Generate routes from an OpenAPI specification Windmill can generate HTTP routes directly from an OpenAPI 2.0+ specification in JSON or YAML format. You can provide the specification in one of three ways: - Paste raw content - Upload a file - Provide a public URL #### To generate routes: - Navigate to the **Custom HTTP routes** page. - Click the **From OpenAPI spec** button. - Pick a folder for the generated routes. - Choose your input method and provide the OpenAPI spec. - Click **Generate HTTP routes**. #### Behavior and limitations: - If a path object has a `summary` field, it will be used as suffix for the trigger path. - If the `summary` exceeds 255 characters, it will be **automatically truncated** to fit the maximum allowed length. - If no `summary` is defined, Windmill generates a unique route path automatically. - Generated routes **do not include a script or flow binding** (`script_path`) by default. - This means requests to the route will return an error until a runnable is attached. - You can: - **Save routes immediately without modifying them**. - **Edit any route before or after saving**, to assign a runnable, change route path, etc. - External `$ref` references (e.g., referencing outside the spec) are **not supported**. - You must resolve them beforehand. - Only internal references (e.g., `#/components/...`) are supported. You can use `:param` in the route path and access these as `params` in a [preprocessor](../../core_concepts/43_preprocessors/index.mdx). > ℹ️ **Only workspace admins** can create routes. > Once created, all properties of a route **except the HTTP path** can be modified by any user with **write access** to the route. > Learn more about [Admins workspace](../../advanced/18_instance_settings/index.mdx#admins-workspace). --- ### Enable/disable Each HTTP route has an **enabled** toggle that can be flipped without deleting the trigger. Disabled routes stop matching incoming requests and return `404` until re-enabled, which is useful when temporarily taking an endpoint offline for maintenance, cutting over between versions, or pausing a route that is misbehaving without losing its configuration. The toggle is available from the route editor and the HTTP routes list page. Only users with write access to the trigger can change its enabled state. --- ### Workspace prefix On Windmill Cloud, all HTTP routes are automatically prefixed by the `workspace_id` (e.g., `{workspace_id}/{path}`). This ensures that different workspaces can define the same route paths independently. On self-hosted Windmill, you can optionally enable the **workspace prefix** setting to achieve the same behavior. When workspace prefix is enabled: - Multiple workspaces can define the same route path without conflict. - HTTP triggers can be deployed across different workspaces if no conflicting route exists. When workspace prefix is disabled (on self-hosted): - Route paths will be **globally unique** across the entire instance. - A route path cannot be reused by another workspace unless it is first deleted. **Example:** If workspace A creates the route `/webhooks/github`, then without workspace prefix, no other workspace can create `/webhooks/github`. With workspace prefix enabled, workspace A could have `/workspace_a/webhooks/github` and workspace B could have `/workspace_b/webhooks/github`. #### Enforcing the workspace prefix instance-wide On self-hosted instances, superadmins can enforce the workspace prefix for every HTTP route via the **HTTP route workspace prefix** instance setting (under `/#superadmin-settings`). When enabled: - New and existing routes are served under `/api/r/{workspace_id}/{route}` regardless of the per-route toggle. - The per-route workspace-prefix toggle in the route editor is disabled and labeled as enforced by the instance setting. - The setting can only be turned off again if no two workspaces share the same route path; otherwise a collision error is returned. This mirrors the equivalent `app_workspaced_route` setting for apps and is the recommended default for multi-tenant self-hosted instances. --- ### Select a script or flow - Pick the runnable to be triggered when the route is called. - Use the “Create from template” button to generate a boilerplate if needed. Example script: ```ts export async function main(/* args from the request body */) { // your code here } ``` With a preprocessor: ```ts path: string; method: string; params: Record; query: Record; headers: Record; } ) { if (event.kind === 'http') { const { name, age } = event.body; return { user_id: event.params.id, name, age, }; } throw new Error(`Expected trigger of kind 'http', but received: ${event.kind}`); } export async function main(user_id: string, name: string, age: number) { // Do something } ``` --- ## Generate an OpenAPI specification from HTTP routes and webhooks Windmill supports generating a compliant OpenAPI 3.1 specification from your existing HTTP routes and webhook triggers. ### How it works You can export a unified OpenAPI specification that includes: - HTTP routes and webhook triggers (filtered by path or type) - Route metadata: `summary`, `description`, async/sync behavior - Security models for supported authentication types - Parameter inference (e.g., `:id` becomes an OpenAPI `path` parameter) - Auto-generated `servers`, `components`, and reusable definitions ### To generate a spec 1. Go to **Custom HTTP routes** 2. Click **To OpenAPI spec** 3. (Optional) Provide API metadata: - **Title** and **version** (default to `"Windmill API"` and `"1.0.0"` if omitted) - **Description**, **contact info**, and **license** are also supported 4. Add **filters** to include specific routes or webhooks: - HTTP Routes: by folder, path, and route pattern - Webhooks: by kind (`script` or `flow`) and path 5. Choose output format: **YAML** or **JSON** 6. Click **Generate OpenAPI document** You can then: - Preview and copy the document - Download it - Copy a ready-made `cURL` command to call the generation API ### Security mapping The OpenAPI document reflects security as follows: - **HTTP Routes**: - ✅ `Windmill Auth` → JWT bearer scheme (`bearerFormat: JWT`) - ✅ `Basic Auth` → HTTP Basic auth scheme - ✅ `API Key` → API key in header (with exact header name) - ❌ Other methods (e.g., Custom Script, Signature Auth) will not be included in the `security` field - **Webhooks**: - Always mapped to `JWT bearer` authentication (`bearerFormat: JWT`) All defined security schemes are included under `components.securitySchemes`. ### Additional behavior - For **HTTP routes**, any route segments like `:user_id` are automatically converted to OpenAPI `path` parameters - For **webhooks**, both **asynchronous** and **synchronous** endpoint variants are included in the spec - Metadata such as `summary` and `description` is included when available - Standardized `requestBodies` and `responses` are defined in the `components` section --- ## Authentication options Windmill supports several ways to secure HTTP triggers: | Method | Description | |------------------|----------------------------------------------------------------------------------------------| | **None** | Open to anyone (use only in trusted environments) | | **Windmill Auth**| Uses a Windmill-signed JWT token to ensure the requesting agent has read access to both the runnable and the trigger. The token must be provided either in the Authorization header as Bearer ``, or via a cookie named `token`. You can generate this token from your user settings. When selecting this option, you can generate a token pre-scoped with `http_triggers:read` access to the route directly from the trigger configuration. See [user tokens](../../core_concepts/59_user_tokens/index.mdx) for more details on scoped tokens. | | **API Key** | Checks a header (e.g., `x-api-key`) for a valid key stored as a resource | | **Basic Auth** | Uses HTTP Basic Authentication via a configured resource | | **Signature Auth** | Verifies a signature using HMAC or third-party formats (Stripe, GitHub, etc.) | --- ### Token scopes When **Windmill Auth** is selected, the JWT token used to call the route must carry an `http_triggers:read` scope for the route path. A fully-scoped user token calling a route at `my/route` needs the scope `http_triggers:read:my/route`. From the route configuration page, Windmill can generate a token pre-scoped to exactly this route: in the **Windmill Auth** section click **Generate token**, and a new token limited to `http_triggers:read:` is created. That token can safely be shared with the caller because it can only read (i.e., trigger) this specific HTTP route and cannot impersonate the issuing user for any other operation. See [User tokens](../../core_concepts/59_user_tokens/index.mdx#token-scopes) for the full scope format and the list of available domains. --- ### Signature Auth (HMAC-based) Use **Signature Auth** to validate incoming HTTP requests using HMAC-style signatures. - Choose a preset (e.g., Stripe, GitHub) or configure a generic HMAC check. - If your provider uses custom logic not covered by presets, you can write a **Custom Script** instead. --- ## Body processing options Depending on your setup, additional arguments can be injected into your runnable: | Option | Argument Provided | Description | |------------------|------------------------|------------------------------------------------------------------------------------------------------| | **Wrap body** | `body` | If enabled, Windmill will wrap the incoming request body inside an object under the `body` key. Useful when the payload structure is dynamic or unknown. | | **Raw body** | `raw_string` | The raw (unprocessed) request body is provided as a `raw_string` argument (type: `string`). Useful for signature verification, binary payloads, etc. | ### Example using `body` ```ts return body; } ``` ### Example using `raw_string` ```ts return JSON.parse(raw_string); } ``` --- ## CORS headers HTTP route responses include permissive CORS headers by default so that routes can be called directly from browser clients: | Header | Default value | | --- | --- | | `Access-Control-Allow-Origin` | `*` | | `Access-Control-Allow-Methods` | `GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS` | | `Access-Control-Allow-Headers` | `content-type, authorization` | `OPTIONS` preflight requests are handled automatically by the HTTP trigger handler. To override any of these defaults for a route, set the corresponding header on the response returned by your runnable using the [`wm_headers`](../../core_concepts/4_webhooks/index.mdx#custom-response-headers) mechanism (or its longer alias `windmill_headers`). Only headers that are **not** already present in the response are filled in with the defaults above, so you can restrict origins or allowed headers on a per-route basis: ```ts } ``` --- ## Serving static files or websites HTTP routes can also serve: - **Static files**: Pick a file from S3. - **Static websites**: Choose an S3 folder. Windmill will host them under your custom path, using `index.html` as a fallback if necessary. --- ## Best practices - Use **preprocessors** to parse, validate, or transform payloads before the `main()` function. - Prefer **Signature Auth** for third-party integrations that support webhook signing (e.g., Stripe, GitHub). - Use **Custom Script** authentication only when predefined options are not flexible enough. - Enable **raw_string** if you need access to the raw body for signature verification or special payloads. --- ## Troubleshooting - If the script isn't triggered: - Check that the HTTP method matches (e.g., POST vs GET). - Verify authentication is correctly set. - Ensure any custom scripts throw errors to help debug failures. - For signature validation failures: - Double-check the secret key and signature header. - Ensure `raw_body` is enabled if validation depends on the raw body. --- ## Custom script authentication (Advanced) Use a **Custom Script** for full control over authentication and validation when built-in methods are not enough. This gives access to: - Raw payload - Headers, query, and route parameters - Secrets stored as [variables](../../core_concepts/2_variables_and_secrets/index.mdx) Example script for HMAC signature validation: ```ts const SECRET_KEY_VARIABLE_PATH = "u/admin/well_backlit_variable"; body: any; raw_string: string | null; route: string; path: string; method: string; params: Record; query: Record; headers: Record; }, ) { if (event.kind !== 'http') { throw new Error('Expected a http event'); } if (!event.raw_string) { throw new Error('Missing raw_string in event'); } const signature = event.headers['x-signature'] || event.headers['signature']; if (!signature) { throw new Error('Missing signature in request headers.'); } const timestamp = event.headers['x-timestamp'] || event.headers['timestamp']; if (timestamp) { const timestampValue = parseInt(timestamp, 10); const currentTime = Math.floor(Date.now() / 1000); const TIME_WINDOW_SECONDS = 5 * 60; if (isNaN(timestampValue)) { throw new Error('Invalid timestamp format.'); } if (Math.abs(currentTime - timestampValue) > TIME_WINDOW_SECONDS) { throw new Error('Request timestamp is outside the acceptable window.'); } } const isValid = await verifySignature(signature, event.raw_string, timestamp); if (!isValid) { throw new Error('Invalid signature.'); } return JSON.parse(event.raw_string); } async function verifySignature(signature: string, body: string, timestamp?: string): Promise { const dataToVerify = timestamp ? `${body}${timestamp}` : body; const secretKey = await wmill.getVariable(SECRET_KEY_VARIABLE_PATH); const expectedSignature = crypto .createHmac('sha256', secretKey) .update(dataToVerify) .digest('hex'); try { return crypto.timingSafeEqual( Buffer.from(signature), Buffer.from(expectedSignature) ); } catch (error) { console.error('Signature comparison error:', error); return false; } } ``` > ℹ️ When using **Custom Script**, the `raw_body` option is automatically enabled. ## Error handling HTTP routes support local error handlers that override workspace error handlers for specific routes. See the [error handling documentation](../../core_concepts/10_error_handling/index.mdx#trigger-error-handlers) for configuration details and examples. --- ## Kafka triggers Source: https://www.windmill.dev/docs/triggers/kafka_triggers # Kafka triggers Windmill can connect to [Kafka](https://kafka.apache.org/) brokers servers and trigger runnables (scripts, flows) when a message is received. Listening is done from the servers, so it doesn't take up any workers. Kafka triggers is a [self-hosted Enterprise](/pricing) feature. ![Kafka triggers](./kafka_triggers.png 'Kafka triggers') ## Kafka resource configuration Before creating a Kafka trigger, you need to set up a Kafka [resource](../../core_concepts/3_resources_and_types/index.mdx). Head to the Resources page, click "Add resource" and select `kafka`. The resource requires: - **Brokers**: List of broker hostnames in the format `hostname:port` - **Security**: Authentication and encryption settings ### Security options | Security mode | Description | |--------------|-------------| | PLAINTEXT | No authentication or encryption. Use only for development. | | SASL_PLAINTEXT | Username/password authentication without encryption. Supports PLAIN, SCRAM-SHA-256, SCRAM-SHA-512 mechanisms. | | SSL | TLS encryption with optional client certificate authentication. | | SASL_SSL | Username/password authentication with TLS encryption. | | SASL_GSSAPI | Kerberos (GSSAPI) authentication without encryption. | | SASL_SSL_GSSAPI | Kerberos (GSSAPI) authentication with TLS encryption. | | SASL_SSL_OAUTHBEARER | OAuth 2.0 / OIDC `client_credentials` authentication with TLS encryption. | ### Kerberos (GSSAPI) authentication For enterprise environments using Kerberos, select `SASL_GSSAPI` or `SASL_SSL_GSSAPI` security mode. | Property | Description | Required | |----------|-------------|----------| | kerberos_service_name | Kerberos principal name of the Kafka broker service (default: `kafka`) | No | | kerberos_principal | Client's Kerberos principal (e.g., `user@REALM.COM`) | Yes | | keytab_path | Path to keytab file mounted on the server | No* | | keytab_base64 | Base64-encoded keytab content | No* | *Either `keytab_path` or `keytab_base64` must be provided. **Using keytab_base64**: Encode your keytab file with `base64 -w0 /path/to/keytab` and paste the result. This is stored securely and written to a temporary file at runtime. **Using keytab_path**: Mount the keytab file on the server container and provide the path. This is useful when deploying with Kubernetes secrets or Docker volumes. :::warning Non-root server and keytab permissions When running the Windmill server as a non-root user (e.g., uid 1000 or 1001), the keytab file must be readable by that user. Keytab files are typically created with restrictive permissions (600, owner-only). If you see `kinit: Permission denied` errors, either: - Use `keytab_base64` instead of `keytab_path` - Windmill writes the keytab with the server's ownership - In Kubernetes, set `defaultMode: 0644` on your Secret volume mount: ```yaml volumes: - name: keytab secret: secretName: kafka-keytab defaultMode: 0644 ``` - In Docker Compose, ensure the keytab file on the host has readable permissions before mounting ::: :::info Kerberos configuration Kafka triggers run on the **server** (`windmill-app` pod), not on workers. When using `keytab_path`, the keytab file and `/etc/krb5.conf` must be mounted on the server pod, not worker pods. When using SASL_SSL_GSSAPI, you can also provide CA certificates for TLS verification. ::: #### Example krb5.conf ```ini [libdefaults] default_realm = EXAMPLE.COM dns_lookup_realm = false dns_lookup_kdc = false # Recommended for containerized environments - see troubleshooting below dns_canonicalize_hostname = false [realms] EXAMPLE.COM = { kdc = kdc.example.com admin_server = kdc.example.com } [domain_realm] .example.com = EXAMPLE.COM example.com = EXAMPLE.COM ``` #### Docker Compose example ```yaml services: windmill_server: image: ghcr.io/windmill-labs/windmill-ee:latest volumes: - ./krb5.conf:/etc/krb5.conf:ro ``` #### Helm/Kubernetes example Create a ConfigMap for krb5.conf: ```yaml apiVersion: v1 kind: ConfigMap metadata: name: krb5-config data: krb5.conf: | [libdefaults] default_realm = EXAMPLE.COM dns_canonicalize_hostname = false [realms] EXAMPLE.COM = { kdc = kdc.example.com } ``` Then in your Helm values: ```yaml windmill: app: volumes: - name: krb5-config configMap: name: krb5-config volumeMounts: - name: krb5-config mountPath: /etc/krb5.conf subPath: krb5.conf ``` :::warning Troubleshooting "Server not found in Kerberos database" If you encounter this error, GSSAPI may be constructing the wrong service principal due to DNS canonicalization. GSSAPI performs reverse DNS lookups on the broker IP to determine the hostname for the service principal. Verify that reverse DNS for your broker IP returns the expected hostname matching your Kerberos SPN. For example, if your SPN is `kafka/kafka.example.com@REALM`, then `host ` should return `kafka.example.com`. In containerized environments (Docker, Kubernetes) where reverse DNS may return internal container/pod names instead of the expected hostname, add to your krb5.conf: ```ini [libdefaults] dns_canonicalize_hostname = false ``` This tells GSSAPI to use the hostname as configured in your broker list without reverse DNS canonicalization. ::: ### OAUTHBEARER (OIDC) authentication For brokers that delegate authentication to an external identity provider (IdP), select `SASL_SSL_OAUTHBEARER`. Windmill performs the OAuth 2.0 `client_credentials` grant against the IdP's token endpoint and forwards the resulting bearer token to the broker. | Property | Description | Required | |----------|-------------|----------| | client_id | OAuth client ID registered with the IdP | Yes | | client_secret | OAuth client secret registered with the IdP | Yes | | token_endpoint_url | Full URL of the IdP token endpoint (e.g., `https://login.example.com/realms/kafka/protocol/openid-connect/token`) | Yes | | scope | Space-separated list of OAuth scopes to request from the IdP | No | | extensions | Comma-separated `key=value` pairs sent as SASL/OAUTHBEARER extensions | No | | ca | PEM-encoded CA certificate for verifying the broker | No | | certificate | PEM-encoded client certificate for mutual TLS | No | | key | PEM-encoded client private key | No | | key_password | Password for the client private key, if encrypted | No | :::info OIDC token flow Token acquisition happens on the **server** (`windmill-app` pod), not on workers. Tokens are cached and refreshed automatically by `librdkafka` before they expire, so the trigger maintains a long-lived consumer connection without re-issuing credentials on every message. ::: #### Example ``` client_id = my-kafka-client client_secret = token_endpoint_url = https://login.example.com/realms/kafka/protocol/openid-connect/token scope = kafka ``` :::warning TLS is required OAUTHBEARER must be paired with TLS (`SASL_SSL_OAUTHBEARER`). Sending bearer tokens over a plaintext connection would expose them on the wire, so the unencrypted variant is intentionally not offered. ::: ## How to use Create a new trigger on the Kafka triggers page. Add a Kafka resource with the broker hostnames (hostname:port) and the security settings. Specify the topics the trigger should listen to. The group id is automatically filled in from the current workspace and the trigger path. You can change it if necessary. It indicates the consumer group to which the trigger belongs. This guarantees that even if the trigger stops listening for a while, it will receive the messages it missed when it starts listening again. The following settings are available under the **Advanced** section of the Kafka trigger editor. ### Initial offset The 'Initial offset' setting controls where the consumer starts reading when the consumer group has no committed offset (i.e. the first time the trigger is created or when using a new group ID): - Latest (default): Only new messages produced after the trigger starts will be consumed. - Earliest: All existing messages in the topic will be consumed from the beginning. This corresponds to Kafka's `auto.offset.reset` configuration. It has no effect once the consumer group has a committed offset. ### Auto-commit offsets By default, Windmill commits offsets once the job has been successfully pushed to the queue. You can disable auto-commit to get manual control over when offsets are committed. When auto-commit is disabled: - Messages are delivered to your script but offsets are **not** committed automatically. - If the trigger restarts before you commit, unacknowledged messages will be re-delivered. - You must call the SDK helper from your script to commit offsets. The [preprocessor](../../core_concepts/43_preprocessors/index.mdx) event payload includes `trigger_path`, `partition` and `offset` fields that you pass to the commit function. The SDK call stores the offset in the database and returns immediately. Every 5 seconds, the trigger's consumer picks up stored offsets and commits the highest one for each topic/partition pair to Kafka. ```TypeScript trigger_path: string; payload: string; brokers: string[]; topic: string; group_id: string; partition: number; offset: number; } ) { const msg = JSON.parse(atob(event.payload)); return { data: msg, topic: event.topic, partition: event.partition, offset: event.offset, trigger_path: event.trigger_path }; } } ``` ```Python import wmill import base64 import json def preprocessor(event: dict): msg = json.loads(base64.b64decode(event["payload"])) return { "data": msg, "topic": event["topic"], "partition": event["partition"], "offset": event["offset"], "trigger_path": event["trigger_path"], } def main( data: dict, topic: str, partition: int, offset: int, trigger_path: str, ): # process the message # ... # commit the offset after successful processing wmill.commit_kafka_offsets(trigger_path, topic, partition, offset) ``` ### Reset offset to earliest For existing triggers, clicking the 'Reset offset to earliest' button restarts the consumer with the offset reset to the beginning of the topic, causing all messages to be re-processed. ### Runnable configuration Once the Kafka resource and settings are set, select the runnable that should be triggered by this trigger. The received webhook base64 encoded payload will be passed to the runnable as a string argument called `payload`. Here's an example script: ```TypeScript export async function main(payload: string) { // do something with the message } ``` And if you use a [preprocessor](../../core_concepts/43_preprocessors/index.mdx), the script could look like this: ```TypeScript payload: string, // base64 encoded payload brokers: string[]; topic: string; // the specific topic the message was received from group_id: string; partition: number; offset: number; } ) { if (event.kind !== "kafka") { throw new Error(`Expected a kafka event`); } // assuming the message is a JSON object const msg = JSON.parse(atob(event.payload)); // define args for the main function return { message_content: msg.content, topic: event.topic, partition: event.partition, offset: event.offset }; } export async function main(message_content: string, topic: string, partition: number, offset: number) { // do something with the message content and topic } ``` ## Filters Kafka triggers support message filtering so that only messages matching specified criteria trigger the runnable. Filters match against the message parsed as JSON, referencing its top-level keys (e.g. `type`). The base64 encoding only applies to the payload the script/flow receives — filters run on the message before it is encoded. Each filter is a key/value pair where: - **Key**: A top-level key of the message (e.g. `type`). Keys are matched literally — to match a nested field, use the parent key with an object value (see the superset note below). - **Value**: The expected value at that key (can be a string, number, object, or array) When multiple filters are configured, a filter logic selector lets you choose how they are combined: - **AND** (default): the message must match all filters to trigger the runnable. - **OR**: the message triggers the runnable as soon as it matches any one of the filters. The matching uses a superset check — the message value at the given key must contain all fields from the filter value (additional fields in the message are allowed). For example, to match a nested field, use key `data` with value `{"status": "active"}` to match `{"data": {"status": "active", ...}}`. Existing triggers without an explicit filter logic default to **AND**. Filters can be configured in the Kafka trigger editor UI using the filter section. You can add multiple key/value pairs, and a human-readable preview is shown for each filter (e.g. `payload.type == "order_created"`). :::info Filters use the same mechanism as [WebSocket trigger](../3_websocket_triggers/index.mdx) filters and are evaluated server-side before dispatching the job. ::: ## Error handling Kafka triggers support local error handlers that override workspace error handlers for specific triggers. See the [error handling documentation](../../core_concepts/10_error_handling/index.mdx#trigger-error-handlers) for configuration details and examples. --- ## Mqtt triggers Source: https://www.windmill.dev/docs/triggers/mqtt_triggers # MQTT triggers Windmill can connect to an [**MQTT**](https://mqtt.org/mqtt-specification/) broker and trigger runnables (scripts, flows) in response to messages published to specified topics. MQTT triggers are not available on the [Cloud](/pricing). ## How to use ### Configure MQTT resource - Select an existing [MQTT resource](https://hub.windmill.dev/resource_types/225/mqtt) or create a new one - Provide broker hostname and port - Add authentication credentials and certificates as required by your broker ### Select runnable - Choose the script or flow to execute when messages are published to your subscribed topics ### Configure topic subscriptions - Specify one or more topics to subscribe to - Set appropriate QoS level for each topic #### Quality of Service (QoS) levels | Level | Description | When to use | |-------|-------------|-------------| | **0** | **At most once** – Message delivered once or not at all without confirmation | Choose when it is okay for your script/flow to not be triggered (if the message is lost) or triggered only once. | | **1** | **At least once** – Guaranteed delivery but may arrive multiple times | Choose when it is okay for your script/flow to be triggered again by an already received message from the broker. | | **2** | **Exactly once** – Guaranteed delivery exactly once | Choose when you need your script/flow to be triggered only once and avoid any duplicates. | For more information about MQTT QoS, see the [MQTT QoS Documentation](https://www.hivemq.com/blog/mqtt-essentials-part-6-mqtt-quality-of-service-levels/). #### MQTT topic structure MQTT topics are case-sensitive and follow a hierarchical structure (e.g., `home/sensor/temperature`). For best practices on MQTT topics, see the [MQTT Topics Documentation](https://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices/). ### Advanced MQTT options By default, Windmill uses **MQTT version 5**. However, you can choose to use **MQTT version 3** or **MQTT version 5** with specific associated options. - **MQTT v3 options**: - **Clean Session** (default: true): [Learn more](https://www.emqx.com/en/blog/mqtt5-new-feature-clean-start-and-session-expiry-interval#clean-session-in-mqtt-3-1-1) - **Client ID**: [Learn more](https://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html) - **MQTT v5 options**: - **Clean Start** (default: true): [Learn more](https://www.emqx.com/en/blog/mqtt5-new-feature-clean-start-and-session-expiry-interval#introduction-to-clean-start) - **Session Expiry Interval**: [Learn more](https://www.emqx.com/en/blog/mqtt5-new-feature-clean-start-and-session-expiry-interval#introduction-to-session-expiry-interval) - **Topic Alias Maximum**: [Learn more](https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901051) - **Client ID**: [Learn more](https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901059) ## Implementation examples Below are code examples demonstrating how to handle MQTT messages in your Windmill scripts. You can either process messages directly in a basic script or use a preprocessor for more advanced message handling and transformation before execution. ### Basic script ```typescript // Parse JSON if applicable try { const jsonData = JSON.parse(textPayload); console.log("Received JSON data:", jsonData); // Process JSON data } catch (e) { // Handle as plain text console.log("Received text data:", textPayload); // Process text data } return { processed: true, message: "MQTT message processed successfully" }; } ``` ### Script with preprocessor If you use a [preprocessor](../../core_concepts/43_preprocessors/index.mdx), the preprocessor function receives the message payload as base64 encoded string and an MQTT object with the following fields: #### MQTT object - **`topic`**: The MQTT topic on which the message was received. - **`retain`**: Boolean indicating if the message is retained. - **`pkid`**: Packet identifier (if QoS > 0). - **`qos`**: Quality of Service level. - **`v5`**: MQTT v5 properties (optional). #### MQTT v5 properties - **`payload_format_indicator`**: Indicates if the payload is UTF-8 encoded or binary. - **`topic_alias`**: An alias for the topic name. - **`response_topic`**: A topic for the recipient to send a response to. - **`correlation_data`**: Correlation data for request/response. - **`user_properties`**: A list of user-defined properties. - **`subscription_identifiers`**: Subscription identifiers. - **`content_type`**: The content type of the payload. For more information about MQTT v5 properties, see the [MQTT v5 Properties Documentation](https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901109). ```typescript /** * General Trigger Preprocessor * * ⚠️ This function runs BEFORE the main function. * * It processes raw trigger data (e.g., MQTT, HTTP, SQS) before passing it to `main()`. * Common tasks: * - Convert binary payloads to string/JSON * - Extract metadata * - Filter messages * - Add timestamps/context * * The returned object determines `main()` parameters: * - `{a: 1, b: 2}` → `main(a, b)` * - `{payload}` → `main(payload)` * * @param event - Trigger data and metadata (e.g., MQTT, HTTP) * @returns Processed data for `main()` */ const uint8Payload = Uint8Array.from(payloadAsString, (c) => c.charCodeAt(0)); return { contentType: event.v5?.content_type, payload: uint8Payload, payloadAsString }; } // We assume the script is triggered by an MQTT message, which is why an error is thrown for other trigger kinds. // If the script is intended to support other triggers, update this logic to handle the respective trigger kind. throw new Error(`Expected mqtt trigger kind got: ${event.kind}`) } /** * Main Function - Handles processed trigger events * * ⚠️ Called AFTER `preprocessor()`, with its return values. * * @param payload - Raw binary payload * @param payloadAsString - Decoded string payload * @param contentType - MQTT v5 content type (if available) */ export async function main(payload: Uint8Array, payloadAsString: string, contentType?: string) { } ``` ## Error handling MQTT triggers support local error handlers that override workspace error handlers for specific triggers. See the [error handling documentation](../../core_concepts/10_error_handling/index.mdx#trigger-error-handlers) for configuration details and examples. --- ## Native triggers Source: https://www.windmill.dev/docs/triggers/native_triggers # Native triggers Native triggers allow external services to push events directly to Windmill and trigger [scripts](../../script_editor/index.mdx) and [flows](../../flows/1_flow_editor.mdx). Unlike [scheduled polls](../../flows/10_flow_trigger.mdx), native triggers receive real-time push notifications from the external service via webhooks, so your runnables execute as soon as events occur. Currently supported services: - [Nextcloud](#nextcloud-triggers) - file, folder, and calendar change events - [Google](#google-triggers) - drive and calendar change events --- ## How it works Native triggers use [OAuth](../../advanced/27_setup_oauth/index.mdx) to authenticate with the external service. Each trigger registers a webhook or watch channel on the external service, which sends notifications to Windmill when events occur. Windmill then executes the configured script or flow with the event data. The general setup follows two steps: 1. **Connect the service** via workspace settings (OAuth authentication) 2. **Create a trigger** on a script or flow, selecting the service and configuring which events to watch --- ## Setup ### Workspace integration Before creating native triggers, you need to connect the external service in your workspace settings. 1. Go to **Workspace settings** > **Integrations** > **Native triggers** 2. Click **Connect** on the service you want to use (Nextcloud or Google) 3. Provide the [OAuth](../../advanced/27_setup_oauth/index.mdx) credentials (client ID and client secret) 4. Complete the OAuth authorization flow 5. Choose a [resource](../../core_concepts/3_resources_and_types/index.mdx) path where the connection will be stored in the workspace ![Workspace integration](./workspace_integration.png "Workspace settings showing native triggers integrations for Nextcloud and Google") The integration creates a [resource](../../core_concepts/3_resources_and_types/index.mdx) in your workspace at the path you specify. The OAuth token is refreshed automatically, so you don't need to re-authenticate manually. You can use this resource in your scripts to query the service API with the same credentials used by the trigger. For Google integrations, the created [resource](../../core_concepts/3_resources_and_types/index.mdx) is of type `gworkspace`. The OAuth scopes used are: - `https://www.googleapis.com/auth/drive.readonly` - `https://www.googleapis.com/auth/calendar.readonly` - `https://www.googleapis.com/auth/calendar.events` If you need additional scopes (e.g. write access), set up a separate resource with the desired scopes. A superadmin can share [instance-level](../../advanced/18_instance_settings/index.mdx) Google workspace settings so that workspace admins can connect Google native triggers without configuring their own OAuth client. The credentials are not exposed to workspace admins. ![Instance gworkspace settings](./instance_gworkspace.png "Instance-level gworkspace settings with shared credentials toggle and redirect URI") For Nextcloud, you also need to provide the base URL of your Nextcloud instance. ### Create a trigger Once the workspace integration is configured: 1. Go to **Nextcloud** or **Google** tab (only visible once the corresponding [workspace integration](#workspace-integration) is configured) 2. Click **New trigger** 3. Select the script or flow to trigger 4. Configure the service-specific options (see sections below) 5. Save the trigger You can also create native triggers directly from a script or flow's **Triggers** tab. --- ## Nextcloud triggers Firs of all make sure you set up the [workspace integration](#workspace-integration) for Nextcloud. Nextcloud native triggers watch for file, folder, and calendar changes on a [Nextcloud](https://nextcloud.com/) instance and trigger a script or flow when events occur. ### Prerequisites - Nextcloud 33 or later - The [Windmill integration app](https://apps.nextcloud.com/apps/integration_windmill) installed on your Nextcloud instance - [Pretty URLs](https://docs.nextcloud.com/server/latest/admin_manual/installation/source_installation.html#pretty-urls) enabled on your Nextcloud instance ### Configuration When creating a Nextcloud trigger, pick a script or flow (or use **Create from template**), then configure: - **Event** - select the event type to listen for. Available events include: - Calendar events: object created, moved, trashed, restored, or changed in a Nextcloud calendar - File/folder events: node created, changed, or written ![Nextcloud trigger configuration](./nextcloud_trigger.png "Nextcloud trigger creation with event type selection") ### Event payload Nextcloud sends the full event data to your script. The payload includes the authenticated user, a timestamp, and the event details: ```typescript trigger: RT.Nextcloud; }, user: { uid: string; displayName: string; }, time: number, event: any ) { // event contains the Nextcloud event details (file, folder, or calendar object) } ``` --- ## Google triggers First of all make sure you set up the [workspace integration](#workspace-integration) for Google. Google native triggers watch for changes in [Google Drive](https://drive.google.com/) or [Google Calendar](https://calendar.google.com/) and trigger a script or flow when events occur. ### Prerequisites - A Google Cloud project with the relevant API enabled ([Drive API](https://developers.google.com/drive/api), [Calendar API](https://developers.google.com/calendar/api), or both) - [OAuth 2.0](https://console.cloud.google.com/apis/credentials) credentials (client ID and client secret) configured with the appropriate scopes ### Configuration When creating a Google trigger, pick a script or flow (or use **Create from template**), then configure: - **Trigger Type** - choose between **Drive** and **Calendar** - **Watch Mode** (Drive only) - choose between: - **Specific file** - watch a single file or folder from My Drive, Shared with me, or Shared drives - **All changes** - watch all changes across your Google Drive (fires whenever any file is created, modified, or deleted) - **Calendar** (Calendar only) - select a calendar from your Google account to watch for event changes ![Google trigger configuration](./google_trigger.png "Google trigger creation with trigger type and watch mode selection") Both use push notifications via Google watch channels. Windmill automatically renews channels before they expire. The expiration period differs by service: 24 hours for Drive, 7 days for Calendar. ### Event payload Google push notifications only contain metadata about the change, not the full event details. To get the actual content of the change, use the [`gworkspace` resource type](../../core_concepts/3_resources_and_types/index.mdx) created during workspace integration to query the Google API. ```typescript type GoogleTriggerPayload = { channel_id: string; resource_id: string; resource_state: string; // "sync" | "exists" | "not_exists" | "update" resource_uri: string; message_number: string; channel_expiration: string; channel_token: string; // custom token set when creating the watch changed: string; // Drive-only: comma-separated list (e.g. "content,properties,permissions") }; export async function main(payload: GoogleTriggerPayload) { // Use the gworkspace resource to query the Google API for full details } ``` Use the [Google native trigger template script](https://hub.windmill.dev/scripts/gworkspace/22221/google-native-trigger-template-script-gworkspace) (also available from the UI when creating a trigger) as a starting point. --- --- ## Nats triggers Source: https://www.windmill.dev/docs/triggers/nats_triggers # NATS triggers Windmill can connect to [NATS](https://nats.io/) servers and trigger runnables (scripts, flows) when a message is received. Listening is done from the servers, so it doesn't take up any workers. NATS triggers is a [self-hosted Enterprise](/pricing) feature. ![NATS triggers](./nats_triggers.png 'NATS triggers') ## How to use ### Core NATS Create a new trigger on the NATS triggers page. Add a NATS resource with the server hostnames (`hostname[:port]`, without the `nats://` prefix) and the authentication configuration. Specify the subjects the trigger should listen to (wildcards are supported). Only one subject is supported in core NATS. Once the NATS resource and settings are set, select the runnable that should be triggered by this trigger. The received webhook base64 encoded payload will be passed to the runnable as a string argument called `payload`. Here's an example script: ```TypeScript export async function main(payload: string) { // do something with the message } ``` And if you use a [preprocessor](../../core_concepts/43_preprocessors/index.mdx), the script could look like this: ```TypeScript subject: string; // the specific subject the message was received from length: number; headers?: Record; status?: number; description?: string; } ) { if (event.kind !== "nats") { throw new Error(`Expected a nats event`); } // assuming the message is a JSON object const msg = JSON.parse(atob(event.payload)); // define args for the main function // let's assume we want to use the message content and the subject return { message_content: msg.content, subject: event.subject }; } export async function main(message_content: string, subject: string) { // do something with the message content and subject } ``` ### JetStream [JetStream](https://docs.nats.io/nats-concepts/jetstream) is also supported and enables persistence as well as listening to multiple subjects. Persistence makes sure that even if the trigger stops listening for a while, it will receive the messages it missed when it starts listening again. A stream will be created with the specified name and subjects, if no stream exists with this name. If one already exists, whether created outside or inside of Windmill, its config will be adapted to include the specified subjects but only if they are missing (considering wildcards). The rest of the config will be left untouched. **Stream subjects are never deleted automatically. If you need to delete old subjects, you have to do it manually.** A [durable push-consumer](https://docs.nats.io/nats-concepts/jetstream/consumers) will be created with the specified name and subjects. If one already exists, it will be overwritten. The consumer name is also used as the `DeliverSubject`, so make sure it's unique. ## Error handling NATS triggers support local error handlers that override workspace error handlers for specific triggers. See the [error handling documentation](../../core_concepts/10_error_handling/index.mdx#trigger-error-handlers) for configuration details and examples. --- ## Postgres triggers Source: https://www.windmill.dev/docs/triggers/postgres_triggers # Postgres triggers Windmill can connect to a [Postgres](https://www.postgresql.org/) database and trigger runnables (scripts, flows) in response to database transactions (INSERT, UPDATE, DELETE) on specified tables, schemas, or the entire database. Listening is done using Postgres's logical replication streaming protocol, ensuring efficient and low-latency triggering. Postgres triggers are not available on the [Cloud](/pricing). ## What is logical replication? Windmill's Postgres trigger feature is built on Postgres's logical replication protocol, which allows changes to a database to be streamed in real time to subscribers. Logical replication provides fine-grained control over what data is replicated by allowing the user to define publications and subscribe to specific changes. ### How logical replication works 1. **Publications**: Define what changes (e.g., INSERT, UPDATE, DELETE) should be made available for replication. Publications allow you to select specific tables or schemas to track. 2. **Replication slots**: Ensure that all changes from a publication are retained until they are successfully delivered to the subscriber (e.g., Windmill triggers). This guarantees data reliability and prevents data loss. Windmill uses logical replication to efficiently stream database changes to your configured triggers, ensuring minimal latency and high reliability. For more details, see the [Postgres documentation on logical replication](https://www.postgresql.org/docs/current/logical-replication.html). For more details, see the [Postgres documentation on logical replication streaming protocol](https://www.postgresql.org/docs/current/protocol-logical-replication.html). ## Requirements Before using Postgres triggers with Windmill, your database must be properly configured for logical replication. The primary requirement is setting the Write-Ahead Log (WAL) level to `'logical'`. ### Setting `wal_level` to `logical` You have two options to configure this setting. Both options require a restart of your Postgres instance to take effect. #### Option 1: Using SQL (requires database restart) 1. Run the following SQL command to set `wal_level` to `'logical'`: ```sql ALTER SYSTEM SET wal_level = 'logical'; ``` 2. After executing the command, restart your Postgres instance for the changes to take effect. #### Option 2: Editing the `postgresql.conf` file (requires database restart) 1. Locate and open your `postgresql.conf` file. The location of this file may vary depending on your installation. 2. Look for the `wal_level` setting. If it's not already present, **add** the following line to the file: ```ini wal_level = logical ``` If the setting is already there, **update** it to `logical`. 3. Save the file and restart your Postgres instance for the changes to take effect. ### Verifying logical replication You can verify that logical replication is enabled by running the following query: ```sql SHOW wal_level; ``` This should return: ```plaintext wal_level ----------- logical ``` ### Impact of enabling logical replication Enabling logical replication turns on detailed logging, which is essential for supporting the replication process. Be aware that this will increase the amount of data written to the Write-Ahead Log (WAL). Typically, you can expect a 10% to 30% increase in the amount of data written to the WAL, depending on the volume of write activity in your database. --- ## Additional configuration for logical replication For logical replication to work properly, you need to configure additional parameters in your `postgresql.conf` file. These parameters control the number of replication processes and slots available for replication. Both settings require a restart of your Postgres instance to take effect. #### `max_wal_senders` The `max_wal_senders` setting determines the maximum number of **walsender** processes that can run concurrently. A **walsender** is responsible for sending the Write-Ahead Log (WAL) data to subscribers for logical replication. The default value is 10, but you can increase this based on your replication needs. ```ini #max_wal_senders = 10 # max number of walsender processes (change requires restart) ``` - **Impact on Triggers**: Each active trigger in logical replication will use a **walsender** process. So, if `max_wal_senders` is set to 10, only 10 active triggers can be used at the same time. If you reach this limit, you will need to increase the `max_wal_senders` value to accommodate more active triggers. #### `max_replication_slots` The `max_replication_slots` setting determines how many **replication slots** can be created. Replication slots are used to maintain state for each logical replication subscription. This setting also limits the number of triggers that can be created for logical replication. ```ini #max_replication_slots = 10 # max number of replication slots (change requires restart) ``` - **Impact on Trigger Creation**: You can only create as many triggers as there are replication slots available. So if `max_replication_slots` is set to 10, you will be able to create a maximum of 10 triggers. If you need more triggers, you will need to increase the `max_replication_slots` value. ### Summary of limits - **Active triggers**: The number of active triggers you can have is limited by `max_wal_senders`. If you set `max_wal_senders` to 10, only 10 active triggers can be running simultaneously. - **Trigger creation**: The number of triggers you can create is limited by `max_replication_slots`. If you set `max_replication_slots` to 10, you can only create 10 triggers in total. --- ### Final considerations When configuring these settings, make sure to account for the number of active triggers and replication slots needed for your application. If you expect to have many triggers or high replication activity, you may need to increase both `max_wal_senders` and `max_replication_slots`. ## How to use Learn how to set up and configure Postgres triggers in Windmill through these key steps. ### Create a Postgres trigger To begin, navigate to the Postgres triggers page and create a new trigger. Follow the steps below to set up your environment. ### Set up a Postgres resource You need to either: - Create a new Postgres resource by providing: - Hostname, port, database name, username, and password. - Advanced options such as SSL settings if needed. - Reuse an existing Postgres resource. ### Define what to track Once the Postgres resource is configured, you can choose what to track. #### All tables The trigger will listen for transactions on all tables in the database. Example: ![Track all tables example](./track_all_tables.png 'Track all tables example') #### Specific schemas The trigger will listen for transactions on all tables within the selected schemas. Any new tables added to these schemas in the future will also be tracked automatically. Example: Tracking the `public` and `marketing` schemas: ![Track specific schemas example](./track_specific_schemas_marketing_public.png 'Track specific schemas: marketing and public') #### Specific tables The trigger will listen only for transactions on the specified tables. You can also choose which columns to retrieve when tracking specific tables. Example: In this setup, the `bakery` table in the `paris` schema is tracked, but only the `name` and `address` columns are retrieved. ![Track specific tables example](./track_specific_tables_bakery.png 'Track specific tables: user and bakery') --- ## Limitations and examples This section outlines the supported and unsupported combinations of tracking configurations, helping you avoid common setup issues and ensure your triggers work as intended. ### Valid configuration You can combine: - Schema-level tracking (e.g., `public` schema). - Specific table tracking without selecting columns. Example: Tracking the `bakery` table in the `paris` schema and all tables in the `private` and `public` schemas: ![Valid configuration example](./valid_config.png 'Valid configuration example') ### Invalid configuration You cannot combine: - Schema-level tracking with specific table tracking that includes column selection. Example: Tracking all tables in the `public` schema and the `bakery` table in the `paris` schema with selected columns (`name` and `address`): ![Invalid configuration example](./invalid_config.png 'Invalid configuration example') --- ## Additional options The following section showcases additional options provided by PostgreSQL's logical replication feature that Windmill integrates with. ### Filtering rows with WHERE condition When tracking specific tables, you can filter rows by providing a WHERE condition. Key notes: - The `WHERE` clause allows only simple expressions. - It cannot contain: - User-defined functions, operators, types, and collations. - System column references. - Non-immutable built-in functions. Important: - You only need to provide the condition, not the entire `WHERE` clause. For example, instead of writing `WHERE speciality = 'croissant'`, just provide the condition: `speciality = 'croissant'`. - If your trigger is set to track `UPDATE` and/or `DELETE` transactions, the `WHERE` clause can only reference columns that are part of the table’s replica identity. See the [REPLICA IDENTITY documentation](https://www.postgres.org/docs/current/sql-altertable.html#SQL-ALTERTABLE-REPLICA-IDENTITY) for more details. - If your trigger tracks only `INSERT` transactions, the `WHERE` clause can reference any column. For more details, refer to the [Postgres WHERE clause documentation](https://www.postgresql.org/docs/current/logical-replication-row-filter.html#LOGICAL-REPLICATION-ROW-FILTER-RESTRICTIONS). Illustration: Here’s an example showing how to filter rows based on the condition `speciality = 'croissant'` in the `bakery` table of the `paris` schema: ![Where condition example](./where_condition_paris_bakery.png 'Filtering rows example: speciality = croissant') --- ### Selecting specific columns When tracking specific tables, you can reduce the data sent to the triggered function by retrieving only the columns you need. However, **if the transaction being tracked includes `UPDATE` or `DELETE` transactions**, selecting specific columns can introduce constraints: #### Key considerations for `INSERT`, `UPDATE`, and `DELETE` transactions: - **INSERT** transactions are unaffected by this limitation, and you can select any columns, regardless of whether they are included in the replica identity. - For `UPDATE` or `DELETE` transactions, the columns you select **must be part of the table's replica identity**. If the selected columns are not part of the replica identity, the database will fail to process the query. #### What happens if the configuration is invalid? - If a trigger includes `UPDATE` or `DELETE` transactions while excluding columns required for the replica identity, the associated database query will fail. - To resolve this issue, you have two options: 1. **Update the trigger configuration**: Modify the trigger to include the columns that are part of the replica identity. 2. **Delete and recreate the publication**: Delete the existing publication and configure a new one that accommodates the necessary columns. For more details, see the [REPLICA IDENTITY documentation](https://www.postgresql.org/docs/current/sql-altertable.html#SQL-ALTERTABLE-REPLICA-IDENTITY). --- #### Example Consider the `bakery` table in the `paris` schema. The table has three columns: `id`, `name`, and `address`. By default, PostgreSQL uses the `DEFAULT` replica identity, which means it tracks the primary key column(s) to identify rows for updates and deletes. If your trigger includes `UPDATE` or `DELETE` transactions and only non-primary key columns are selected (e.g., `name` and `address`), those transactions will fail because the primary key column is required for tracking changes. #### 1. `DEFAULT` replica identity The default replica identity tracks only the primary key column(s). For the `bakery` table, the primary key could be any column, such as `id`, `bakery_id`, or another column designated by the user. - **Correct configuration**: - The trigger should track the primary key column (e.g., `id`, `bakery_id`, etc.), along with any additional columns required for your logic. - For example, if `id` is the primary key, the trigger should at least track `id`. - This ensures that `UPDATE` and `DELETE` transactions will succeed. - Here’s an example of the correct column selection (assuming the primary key is `id`): ![Correct column selection example](./correct_selection_columns.png 'Correct column selection example') - **Incorrect configuration**: - If only `name` and `address` are selected, and the primary key column (e.g., `id`) is not included in the tracked columns, `UPDATE` and `DELETE` transactions will fail. - To fix this: - Add the primary key column (e.g., `id`, `bakery_id`, or whatever the primary key column is named) to the trigger's tracked columns. - Alternatively, update or delete the publication and recreate it with the correct configuration. #### 2. `USING INDEX` replica identity With `USING INDEX index_name`, the replica identity tracks the columns of a unique index. - **Correct configuration**: - The trigger must track at least the columns that are part of the unique index. - If the `bakery` table has a unique index (e.g., `idx_bakery_name_address`) covering `name` and `address`, the trigger should track **at least** those two columns, but it can also track other columns, such as `id`, if needed for your logic. - **Incorrect configuration**: - If the columns tracked by the trigger do not match those in the unique index, `UPDATE` and `DELETE` transactions will fail. - To fix this: - Ensure the trigger tracks **at least** the columns in the `USING INDEX` replica identity or create an appropriate unique index. #### 3. `FULL` replica identity The `FULL` replica identity records the old values of **all columns** in the row. - **Correct configuration**: - The trigger can track any combination of columns, as all columns are tracked with `FULL` replica identity. - For example, tracking `name`, `address`, and the primary key column (e.g., `id`, `bakery_id`) is completely acceptable and won’t cause any issues. - **Incorrect configuration**: - There is no issue with tracking any columns when using the `FULL` replica identity because the replica identity covers all columns in the row. However, using more columns than necessary may be inefficient. - If performance is a concern, it's recommended to limit the tracked columns to those that are necessary for the trigger. #### 4. `NOTHING` replica identity The `NOTHING` replica identity records no information about the old row, which is typically used for system tables. - **Correct configuration**: - This configuration is generally not applicable to user tables like `bakery`, but if applied to the `bakery` table, the trigger would not be able to track `UPDATE` or `DELETE` transactions. - This means no `UPDATE` or `DELETE` operations would work with the trigger, as no data would be recorded for the affected rows. - **Incorrect configuration**: - The `NOTHING` replica identity will cause the trigger to fail for `UPDATE` and `DELETE` transactions. You cannot fix this without changing the replica identity to one of the other options (`DEFAULT`, `USING INDEX`, or `FULL`). #### Conclusion - **For `UPDATE` and `DELETE` operations**, ensure that the tracked columns include the primary key column (e.g., `id`, `bakery_id`, or whatever the primary key column is named) or match the requirements of the replica identity configuration. - If the default replica identity is used (`DEFAULT`), ensure that the primary key column is included in the tracked columns to avoid failures with `UPDATE` and `DELETE`. - For `USING INDEX`, make sure the trigger tracks **at least** the columns in the unique index. You can also track other columns, but the index columns must be tracked to ensure the correct behavior for `UPDATE` and `DELETE` transactions. - If the replica identity is set to `FULL`, you can safely track any columns. - Avoid using `NOTHING` replica identity if you need to track `UPDATE` and `DELETE` operations. --- ## Advanced The Advanced section provides granular control over publications and replication slots, offering flexibility beyond Windmill's default automatic management. ### Managing Postgres publications By default, Windmill automatically creates a publication and a replication slot for you when setting up a trigger. However, in the Advanced section, you can: - **Create a custom publication**: If you prefer to use your own publication, you can create it directly from the interface. - Example: Create a publication named `windmill_publication_gracious`, which tracks all tables in the `public` and `private` schemas, and is set to track only update and delete transactions. ![Creating publication example](./create_publication_example.png 'Create a custom publication: windmill_publication_illuminating') - **Choose an existing publication**: Instead of relying on the default publication created by Windmill, you can select an existing publication from your database to use for your trigger. - For example, when retrieving the publication `windmill_publication_non_violent`, all tables are tracked, and the publication tracks insert, update, and delete transactions by default. - In the image below, the tracked tables and insert transaction type for the publication are displayed. You can use the publication as is or: - Update the publication by adding or removing tables and schemas being tracked, or modifying the transaction types. - Delete the publication if no longer needed. ![Retrieving and managing publication example](./retrieve_publication_example.png 'Retrieve and manage publication: windmill_publication_non_violent') For more information on Postgres publications, refer to the [Postgres documentation on publications](https://www.postgresql.org/docs/current/logical-replication-publication.html). --- ### Managing Postgres replication slots In the Advanced section, you can also manage your replication slots. Windmill will automatically create a replication slot for you by default, but you can interact with replication slots as follows: - **Create a custom replication slot**: If needed, you can create your own replication slot directly in the interface. - Example: Create a replication slot named `windmill_replication_adored`. ![Creating Replication Slot Example](./create_replication_slot_example.png 'Creating Replication Slot Example') - **Choose an existing replication slot**: You can select an existing replication slot from your database and link it to the trigger. - Example: Retrieve and manage the replication slot `windmill_1737909146368_4zuvg52h3pge` for your trigger. ![Retrieving Replication Slot Example](./retrieve_replication_slot_example.png 'Retrieving Replication Slot Example') - **Delete a replication slot**: If a replication slot is no longer necessary, you can delete it through the interface. Windmill will display only logical replication slots and inactive slots. For more details, refer to the [Postgres documentation on replication slots](https://www.postgresql.org/docs/current/warm-standby.html#STREAMING-REPLICATION-SLOTS). --- ## Creating a script from tracked tables Windmill enables you to automatically generate a script template for specific tables and/or schemas tracked by a trigger. This feature simplifies the creation of a TypeScript script with the necessary structure to handle data passed by the trigger. ### Prerequisites - Postgres resource: a Postgres [resource](../../core_concepts/3_resources_and_types/index.mdx) must be configured in your environment to enable this feature. - At least one schema: you need to select at least one schema to track. - Specific tables and/or schema: This feature works only for specific tables and/or schemas. Make sure your selection matches the criteria for script generation. ### How to use 1. Set up Postgres resource: Ensure that a Postgres resource is configured in the resources page. 2. Select schema and tables: Choose the schema and tables you want to track. Note that this feature does not work for all tables; only those meeting the criteria for tracking will be available. 3. Click on "Create from template": After selecting the desired tables and schemas, click on the Create from Template button. This will open a new tab containing a TypeScript script. - The script will include a main function that takes an argument representing the information sent to the script. This argument is a JSON object structured as follows: ```json { "transaction_type": "insert" | "update" | "delete", "schema_name": "string", "table_name": "string", "old_row?": { ... }, "row": { ... } } ``` ### Explanation of fields - transaction_type: Specifies the type of change (either `insert`, `update`, or `delete`). - schema_name: The name of the schema being tracked (type: `string`). - table_name: The name of the table being tracked (type: `string`). - old_row (optional): Contains the previous state of the row before the change occurred. This field is only present for `update` transactions and reflects the values of the row prior to the update. - row: Contains the data of the row involved in the transaction. The data type of each field in `row` depends on the column's data type in the Postgres table. For `insert` and `update` transactions, this represents the new or updated values. For `delete` transactions, this represents the deleted row. #### Example table schema Consider a table `users` in the public schema with the following SQL definition: ```sql CREATE TABLE users ( id SERIAL PRIMARY KEY, name VARCHAR(100) NOT NULL, lastname VARCHAR(100) NOT NULL, age INT CHECK (age > 0), personal_information JSONB, created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL, updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL ); ``` For this schema, the corresponding row in the JSON object would look like this: ```json { row: { id?: number, name?: string, lastname?: string, age: number, personal_information: unknown, created_at?: Date, updated_at?: Date, } } ``` ### Key notes - Transaction types: The `transaction_type` field in the JSON object can be one of `insert`, `update`, or `delete`, depending on the change type in the tracked table. - Row data: The `row` field contains the data of the specific table or schema, and can be used directly in your script for processing. - Old row: The `old_row` field is included only for `update` transactions and contains the previous values of the row before the update occurred. This is useful for comparing changes or auditing modifications. ### Script template example Once the template is generated, you can modify it to meet your needs. Below is an example of the generated script template, based on a sample transaction, with a `users` table in the public schema. ```typescript export async function main( transaction_type: "insert" | "update" | "delete", schema_name: string, table_name: string, old_row?: { id?: number, name?: string, lastname?: string, age: number, personal_information: unknown, created_at?: Date, updated_at?: Date, }, row: { id?: number, name?: string, lastname?: string, age: number, personal_information: unknown, created_at?: Date, updated_at?: Date, } ) { } ``` --- ## Handling external database hosters When using a Postgres database hosted by external providers, special configurations might be necessary to ensure compatibility with Windmill's Postgres triggers feature. This section provides guidelines for handling various database hosters to avoid common connection issues. --- ### Neon If your Postgres database is hosted on [**Neon**](https://neon.tech), special considerations are required when configuring your Postgres resource in Windmill. By default, Neon uses **pooled connections** with `pgbouncer`, which are not compatible with Windmill's triggers due to restrictions on the `replica` parameter. #### Why pooled connections fail by default Neon’s pooled connections are managed using `pgbouncer`. By default, `pgbouncer` allows only a specific subset of startup parameters, and **logical replication**, which Windmill uses to create triggers, requires the `replica` parameter, which is not allowed by default. If you attempt to use Neon's `-pooler` host without modifying the `pgbouncer` configuration, Windmill's triggers will fail to connect because `pgbouncer` will reject the `replica` parameter. --- #### Avoiding common pitfalls: Configuring Neon with Windmill When entering the database details manually (`host`, `port`, `db_name`, `password`, `ssl_mode`, `root_certificate_pem`, etc.), you have two options: --- #### Option 1: Use the non-pooled connection host **Recommended for simplicity.** Avoid using the `-pooler` host provided by Neon. 1. Remove the `-pooler` suffix from the host. Example: - **Original `host` (with pooled connection)**: `-pooler.neon.tech` - **Updated `host` (without pooled connection)**: `.neon.tech` 2. Enter the remaining parameters as provided by Neon: - **`db_name`**: The name of the database - **`password`**: The password for your database - **`port`**: The port (usually `5432`) - **`ssl_mode`**: Typically `require` - **`root_certificate_pem`**: Optional certificate for secure SSL connections This configuration works without requiring changes to Neon's `pgbouncer` settings. ##### **Good configuration** For example, if Neon provides the following details: - `host`: `-pooler.neon.tech` - `db_name`: `my_database` - `password`: `my_password` - `port`: `5432` - `ssl_mode`: `require` Configure the Postgres resource as follows: - **`host`**: `.neon.tech` _(Remove the `-pooler` suffix.)_ - **`db_name`**: `my_database` - **`password`**: `my_password` - **`port`**: `5432` - **`ssl_mode`**: `require` - **`root_certificate_pem`**: Provide if required by Neon. --- #### Option 2: Enable pooled connections by modifying `pgbouncer` **For advanced users who prefer using pooled connections.** To use Neon’s `-pooler` host with Windmill's Postgres triggers, you must update the `pgbouncer` configuration on your Neon instance to allow the `replica` parameter. 1. **Locate the `pgbouncer` configuration file**: Check [Neon’s documentation](https://neon.tech/docs/introduction) for guidance on accessing and modifying the `pgbouncer` configuration. 2. **Update the `ignore_startup_parameters` setting**: Add `replica` to the `ignore_startup_parameters` list in the `pgbouncer` configuration file. Example: ```txt ignore_startup_parameters = replica --- ### Future updates for other database hosting services This section will be updated if additional database hosting services require special configurations for Postgres triggers. If you are using a database service other than Neon and encounter issues when setting up triggers, please **contact Windmill support** for assistance. We’ll work with you to identify and document the specific requirements needed for compatibility. --- ## Troubleshooting If you're experiencing issues when creating or running Postgres triggers, here are some common problems and how to resolve them. ### Failed to start trigger When a trigger fails to start, you may encounter one of these issues: #### 1. WAL sender limit reached When starting a trigger, you may encounter an error indicating no more room for WAL sender connections. This occurs when an active trigger attempts to establish a connection but the `max_wal_senders` limit has been reached. ##### Try any of these following solutions: - **Disable one of the running triggers**: You can disable one of the triggers using the same Postgres resource. By doing so, the resource will free up a connection, allowing you to use the newly created trigger. - **Increase the `max_wal_senders` limit**: You can increase this limit on your Postgres server to allow more connections. For more information, refer to the [Postgres documentation on max_wal_senders](https://www.postgresql.org/docs/current/runtime-config-replication.html#GUC-MAX-WAL-SENDERS). **Note**: Simply increasing the limit may not be enough if the `max_replication_slots` limit is also reached. ##### Example error: In this example, the database has `max_wal_senders` set to 2. Two triggers are already running (shown as "currently 2" in the error), preventing a third trigger from starting. To resolve this, either disable one of the running triggers or increase the database's `max_wal_senders` limit. ![Error: No more room left for WAL sender](./no_more_room_wal_sender_error.png) ### Failed to create trigger When creating a new trigger, you may encounter these issues: #### 1. Replication slot limit reached If you encounter an error stating that the `max_replication_slots` limit is reached, this error happens because Windmill, in basic mode, tries to create both a replication slot and a publication when a new trigger is set up. If the `max_replication_slots` limit is exceeded, the new replication slot cannot be created. ##### Try any of these following solutions: - **Delete an existing replication slot**: You can navigate to the [Advanced](#advanced) section and delete an unused replication slot. This will free up space for a new replication slot. - **Increase the `max_replication_slots` limit**: Increase the replication slots limit on your Postgres server to allow more replication slots. For more information, refer to the [Postgres documentation on max_replication_slots](https://www.postgresql.org/docs/current/runtime-config-replication.html#GUC-MAX-REPLICATION-SLOTS). - **Manage replication slots and publications directly**: Since Windmill automatically creates a publication and replication slot when setting up a trigger, you can go to the Advanced section of the interface to manage these elements manually. This includes creating, selecting, or deleting replication slots and publications as needed. For more information, see the section on [Managing Postgres publications and replication slots](#advanced). ##### Example error: In this example, the database has reached its maximum number of replication slots (shown as "all replication slots are in use" in the error). This prevents the creation of a new trigger since Windmill cannot create the required replication slot. To resolve this, either delete unused replication slots or increase the database's `max_replication_slots` limit. ![Error: No replication slots available](./no_replication_slots_available_error.png) --- For more help with troubleshooting, refer to the Postgres logs or contact Windmill support. ## Error handling Postgres triggers support local error handlers that override workspace error handlers for specific triggers. See the [error handling documentation](../../core_concepts/10_error_handling/index.mdx#trigger-error-handlers) for configuration details and examples. --- ## Sqs triggers Source: https://www.windmill.dev/docs/triggers/sqs_triggers # SQS triggers Windmill can connect to an [SQS](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/welcome.html) queue and trigger runnables (scripts, flows) in response to messages received. SQS triggers is a self-hosted Enterprise feature. --- ## How to use - **Pick an AWS or AWS OIDC resourcee** - Select an existing [AWS resource](../../integrations/aws.md#aws-resource) or [AWS OIDC resource](../../integrations/aws.md#aws-oidc-resource) or create a new one. - The AWS resource must have permissions to interact with SQS. - **Select the runnable to execute** - Choose the runnable (script or flow) that should be executed when a message arrives in the queue. - **Provide an SQS queue URL** - Enter the **Queue URL** of the SQS queue that should trigger the runnable. - You can find the Queue URL in the AWS Management Console under SQS. - For more details, see the [SQS Queue URL Documentation](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-queue-message-identifiers.html#sqs-queue-url). - **Choose (optional) message attributes** - Specify which message attributes should be included in the triggered event. - These attributes can carry metadata, such as sender information or priority levels. - For more details, see the [SQS Message Attributes Documentation](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-message-metadata.html#sqs-message-attributes). ## Example Below are code examples demonstrating how to handle SQS messages in your Windmill scripts. You can either process messages directly in a basic script or use a preprocessor for more advanced message handling and transformation before execution. ### Basic script example ```TypeScript export async function main(msg: string) { // do something with the message } ``` ### Using a preprocessor If you use a [preprocessor](../../core_concepts/43_preprocessors/index.mdx), the preprocessor function receives an SQS message with the following fields: #### Field descriptions - **`queue_url`**: The URL of the SQS queue that received the message. - **`message_id`**: A unique identifier assigned to each message by SQS. - [More details](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_ReceiveMessage.html) - **`receipt_handle`**: A token used to delete the message after processing. - [More details](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-queue-message-identifiers.html) - **`attributes`**: Metadata attributes set by SQS, such as `SentTimestamp`. - [Full list of system attributes](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-message-metadata.html#sqs-message-system-attributes) - **`message_attributes`**: User-defined attributes that can be attached to the message. - `string_value`: The string representation of the attribute value. - `data_type`: The data type of the attribute (e.g., `"String"`, `"Number"`, `"Binary"`). - [More details on message attributes](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-message-metadata.html#sqs-message-attributes) ```TypeScript export async function preprocessor( event: { kind: "sqs", msg: string, queue_url: string, message_id?: string, receipt_handle?: string, attributes: Record, message_attributes?: Record } ) { if (event.kind !== "sqs") { throw new Error(`Expected a SQS event`); } // assuming the message is a JSON object const data = JSON.parse(event.msg); return { content: data.content, metadata: { sentAt: event.attributes.SentTimestamp, messageId: event.message_id } }; } export async function main(content: string, metadata: { sentAt: string, messageId: string }) { // Process transformed message data console.log(`Processing message ${metadata.messageId} sent at ${metadata.sentAt}`); console.log("Content:", content); } ``` ## Error handling SQS triggers support local error handlers that override workspace error handlers for specific triggers. See the [error handling documentation](../../core_concepts/10_error_handling/index.mdx#trigger-error-handlers) for configuration details and examples. --- ## Websocket triggers Source: https://www.windmill.dev/docs/triggers/websocket_triggers # WebSocket triggers Windmill can connect to [WebSocket](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API) servers and trigger runnables (scripts, flows) when a message is received. Listening is done from the servers, so it doesn't take up any workers. WebSocket triggers are not available on the [Cloud](/pricing) Free and Team plans. ## How to use Create a new trigger on the WebSocket triggers page. Specify the URL of the WebSocket server. Instead of a static URL, you can also specify a script or flow to return the connect URL. This is useful when you need to pass an authentication query parameter to the connect URL. The runnable must return a string. ![WebSocket URL and runnable](./static_url.png 'WebSocket static URL') ![WebSocket URL as runnable result](./runnable_url.png 'WebSocket URL as runnable result') Once the URL set, select the runnable that should be triggered by this trigger. The received WebSocket message will be passed to the runnable as a string argument called `msg`. Here's an example script: ```TypeScript export async function main(msg: string) { // do something with the message } ``` And if you use a [preprocessor](../../core_concepts/43_preprocessors/index.mdx), the script could look like this: ```TypeScript export async function preprocessor( event: { kind: "websocket", msg: string, url: string, } ) { if (event.kind !== "websocket") { throw new Error(`Expected a websocket event`); } // assuming the message is a JSON object const msg = JSON.parse(event.msg); // define args for the main function // let's assume we want to use the message content and the url return { message_content: msg.content, url: event.url }; } export async function main(message_content: string, url: string) { // do something with the message content and url } ``` The trigger also supports additional configuration options: ### Send runnable result to WebSocket server If you enable the "Send runnable result" toggle, the runnable result will be sent to the WebSocket server as a message, as long as the job is a success and the result is not null. Like for [sync webhooks](../../core_concepts/4_webhooks/index.mdx#synchronous), if the flow has the [early return](../../flows/19_early_return.mdx) setting set, the chosen node result will be sent and the rest of the flow will continue asynchronously. By default, only successful results are sent back. Enable the "Send result even on error" toggle to also forward error results to the WebSocket server, so the remote peer can react to failures (e.g. reply with an error frame). When disabled, failed jobs produce no outgoing message. ### Initial messages You can specify a list of initial messages to send to the WebSocket server when connection to the server is established. This is useful for authentication or subscription messages. They can be static strings or runnables that return the message. The static string field is in JSON format and will be stringified before sending. If the JSON value is a string, it will be sent without the wrapping quotes. The runnable can return a string or a JSON object, which will be stringified before sending. The messages are sent in the order they are specified. ![Initial messages](./initial_messages.png 'Initial messages') ### Filters Instead of having all messages trigger the runnable, you can specify filters to restrict which messages trigger the runnable. Windmill supports the following filter: - **JSON**: The message is parsed as a JSON object and the filter checks that the filter `key` exists and the value at the key is equal to or is a superset of the filter `value`. Keys are matched literally against the message's top-level fields (e.g. `type`); to match a nested field, use the parent key with an object value (e.g. key `data`, value `{"status": "active"}`). The runnable receives the message as the raw string argument `msg`. When multiple filters are configured, a filter logic selector lets you choose how they are combined: - **AND** (default): the message must match all filters to trigger the runnable. - **OR**: the message triggers the runnable as soon as it matches any one of the filters. Existing triggers without an explicit filter logic default to **AND**. The selector is only shown when at least one filter is present. ![Filters](./filters.png 'Filters') ### Application-level heartbeat Some WebSocket protocols (such as [Discord Gateway](https://discord.com/developers/docs/events/gateway), STOMP, or custom APIs) require the client to send periodic keep-alive messages at the application level to maintain the connection. Windmill supports this natively with the heartbeat configuration. When enabled, Windmill sends a configurable message at a fixed interval through the WebSocket connection. This happens at the Rust level with zero job overhead — no scripts are executed for heartbeats. #### Configuration - **Interval (seconds)**: How often to send the heartbeat message. - **Message**: The message to send. You can use the `{{state}}` placeholder to include a value extracted from incoming messages. - **State field** (optional): A top-level JSON field to extract from every incoming message. The extracted value replaces `{{state}}` in the heartbeat message. ![Heartbeat configuration](./heartbeat.png 'Heartbeat configuration for Discord Gateway') #### Examples **Static heartbeat** (STOMP, MQTT, simple APIs): | Field | Value | |-------|-------| | Interval | `10` | | Message | `{"type": "ping"}` | | State field | *(empty)* | **Stateful heartbeat** (Discord Gateway): Discord requires a heartbeat message that includes the last received sequence number (`s` field): | Field | Value | |-------|-------| | Interval | `41` | | Message | `{"op": 1, "d": {{state}}}` | | State field | `s` | Windmill extracts the `s` field from every incoming Discord event and substitutes it into the heartbeat message automatically. See the [Discord bot guide](/docs/misc/guides/discord_bot) for a complete walkthrough. ## Error handling WebSocket triggers support local error handlers that override workspace error handlers for specific triggers. See the [error handling documentation](../../core_concepts/10_error_handling/index.mdx#trigger-error-handlers) for configuration details and examples.