Kestra benchmarks

Benchmark conclusions

Conclusions for each benchmark run for all engines in all languages and settings

Summary

Kestra delivered middle-of-the-pack performance — relatively stable for long-running tasks, but slower than competitors in lightweight workloads. While more modern than Airflow and Prefect, it falls short of the orchestration efficiency seen in Temporal, Hatchet, or Windmill.

Kestra setup

We set up Kestra version v0.22.3 using the docker-compose.yml from their official Documentation. We made some adjustments to it to have a similar setup compared to the other orchestrator.

The flow we used to run the benchmarks is the following:

Python

id: benchmark
namespace: company.team
inputs:
- id: n
type: INT
- id: iters
type: INT
tasks:
- id: processIterations
type: io.kestra.plugin.core.flow.ForEach
values: '{{ range(0, inputs.iters - 1) }}'
concurrencyLimit: 1
tasks:
  - id: python
    type: io.kestra.plugin.scripts.python.Script
    containerImage: python:slim
    taskRunner:
      type: io.kestra.plugin.core.runner.Process
    script: |

      def fibo(n: int):
        if n <= 1:
            return n
        else:
            return fibo(n - 1) + fibo(n - 2)

      print(str(fibo({{ inputs.n }})))

JavaScript

id: benchmark_js
namespace: company.team
inputs:
- id: n
type: INT
- id: iters
type: INT
tasks:
- id: processIterations
type: io.kestra.plugin.core.flow.ForEach
values: '{{ range(0, inputs.iters - 1) }}'
concurrencyLimit: 1
tasks:
  - id: script
    type: io.kestra.plugin.scripts.node.Script
    taskRunner:
      type: io.kestra.plugin.core.runner.Process
    script: |
        function fibo(n) {
          if (n <= 1) {
            return n
          } else {
            return fibo(n-1) + fibo(n-2)
          }
        }
        console.log(fibo({{ inputs.n }}))

We executed it once with n=33, iters=10 and once with n=10 and iters=40. Note that we set the concurrency limit to 1 meaning all task will run sequentially on one worker. Furthermore, no extra python dependencies had to be installed during the execution of those flows, , and we use a Process task runner to avoid starting a Docker container for each task execution.

Fibonacci 40 iterations, n=10

This benchmark measures the performance of Kestra when handling a large number of lightweight tasks. The tasks are Fibonacci calculations with 40 iterations and a base value of 10.

Python

Visualization

Statistics

	Kestra	Windmill	Windmill Dedicated
Total duration (in seconds)	6.044	4.383	2.092
Assignment	2.679 (44.32%)	1.836 (41.89%)	1.795 (85.80%)
Execution	1.499 (24.80%)	2.215 (50.54%)	0.122 (5.83%)
Transition	1.866 (30.87%)	0.332 (7.57%)	0.175 (8.37%)

Timing details

View task timing details

Task	Created at	Started at	Completed at
task_00	0.000	0.931	0.966
task_01	1.007	1.037	1.073
task_02	1.108	1.142	1.175
task_03	1.220	1.246	1.286
task_04	1.316	1.376	1.415
task_05	1.437	1.485	1.523
task_06	1.575	1.617	1.653
task_07	1.674	1.697	1.735
task_08	1.793	1.853	1.891
task_09	1.924	1.984	2.018
task_10	2.046	2.064	2.105
task_11	2.155	2.220	2.258
task_12	2.304	2.353	2.390
task_13	2.450	2.510	2.550
task_14	2.589	2.620	2.664
task_15	2.702	2.754	2.789
task_16	2.840	2.886	2.922
task_17	2.978	3.017	3.053
task_18	3.104	3.149	3.184
task_19	3.259	3.307	3.344
task_20	3.403	3.438	3.473
task_21	3.548	3.596	3.634
task_22	3.662	3.702	3.737
task_23	3.790	3.835	3.871
task_24	3.937	3.992	4.033
task_25	4.102	4.152	4.190
task_26	4.244	4.310	4.345
task_27	4.412	4.469	4.505
task_28	4.589	4.654	4.692
task_29	4.752	4.811	4.844
task_30	4.915	4.942	4.986
task_31	5.048	5.074	5.110
task_32	5.137	5.180	5.217
task_33	5.270	5.311	5.349
task_34	5.381	5.418	5.459
task_35	5.490	5.525	5.566
task_36	5.608	5.658	5.696
task_37	5.730	5.765	5.804
task_38	5.843	5.900	5.936
task_39	5.965	6.006	6.044

Windmill Comparison

Task	Created at	Started at	Completed at
task_00	0.000	0.003	0.059
task_01	0.067	0.113	0.171
task_02	0.180	0.226	0.280
task_03	0.290	0.335	0.389
task_04	0.398	0.446	0.501
task_05	0.510	0.558	0.614
task_06	0.622	0.669	0.725
task_07	0.732	0.780	0.834
task_08	0.842	0.889	0.942
task_09	0.950	0.997	1.052
task_10	1.061	1.108	1.166
task_11	1.175	1.220	1.274
task_12	1.283	1.330	1.385
task_13	1.394	1.440	1.494
task_14	1.503	1.550	1.605
task_15	1.612	1.661	1.716
task_16	1.723	1.770	1.823
task_17	1.831	1.878	1.930
task_18	1.939	1.986	2.041
task_19	2.049	2.096	2.152
task_20	2.161	2.209	2.266
task_21	2.274	2.320	2.376
task_22	2.384	2.431	2.486
task_23	2.495	2.542	2.596
task_24	2.604	2.652	2.706
task_25	2.715	2.761	2.816
task_26	2.825	2.872	2.925
task_27	2.933	2.979	3.033
task_28	3.042	3.090	3.145
task_29	3.154	3.201	3.269
task_30	3.278	3.325	3.382
task_31	3.391	3.437	3.493
task_32	3.501	3.548	3.602
task_33	3.611	3.660	3.715
task_34	3.723	3.770	3.823
task_35	3.833	3.879	3.934
task_36	3.942	3.990	4.045
task_37	4.053	4.101	4.157
task_38	4.165	4.212	4.268
task_39	4.277	4.324	4.383

Windmill Dedicated Comparison

Task	Created at	Started at	Completed at
task_00	0.000	0.019	0.022
task_01	0.029	0.073	0.077
task_02	0.081	0.125	0.128
task_03	0.134	0.179	0.182
task_04	0.187	0.231	0.234
task_05	0.239	0.284	0.287
task_06	0.292	0.338	0.341
task_07	0.345	0.391	0.394
task_08	0.398	0.444	0.447
task_09	0.451	0.497	0.500
task_10	0.505	0.549	0.552
task_11	0.557	0.603	0.606
task_12	0.610	0.655	0.659
task_13	0.663	0.709	0.712
task_14	0.716	0.761	0.764
task_15	0.768	0.814	0.817
task_16	0.821	0.867	0.870
task_17	0.876	0.921	0.924
task_18	0.929	0.973	0.976
task_19	0.981	1.027	1.030
task_20	1.035	1.080	1.083
task_21	1.087	1.132	1.135
task_22	1.139	1.186	1.189
task_23	1.193	1.238	1.241
task_24	1.246	1.292	1.295
task_25	1.299	1.345	1.348
task_26	1.352	1.398	1.401
task_27	1.405	1.451	1.454
task_28	1.458	1.504	1.507
task_29	1.512	1.557	1.560
task_30	1.564	1.611	1.614
task_31	1.618	1.664	1.667
task_32	1.671	1.717	1.720
task_33	1.724	1.770	1.773
task_34	1.777	1.823	1.826
task_35	1.830	1.876	1.879
task_36	1.884	1.930	1.933
task_37	1.937	1.983	1.986
task_38	1.991	2.036	2.039
task_39	2.043	2.089	2.092

Javascript

Visualization

Statistics

	Kestra	Windmill	Windmill Dedicated
Total duration (in seconds)	9.050	2.973	2.125
Assignment	2.777 (30.69%)	1.427 (48.00%)	1.786 (84.05%)
Execution	4.633 (51.19%)	0.289 (9.72%)	0.126 (5.93%)
Transition	1.640 (18.12%)	1.257 (42.28%)	0.213 (10.02%)

Timing details

View task timing details

Task	Created at	Started at	Completed at
task_00	0.000	0.932	1.042
task_01	1.082	1.139	1.248
task_02	1.307	1.346	1.458
task_03	1.504	1.553	1.667
task_04	1.698	1.735	1.846
task_05	1.875	1.917	2.026
task_06	2.065	2.124	2.237
task_07	2.260	2.281	2.394
task_08	2.414	2.463	2.580
task_09	2.607	2.671	2.789
task_10	2.817	2.853	2.968
task_11	3.006	3.035	3.145
task_12	3.183	3.243	3.358
task_13	3.407	3.450	3.563
task_14	3.601	3.632	3.745
task_15	3.785	3.816	3.928
task_16	3.980	4.024	4.144
task_17	4.191	4.258	4.371
task_18	4.421	4.468	4.576
task_19	4.616	4.650	4.765
task_20	4.813	4.860	4.970
task_21	5.000	5.068	5.219
task_22	5.259	5.305	5.438
task_23	5.476	5.542	5.658
task_24	5.693	5.751	5.862
task_25	5.897	5.933	6.047
task_26	6.102	6.141	6.253
task_27	6.286	6.324	6.439
task_28	6.476	6.507	6.630
task_29	6.670	6.715	6.836
task_30	6.893	6.950	7.061
task_31	7.105	7.163	7.280
task_32	7.314	7.371	7.496
task_33	7.546	7.608	7.723
task_34	7.785	7.842	7.960
task_35	8.027	8.077	8.195
task_36	8.244	8.285	8.401
task_37	8.440	8.493	8.609
task_38	8.676	8.727	8.843
task_39	8.889	8.935	9.050

Windmill Comparison

Task	Created at	Started at	Completed at
task_00	0.000	0.006	0.014
task_01	0.051	0.074	0.081
task_02	0.105	0.155	0.163
task_03	0.184	0.222	0.229
task_04	0.239	0.286	0.294
task_05	0.330	0.364	0.372
task_06	0.411	0.447	0.455
task_07	0.493	0.528	0.536
task_08	0.573	0.610	0.617
task_09	0.655	0.689	0.693
task_10	0.705	0.751	0.759
task_11	0.797	0.832	0.840
task_12	0.876	0.911	0.918
task_13	0.957	0.989	0.993
task_14	1.004	1.051	1.058
task_15	1.098	1.133	1.141
task_16	1.167	1.214	1.222
task_17	1.247	1.283	1.291
task_18	1.313	1.351	1.358
task_19	1.394	1.416	1.424
task_20	1.469	1.480	1.487
task_21	1.513	1.560	1.568
task_22	1.580	1.625	1.632
task_23	1.669	1.704	1.712
task_24	1.751	1.787	1.791
task_25	1.801	1.848	1.855
task_26	1.894	1.929	1.936
task_27	1.976	2.012	2.019
task_28	2.057	2.092	2.099
task_29	2.138	2.174	2.182
task_30	2.222	2.256	2.263
task_31	2.304	2.336	2.344
task_32	2.386	2.420	2.428
task_33	2.467	2.503	2.511
task_34	2.548	2.583	2.591
task_35	2.629	2.666	2.673
task_36	2.712	2.746	2.754
task_37	2.796	2.825	2.832
task_38	2.869	2.903	2.908
task_39	2.920	2.966	2.973

Windmill Dedicated Comparison

Task	Created at	Started at	Completed at
task_00	0.022	0.053	0.057
task_01	0.061	0.105	0.109
task_02	0.113	0.158	0.161
task_03	0.166	0.212	0.215
task_04	0.221	0.266	0.270
task_05	0.275	0.318	0.321
task_06	0.326	0.372	0.375
task_07	0.380	0.424	0.427
task_08	0.433	0.477	0.480
task_09	0.485	0.531	0.534
task_10	0.538	0.584	0.588
task_11	0.592	0.637	0.640
task_12	0.645	0.690	0.693
task_13	0.698	0.743	0.746
task_14	0.751	0.796	0.799
task_15	0.804	0.849	0.852
task_16	0.857	0.902	0.905
task_17	0.910	0.956	0.959
task_18	0.964	1.009	1.012
task_19	1.016	1.061	1.064
task_20	1.069	1.113	1.116
task_21	1.122	1.166	1.169
task_22	1.174	1.220	1.223
task_23	1.228	1.273	1.276
task_24	1.281	1.325	1.328
task_25	1.332	1.378	1.381
task_26	1.385	1.430	1.433
task_27	1.439	1.484	1.487
task_28	1.492	1.537	1.540
task_29	1.544	1.589	1.593
task_30	1.597	1.643	1.646
task_31	1.650	1.695	1.698
task_32	1.704	1.747	1.750
task_33	1.756	1.800	1.804
task_34	1.809	1.853	1.856
task_35	1.861	1.906	1.909
task_36	1.914	1.960	1.963
task_37	1.967	2.014	2.017
task_38	2.023	2.068	2.071
task_39	2.076	2.122	2.125

Fibonacci 10 iterations, n=33

This benchmark measures the performance of Kestra when handling a large number of lightweight tasks. The tasks are Fibonacci calculations with 10 iterations and a base value of 33.

Python

Visualization

Statistics

	Kestra	Windmill	Windmill Dedicated
Total duration (in seconds)	15.786	8.347	7.701
Assignment	1.365 (8.65%)	0.428 (5.13%)	0.370 (4.80%)
Execution	14.029 (88.87%)	7.832 (93.83%)	7.205 (93.56%)
Transition	0.392 (2.48%)	0.087 (1.04%)	0.126 (1.64%)

Timing details

View task timing details

Task	Created at	Started at	Completed at
task_00	0.000	0.849	2.279
task_01	2.324	2.373	3.764
task_02	3.826	3.893	5.293
task_03	5.330	5.383	6.775
task_04	6.826	6.874	8.253
task_05	8.295	8.358	9.759
task_06	9.803	9.870	11.275
task_07	11.323	11.379	12.822
task_08	12.844	12.911	14.303
task_09	14.344	14.390	15.786

Windmill Comparison

Task	Created at	Started at	Completed at
task_00	0.000	0.002	0.846
task_01	0.858	0.906	1.705
task_02	1.715	1.761	2.539
task_03	2.548	2.595	3.365
task_04	3.375	3.421	4.206
task_05	4.215	4.263	5.033
task_06	5.042	5.089	5.857
task_07	5.866	5.913	6.684
task_08	6.693	6.740	7.519
task_09	7.529	7.579	8.347

Windmill Dedicated Comparison

Task	Created at	Started at	Completed at
task_00	0.000	0.023	0.745
task_01	0.776	0.797	1.518
task_02	1.546	1.571	2.292
task_03	2.298	2.340	3.057
task_04	3.063	3.114	3.845
task_05	3.874	3.889	4.608
task_06	4.614	4.661	5.380
task_07	5.385	5.433	6.151
task_08	6.158	6.208	6.925
task_09	6.933	6.981	7.701

Javascript

Visualization

Statistics

	Kestra	Windmill	Windmill Dedicated
Total duration (in seconds)	2.919	0.935	1.077
Assignment	0.778 (26.65%)	0.054 (5.78%)	0.239 (22.19%)
Execution	1.780 (60.98%)	0.773 (82.67%)	0.765 (71.03%)
Transition	0.361 (12.37%)	0.108 (11.55%)	0.073 (6.78%)

Timing details

View task timing details

Task	Created at	Started at	Completed at
task_00	0.000	0.281	0.462
task_01	0.516	0.565	0.742
task_02	0.805	0.850	1.027
task_03	1.043	1.083	1.258
task_04	1.301	1.342	1.523
task_05	1.561	1.627	1.804
task_06	1.848	1.911	2.090
task_07	2.134	2.197	2.375
task_08	2.401	2.481	2.658
task_09	2.691	2.741	2.919

Windmill Comparison

Task	Created at	Started at	Completed at
task_00	0.023	0.026	0.106
task_01	0.115	0.118	0.202
task_02	0.211	0.218	0.298
task_03	0.308	0.315	0.393
task_04	0.403	0.411	0.487
task_05	0.496	0.503	0.578
task_06	0.589	0.596	0.671
task_07	0.680	0.682	0.760
task_08	0.769	0.771	0.845
task_09	0.854	0.862	0.935

Windmill Dedicated Comparison

Task	Created at	Started at	Completed at
task_00	0.022	0.056	0.134
task_01	0.140	0.163	0.241
task_02	0.246	0.268	0.346
task_03	0.352	0.373	0.449
task_04	0.456	0.478	0.554
task_05	0.560	0.582	0.659
task_06	0.664	0.688	0.764
task_07	0.769	0.793	0.869
task_08	0.874	0.897	0.972
task_09	0.978	1.002	1.077