Skip to main content

Amazon S3 Integration


Windmill for data pipelines

You can link a Windmill workspace to an S3 bucket and use it as source and/or target of your processing steps seamlessly, without any boilerplate.


See Windmill for data pipelines for more details.

Self Host

If you're looking for a way to self-host Windmill using AWS, see Self-Host Windmill.

Amazon S3's API follows the same schema as any S3 compatible API.

To integrate Amazon S3 to Windmill, you need to save the following elements as a resource.

PropertyTypeDescriptionDefaultRequiredWhere to FindAdditional Details
bucketstringS3 bucket nametrueAWS Management ConsoleName of the S3 bucket to access
regionstringAWS region for the S3 buckettrueAWS Management ConsoleRegion where the S3 bucket is located. Can also be found by checking the endpoint URL for the bucket. In the form eu-west-3
useSSLbooleanUse SSL for connectionstruefalseAWS Management Console or Custom S3-compatible APISSL/TLS is required for Amazon S3
endPointstringS3 endpointtrueAWS Management Console or Custom S3-compatible APIEndpoint URL will vary by region or custom API provider. Can be found in the AWS documentation in the form s3.eu-west-3.amazonaws.com.
accessKeystringAWS access keytrue for AmazonAWS Management Console (IAM)Access key ID for AWS account owner. Can be found in the IAM section of the AWS Management Console under "My Security Credentials". Make sure the user has the right policies allocated.
pathStylebooleanUse path-style addressingfalsetrueAWS Management Console or Custom S3-compatible APIDefault is virtual-hosted-style URLs
secretKeystringAWS secret keytrue for AmazonAWS Management Console (IAM)Secret access key for AWS account owner. Can be found in the IAM section of the AWS Management Console under "My Security Credentials". Make sure the user has the right policies allocated.


Your resource can be used passed as parameters or directly fetched within scripts, flows and apps.


tip

Find some pre-set interactions with S3 on the Hub.

Feel free to create your own S3 scripts on Windmill.