This article explains how to securely upload and download files directly from a client's browser to a private Amazon Simple Storage Service (S3) bucket via a CloudFront distribution.
This article is part of a series that explores ways to interact with Amazon S3 buckets. Be sure to read the introductory article to understand why S3 can be a valuable addition to your ODC applications and the challenges you might encounter.
💡
Along with this article, there is a demo application on ODC Forge that shows the pattern described, which you can use as a reference. Search for ODC with S3 Patterns.
AWS CloudFront
CloudFront is a content delivery network (CDN) service. CDNs are geographically distributed groups of servers that provide any Internet content efficiently and with low latency. This is achieved by ensuring that content in the CDN is always retrieved from the group of servers — edge servers — that are closest to a client (point of presence).
Content in a CDN can be videos, audio files, images, and documents, but also HTML, JavaScript, and CSS. Complete web presences can be stored and distributed on a CDN, which is particularly interesting for static websites, or for websites generated via a static site generator framework.
Content is not automatically distributed to all CloudFront Edge server groups. Instead, content is initially retrieved from a defined source (origin) — for example, an S3 bucket — and cached for a defined period. Subsequent requests are then served directly from the cache. This means that the first retrieval takes a little longer, as the content must first be retrieved from the source.
CloudFront can be used not only to retrieve objects but also to modify or create objects at the configured origin.
With CDN content, the first thing that comes to mind would be that content is publicly available. With CloudFront, it is possible to make private content securely available, which should only be accessible after authorization. One way to achieve this is by using signed URLs or cookies. The signature is generated from within your OutSystems application using a private key.
Differences between S3 pre-signed URLs and CloudFront Signed URLs
If you've read about the Browser to S3 pattern, you might wonder about the differences between S3 pre-signed URLs and CloudFront Signed URLs.
Both S3 pre-signed URLs and CloudFront Signed URLs provide access to private content. In both cases, a URL is signed, and the signature is added to the URL. When accessed, the signature is checked, and if it's valid, access to the resource is granted.
However, there are some key differences:
S3 pre-signed URLs are signed using IAM credentials. The permissions for accessing a resource are defined by the policies assigned to that IAM user or specified in a bucket policy.
CloudFront Signed URLs are signed using a self-generated private key. The corresponding public key is uploaded to CloudFront and used to verify signatures. Access to a resource is determined by the URL and the validity of the signature. If the signature is valid for a URL, access is granted regardless of other factors.
S3 pre-signed URLs are specific to one object in storage and are limited to a single request type (GET, POST, etc.). CloudFront Signed URLs can use wildcards, allowing access to multiple objects (e.g., all files under a path) using a custom signed URL. CloudFront Signed URLs apply to all allowed request types configured in the CloudFront distribution, which is important to note because you cannot limit it on a per-request basis.
In this pattern we are exploring CloudFront signed URLs for authorizing access to private S3 bucket objects.
💡
Note: CloudFront signed cookies are actually nicer to use. However, due to security restrictions, you can't set a secure cookie from your ODC application for the CloudFront distribution domain without setting a custom domain on your distribution. This setup is more complex and that's why we use signed URLs instead in this pattern. I may add another add-on pattern later.
Prerequisites
There are several steps required to set up the environment before you can try out the reference application.
💡
Please note that the steps described here are the basic configuration settings. Both S3 and CloudFront are extensive services with many configuration options, especially for securing content further. It's important to have a good understanding of both services before using them in a production environment.
S3 Bucket and AWS Credentials
Create an S3 bucket in a region of your choice. Unlike the other patterns in this series, you do not need AWS credentials for this one.
Apply a cross-origin policy in the Permissions tab of your bucket to allow HEAD, GET and PUT requests (you might want to restrict it further).
[
{
"AllowedHeaders": [
"*"
],
"AllowedMethods": [
"HEAD",
"GET",
"PUT"
],
"AllowedOrigins": [
"*"
],
"ExposeHeaders": []
}
]
CloudFront Signing Key Pair
As stated above we need a private key in our OutSystems application to sign URLs for accessing our private S3 files via CloudFront. CloudFront needs the public key to validate the signature.
We are using OpenSSL here to create a key pair. You can download a Microsoft Windows version from Shining Light Productions or use the OpenSSL command from a WSL installation of a Linux distribution.
Create private key
openssl genrsa -out aws_cloudfront_privatekey.pem 2048
Create public key
openssl rsa -pubout -in aws_cloudfront_privatekey.pem -out aws_cloudfront_publickey.pem
Add Public Key to CloudFront
Open the public key PEM file with a text editor and copy the content to your clipboard.
In the AWS console, switch to the CloudFront service.
In the left menu under Key Management, select Public Keys and click on Create Public Key.
Name the public key, paste the contents of the PEM file into the Key field, and then click Create Public Key.
Make a note of the ID of your created public key, as we will need it later.
Create a Key Group
Keys are organized into key groups, which are later referenced by CloudFront distributions. A signature is valid if any of the public keys in the associated key group can validate it.
Under Key Management, select Key Groups and click Create Key Group.
Name the key group and select your added public key from the list of available public keys. Then click on Create Key Group.
Custom Origin Request Policy
Origin Request policies control which values (e.g. query string parameters, headers) are forwarded from CloudFront to the origin, which in our case is a S3 bucket.
In the reference application when uploading files, we are going to set an additional header content-disposition which is not part of the default policies.
Select Include the following headers in Headers and select Origin, Access-Control-Request-Method and Access-Control-Request-Headers from the list. Add another custom header Content-Disposition.
Leave Query string and Cookies to None. Then Save the policy.
💡
If you want to send extra headers, like object metadata, you need to add the specific header here.
CloudFront Distribution
Now it's time to create the CloudFront distribution using our S3 bucket as the origin.
In the CloudFront menu, select Distributions and click Create distribution.
Choose your S3 bucket from the Origin domain list and provide a name for the new distribution.
Under Origin access, select Origin access control settings (recommended).
Click the Create control setting button and create a new setting with the default options
💡
This is the most important setting. If you do not restrict viewer access, all cached objects in CloudFront will be publicly available. When set to Yes, only requests with a valid signed URL or cookie will be served by your CloudFront distribution.
- Set the Origin request policy to the creates custom origin request policy. This policy ensures that the CORS request headers are forwarded to the S3 origin, allowing browser-based requests.
- Leave the rest of the settings as they are and click on Create distribution.
After creating your CloudFront distribution, note your Distribution domain name (e.g., gfds76768das.cloudfront.net) from the General tab. We will need this later to access files.
Try accessing the root of your CloudFront Distribution domain using a browser. You should see an error message like
<Error>
<Code>MissingKey</Code>
<Message>Missing Key-Pair-Id query parameter or cookie value</Message>
</Error>
CloudFront S3 Bucket Permissions
We need to grant CloudFront — our CloudFront distribution to be more specific — access to our S3 bucket. CloudFront, just like any other resource or user, needs permission to retrieve objects from the store to place them into its cache. We do this by applying a bucket policy in our S3 bucket. Luckily CloudFront provides us with a template bucket policy.
In your created CloudFront distribution select the Origins header. Then select your S3 origin from the list and click on Edit.
Click the Copy Policy button to copy the pre-created bucket policy to clipboard.
Switch to S3 service and click on your S3 bucket from the list of buckets.
Select the Permissions tab
Under Bucket Polices click Edit, then paste the copied policy. Then add the PutObject action permission manually.
💡
By default, the copied policy only allows retrieving objects using GetObject. You need to slightly modify the policy before saving it to also include the PutObject permission to allow uploads as well.
Your bucket policy should look like this:
{
"Version": "2008-10-17",
"Id": "PolicyForCloudFrontPrivateContent",
"Statement": [
{
"Sid": "AllowCloudFrontServicePrincipal",
"Effect": "Allow",
"Principal": {
"Service": "cloudfront.amazonaws.com"
},
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::<your S3 Bucket name>/*",
"Condition": {
"StringEquals": {
"AWS:SourceArn": "<arn of your CloudFront distribution>"
}
}
}
]
}
The final step is to apply some settings in the reference application.
To try out the reference application, you will need the following information from the configuration steps you did above.
Distribution domain name — Found on the AWS CloudFront console under the Distribution detail screen.
CloudFront Key ID — The ID of the public key (not the key group) you uploaded, located in the CloudFront console under the Key Management menu in Public Keys.
Private Key — The PEM file of your private key that matches the public key you uploaded to CloudFront.
Configure the app settings of the reference application in the ODC portal.
Enter the values for CloudFrontDomainName (without https://), CloudFrontKeyId, and CloudFrontPrivateKey. For the last one, use the text content of the private key file. Ensure no extra spaces or lines are included.
💡
Please note that the
CloudFrontPrivateKey setting is not considered a secret. This is because secrets have size limitations and cannot store a private key. To protect larger secure items like a private key, you might consider using envelope encryption, as I describe in my article “
Application Level Encryption in OutSystems”.
Before we begin exploring the reference application, I want to briefly explain the two different types of CloudFront Signed URLs.
CloudFront Signed URL Types
CloudFront allows the use of two types of signed URLs to access a private distribution:
Canned Policy Signed URLs - These signed URLs are used for a single resource, whether it already exists or is new.
Custom Policy Signed URLs - These provide more flexibility and can target a group of resources or an entire bucket, using wildcards in their resource description.
As mentioned above, both types of signed URLs allow access to resources, permitting all request types configured in the CloudFront distribution. With CloudFront signed URLs, it is not possible to restrict the request type individually.
The reference application uses both types of signed URLs. The Canned Policy Signed URL is used for uploading an object, and the Custom Policy Signed URL is used for listing objects.
Upload
Let's start with uploading files through our CloudFront distribution to the S3 bucket origin.
💡
If you followed the Browser to S3 pattern, you'll notice that the steps are the same. The only difference is that instead of a pre-signed S3 URL, a CloudFront Canned Policy Signed URL is generated.
Uploading a file from the frontend application requires some JavaScript coding. The reference application includes a complete example that we will use to walk through the process, which is:
Generate a pre-signed URL for a new object and a PUT request in the backend using the AWSSimpleStorage external logic connector.
Use the pre-signed URL in the frontend to perform the upload using JavaScript's XMLHttpRequest class.
💡
Alternatively, you can use a simple fetch PUT operation. However, using XMLHttpRequest raises Progress events that let you create an upload progress bar, as shown in the reference application.
Open the reference application in ODC Studio. On the Interface tab select the AWSCloudFrontUpload block.
Open the AWSCloudFrontUpload widget tree. Inside you will find a HTML input element of type file named UploadElement.
Besides that you will also find a progress element which is used to display the upload progress.
The widgets CSS uses some design tokens (CSS variables) which can be overridden in the screen or in your application theme. You will also note that the HTML input element is hidden and that all styles apply to the wrapping label element.
OnReady and OnDestroy Event Handlers
In the OnReady event handler there is a single JavaScript element registering a change event with the OnFileUploadChange client action.
const uploadElement = document.getElementById($parameters.UploadFileWidgetId);
uploadElement.addEventListener('change', $actions.OnFileUploadChange);
console.debug('Change listener added');
Likewise, in the OnDestroy event handler the change listener is removed.
const fileElement = document.getElementById($parameters.UploadFileWidgetId);
fileElement.removeEventListener('change', $actions.OnFileUploadChange);
console.debug('Change listener removed');
OnFileUploadChange Event Handler
This event handler is executed when the user selects a file to upload. The Event object contains information about the file selected.
💡
The change event handler also runs when the user clicks Cancel.
The JavaScript inside the event handler first checks if a file was selected. If not the script returns.
It then calls another Client Action GetCloudFrontPutUrl — this one is located outside of the widget in the Logic tab — to get a pre-signed URL for a given prefix (a folder) and the filename of the selected file. Given the pre-signed URL it executes the widgets UploadFile client action.
if($parameters.Event.target.files.length == 0) {
return;
}
$actions.GetCloudFrontPutUrl($parameters.Prefix, $parameters.Event.target.files[0].name)
.then((result => {
$actions.UploadFile(
result.PreSignedUrl,
$parameters.Event.target.files[0].name,
$parameters.Event.target.files[0].type,
$parameters.Event.target.files[0].size,
$parameters.Event.target.files[0]
);
}));
GetCloudFrontPutUrl Client Action
In the Logic tab you will find the GetCloudFrontPutUrl client action. It executes the server action CloudFront_GetUploadUrl located directly under Server Actions and returns a generated canned policy signed URL.
The CloudFront_GetUploadUrl uses the GetCannedSignedUrl action from the AWSCloudFrontSigner forge component using the private key and key pair id you configured above.
This creates a canned signed policy URL valid for 1 hour.
UploadFile Client Action
The UploadFile client action first sets values for the block’s CurrentFile local variable. This structure contains information about the file and is used later to track the upload progress.
The JavaScript element creates a new XMLHttpRequest instance for a PUT request to the given pre-signed URL.
let xhr = new XMLHttpRequest();
xhr.open('PUT',$parameters.Url,true);
xhr.setRequestHeader("content-disposition", `attachment; filename="${$parameters.FileName}"`);
xhr.upload.onprogress = (evt) => $actions.OnTransferProgress(evt.loaded);
xhr.upload.onloadstart = (evt) => $actions.OnTransferState('start');
xhr.upload.onload = (evt) => $actions.OnTransferState('success');
xhr.upload.onloadend = (evt) => $actions.OnTransferEnd();
xhr.upload.onerror = (evt) => $actions.OnTransferState('error');
xhr.upload.ontimeout = (evt) => $actions.OnTransferState('timeout');
xhr.upload.onabort = (evt) => $actions.OnTransferState('abort');
xhr.send($parameters.File)
💡
Please note the extra request header, content-disposition. We will discuss this additional header when downloading objects. This header is stored with your object as extra metadata.
OnTransferState Event Handler
This client action runs whenever an upload has:
encountered an error
been aborted
timed out
succeeded
It then updates the State variable in the CurrentFile structure accordingly.
OnTransferEnd Event Handler
After the upload is finished—whether successful or not—this client action is executed. It triggers the OnFileUploaded event with the uploaded file details as the payload and resets the CurrentFile local variable.
Saving a File Record on Upload Completed
The ClientCloudFront screen handles the OnFileUploaded event from the widget and runs the CloudFront_AddItem server action, which creates a new record in the File entity for the uploaded file.
Summary
Uploading an object from the browser via CloudFront to a S3 bucket involves two steps. First, create a canned signed policy URL to a new object. Then, use JavaScript to execute the PUT request with the object's binary data.
The reference implementation is a simple example using XMLHttpRequest. There are many more advanced JavaScript file upload libraries available that you might want to consider, especially for uploading very large files.
Download
When downloading an object or using it as a resource, we follow a two-step process. First, we generate a wildcarded custom policy signed URL and append the generated signature to our requested resource.
When uploading a file using the method described above, we include an extra header called content-disposition with the value attachment; filename="<filename>". This header and its value are stored as metadata attached to our S3 object.
When we request that object with a GET request, the header is sent from S3 to our browser, instructing the browser to treat the content as a downloadable file rather than displaying it inline.
On the ClientCloudFront screen in the Interface tab double-click the GetCloudFrontFiles data action.
GetCloudFrontFiles Data Action
- First, we generate a wildcard custom policy signed URL using the server action GenerateWildcardUrl.
This action uses the GetCustomSignedUrl action from the AWSCloudFrontSigner connector library to create a signed URL that allows access to all objects in the bucket (note the “*” in ResourcePath). It returns the query string of the generated URL, which we can add to our request URL.
- Then we query the File entity for available File records, iterate over the results, and construct a full URL.
Streaming
The download URL can be used not only for downloading objects but also as a source for media streaming, such as in a video or audio player component. Unlike pre-signed S3 URLs, CloudFront Signed URLs provide full streaming support. With a single signed URL, multiple request types are possible, whereas S3 URLs allow only one request type. Media players and document viewers typically first send a HEAD request to check if the resource supports byte-range requests, then stream the resource using a GET request.
Summary
Uploading and downloading files to and from an S3 bucket via CloudFront requires a detailed setup of AWS services, but once completed, the process is straightforward. This method avoids some challenges mentioned in the introductory article and offers additional benefits:
We don't use external logic connector actions for S3 GetObject and PutObject, which have a request or response payload limit of 5.5MB.
We bypass the 28MB request limit from the browser to an ODC application by uploading directly to S3, allowing us to upload even larger files.
The maximum application request timeout is not an issue because we aren't using a server or server action to perform uploads or downloads.
Objects we want to store or retrieve are not passed through our application container, so the container's memory isn't filled with binary data.
Provides full streaming support for all types of media.
Offers regional distribution of content, making large objects quickly available in multiple regions.
However, the described approach has some downsides as well:
This client-side method doesn't work when we need an object stored in S3 for an asynchronous process, like an in-app event handler or a workflow.
The reference application relies on saving an entry to the database after the OnFileUploaded client event is triggered, which can fail. In a production environment, additional steps are necessary to ensure that File entity records in the database match all objects stored in the bucket. S3 Object events, combined with EventBridge and HttpEndpoint targets (webhooks), are a good way to achieve this, but they are beyond the scope of this article and reference application.
I hope you enjoyed reading this article and that I explained the important parts clearly. If not, please let me know by leaving a comment. I invite you to read the other articles in the series about different patterns for storing and retrieving S3 objects.
